Bug 8486 - Failed to do_patch for linux-yocto after add a patch by yocto-kernel tool
Summary: Failed to do_patch for linux-yocto after add a patch by yocto-kernel tool
Status: VERIFIED FIXED
Alias: None
Product: Kernel
Classification: Yocto Project Subprojects
Component: linux-yocto (show other bugs)
Version: 2.0
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.0.1
Assignee: Leonardo Sandoval Gonzalez
QA Contact: Yi Zhao
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-10-12 09:34 UTC by Yi Zhao
Modified: 2016-04-20 00:24 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Used to work)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
yocto-testmod.patch (2.29 KB, patch)
2015-10-12 09:34 UTC, Yi Zhao
no flags Details | Diff
log.do_patch (6.00 KB, application/octet-stream)
2015-10-12 09:34 UTC, Yi Zhao
no flags Details
kern-tools modifications (partial fix). (1.57 KB, patch)
2015-10-14 17:22 UTC, Bruce Ashfield
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yi Zhao 2015-10-12 09:34:11 UTC
Created attachment 2791 [details]
yocto-testmod.patch

Git rev: jethro/eac61f37e36099f74485dab398b57f3812826d17

This is a regression bug, used to work in 3.19 kernel.

Steps:
1. Follow the https://wiki.yoctoproject.org/wiki/Transcript:_Using_the_Yocto_BSP_tools_to_create_a_qemu_BSP to create a myqemuarm layer:

##############################
$ yocto-bsp create myqemuarm qemu
Checking basic git connectivity...
Done.

Which qemu architecture would you like to use? [default: i386]
	1) i386    (32-bit)
	2) x86_64  (64-bit)
	3) ARM     (32-bit)
	4) PowerPC (32-bit)
	5) MIPS    (32-bit)
	6) MIPS64  (64-bit)
3
Would you like to use the default (4.1) kernel? (y/n) [default: y] 
Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y] 
Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-4.1.git...
Please choose a machine branch to base your new BSP branch on: [default: standard/base]
	1) standard/arm-versatile-926ejs
	2) standard/base
	3) standard/beagleboard
	4) standard/beaglebone
	5) standard/edgerouter
	6) standard/fsl-mpc8315e-rdb
	7) standard/mti-malta32
	8) standard/mti-malta64
	9) standard/qemuarm64
	10) standard/qemuppc
1
Would you like SMP support? (y/n) [default: y] 
Does your BSP have a touchscreen? (y/n) [default: n] 
Does your BSP have a keyboard? (y/n) [default: y] 

New qemu BSP created in meta-myqemuarm
################################

2. Follow the https://wiki.yoctoproject.org/wiki/Transcript:_Using_the_Yocto_BSP_tools_to_manage_kernel_patches_and_config_items to add a patch (see the attachment):

#######################
$ yocto-kernel patch add myqemuarm ./yocto-testmod.patch 
Added patches:
	yocto-testmod.patch
[build@pek-usp-7 build]$ yocto-kernel config add myqemuarm CONFIG_MISC_DEVICES=y
Added item:
	CONFIG_MISC_DEVICES=y
[build@pek-usp-7 build]$ yocto-kernel config add myqemuarm CONFIG_YOCTO_TESTMOD=y
Added item:
	CONFIG_YOCTO_TESTMOD=y
#######################

3. Run bitbake linux-yocto

Error message:
#################
| (1/1) yocto-testmod.patch
| [INFO]: check of .kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/links/files/yocto-testmod.patch with "git am" did not pass, trying reduced context.
| [INFO]: Context reduced git-am of .kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/links/files/yocto-testmod.patch with "git am" did not work, trying "apply".
|    mark --> patch_marker.scc
|    mark <-- patch_marker.scc
|    mark --> myqemuarm.scc
|    mark --> myqemuarm-user-patches.scc
|          patch yocto-testmod.patch (.kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/series)
|    mark <-- myqemuarm-user-patches.scc
|    mark <-- myqemuarm.scc
| (1/1) yocto-testmod.patch
| [INFO]: check of .kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/links/files/yocto-testmod.patch with "git am" did not pass, trying reduced context.
| [INFO]: Context reduced git-am of .kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/links/files/yocto-testmod.patch with "git am" did not work, trying "apply".
| Context reduced to (1/1) to apply fragment at 379
| error: patch failed: drivers/misc/Makefile:36
| error: drivers/misc/Makefile: patch does not apply
| error: drivers/misc/yocto-testmod.c: already exists in index
| [ERROR]: Application of .kernel-meta/patches/standard/arm-versatile-926ejs/myqemuarm/links/files/yocto-testmod.patch failed.
|          Patch needs to be refreshed. Sample resolution script:
|              .git/rebase-apply/resolve_rejects
| ERROR. could not update git tree
| WARNING: /buildarea/poky/build/tmp/work/myqemuarm-poky-linux-gnueabi/linux-yocto/4.1.8+gitAUTOINC+3d8f1378d0_a8abc111a9-r0.4/temp/run.do_patch.22535:1 exit 1 from
|   patchme myqemuarm
| ERROR: Function failed: do_patch (log file is located at /buildarea/poky/build/tmp/work/myqemuarm-poky-linux-gnueabi/linux-yocto/4.1.8+gitAUTOINC+3d8f1378d0_a8abc111a9-r0.4/temp/log.do_patch.22535)
#####################

I can reproduced this issue on Ubuntu 14.04 and Fedora 22
Comment 1 Yi Zhao 2015-10-12 09:34:45 UTC
Created attachment 2792 [details]
log.do_patch
Comment 2 Bruce Ashfield 2015-10-13 01:03:07 UTC
I'll have a look.
Comment 3 Bruce Ashfield 2015-10-14 17:21:07 UTC
This was incorrectly working before, due to some 'features' in the
auto-resume logic for patch application.

I have features in quotes .. since in some situations, those features
are bugs and were hiding invalid configurations.

The reason this fails is that the same patch is in the series multiple
times, and with the fixed tools, those patches are always applied, not
detected and skipped.

If you look at the linux-yocto_4.1.bbappend in the layer, it has the
following:

SRC_URI += "file://myqemuarm-standard.scc \
            file://myqemuarm-user-config.cfg \
            file://myqemuarm-user-patches.scc \
            file://myqemuarm-user-features.scc \
           "

And the patch is part of myqemuarm-user-patches.scc. Anything listed
in the SRC_URI that contains patches is added to the end of the series
and applied.

If you look at myqemuarm-standard.scc, it has:

include myqemuarm.scc

And when you look at myqemuarm.scc:

include myqemuarm-user-patches.scc

So you now have that same .scc file, included twice, and hence the
application of the patch twice .. and the patch failure.

This is not something we should allow, so change in behaviour or not, the
tools can't be changed (again) to detect and skip this mis configuration.

Who currently maintains the BSP tool ? We should update it to not include
myqemuarm-user-patches.scc on the SRC_URI.

Once the yocto-bsp script is updated, the attached patch: 0001-kern-tools-avoid-duplicate-.scc-file-processing.patch, fixes the rest of the issue.
Comment 4 Bruce Ashfield 2015-10-14 17:22:24 UTC
Created attachment 2799 [details]
kern-tools modifications (partial fix).
Comment 5 Yi Zhao 2015-10-28 06:07:13 UTC
I found a relevant issue on 2.0 rc2:

If I create a myqemuarm bsp by using yocto-bsp script and don't add any custom patches. The image can be built but can not boot:
#############
VFS: Cannot open root device "vda" or unknown-block(0,0): error -6
Please append a correct "root=" boot option; here are the available partitions:
0100            4096 ram0  (driver?)
0101            4096 ram1  (driver?)
0102            4096 ram2  (driver?)
0103            4096 ram3  (driver?)
0104            4096 ram4  (driver?)
0105            4096 ram5  (driver?)
0106            4096 ram6  (driver?)
0107            4096 ram7  (driver?)
0108            4096 ram8  (driver?)
0109            4096 ram9  (driver?)
010a            4096 ram10  (driver?)
010b            4096 ram11  (driver?)
010c            4096 ram12  (driver?)
010d            4096 ram13  (driver?)
010e            4096 ram14  (driver?)
010f            4096 ram15  (driver?)
VFS: Unable to mount root fs on unknown-block(0,0)
User configuration error - no valid root filesystem found
Kernel panic - not syncing: Invalid configuration from end user prevents continuing
CPU: 0 PID: 1 Comm: swapper Not tainted 4.1.8-yocto-standard #1
Hardware name: ARM-Versatile PB
[<c0016c1c>] (unwind_backtrace) from [<c0013104>] (show_stack+0x20/0x24)
[<c0013104>] (show_stack) from [<c0634744>] (dump_stack+0x20/0x28)
[<c0634744>] (dump_stack) from [<c0630770>] (panic+0x94/0x1ec)
[<c0630770>] (panic) from [<c0897448>] (mount_block_root+0x22c/0x280)
[<c0897448>] (mount_block_root) from [<c089767c>] (mount_root+0xe8/0x110)
[<c089767c>] (mount_root) from [<c089780c>] (prepare_namespace+0x168/0x1c8)
[<c089780c>] (prepare_namespace) from [<c0896f4c>] (kernel_init_freeable+0x228/0x27c)
[<c0896f4c>] (kernel_init_freeable) from [<c062fec4>] (kernel_init+0x18/0xf4)
[<c062fec4>] (kernel_init) from [<c000f3c0>] (ret_from_fork+0x14/0x34)
---[ end Kernel panic - not syncing: Invalid configuration from end user prevents continuing
random: nonblocking pool is initialized
################### 


Steps:
1. Follow the https://wiki.yoctoproject.org/wiki/Transcript:_Using_the_Yocto_BSP_tools_to_create_a_qemu_BSP to create a myqemuarm layer:

##############################
$ yocto-bsp create myqemuarm qemu
Checking basic git connectivity...
Done.

Which qemu architecture would you like to use? [default: i386]
	1) i386    (32-bit)
	2) x86_64  (64-bit)
	3) ARM     (32-bit)
	4) PowerPC (32-bit)
	5) MIPS    (32-bit)
	6) MIPS64  (64-bit)
3
Would you like to use the default (4.1) kernel? (y/n) [default: y] 
Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y] 
Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-4.1.git...
Please choose a machine branch to base your new BSP branch on: [default: standard/base]
	1) standard/arm-versatile-926ejs
	2) standard/base
	3) standard/beagleboard
	4) standard/beaglebone
	5) standard/edgerouter
	6) standard/fsl-mpc8315e-rdb
	7) standard/mti-malta32
	8) standard/mti-malta64
	9) standard/qemuarm64
	10) standard/qemuppc
1
Would you like SMP support? (y/n) [default: y] 
Does your BSP have a touchscreen? (y/n) [default: n] 
Does your BSP have a keyboard? (y/n) [default: y] 

New qemu BSP created in meta-myqemuarm
################################

2. bitbake core-image-sato
Comment 6 Yi Zhao 2015-11-03 02:46:36 UTC
I re-tested it with latest jethro branch: jethro/e1aa897beb33171bc902207b98c8f666b9e97ad3

This issue still happens with same error.
Comment 7 Bruce Ashfield 2015-11-03 06:48:09 UTC
(In reply to comment #6)
> I re-tested it with latest jethro branch:
> jethro/e1aa897beb33171bc902207b98c8f666b9e97ad3
> 
> This issue still happens with same error.

As I mentioned. Someone needs to change the yocto bsp tool .. and that
isn't something that I've done in the past. Who ever is maintaining 
that needs to get this bug now. My part is done.
Comment 8 Leonardo Sandoval Gonzalez 2015-11-03 14:19:07 UTC
Bruce/Yi

Recently I worked on the yocto-bsp script. I take care of both issues seen so far, although the boot issue should be reported in a separate bug. I will file for the latter.
Comment 9 Leonardo Sandoval Gonzalez 2015-11-03 20:30:07 UTC
Patch sent to the mailing list. The boot problem was "moved" to [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=8635
Comment 10 Ke Zou 2016-01-13 07:55:15 UTC
manually tested with poky master:5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e

it still failed if you choose "re-use the existing branch for this BSP" during command "yocto-bsp create".
test procedure(create a BSP and add a kernel patch): https://wiki.yoctoproject.org/wiki/Transcript:_Using_the_Yocto_BSP_tools_to_create_a_qemu_BSP
https://wiki.yoctoproject.org/wiki/Transcript:_Using_the_Yocto_BSP_tools_to_manage_kernel_patches_and_config_items

I've tested on all qemu ARCHs,the results are the same-- the yocto-testmod.patch wasn't  actually added to the kernel.

On the other hand, if you choose "use a new machine branch for this BSP" during command "yocto-bsp create", which is its default option, the yocto-testmod.patch can go into the kernel.
Comment 11 Leonardo Sandoval Gonzalez 2016-02-02 18:34:30 UTC
Patch (embedded on a series) sent to the ML:

http://lists.openembedded.org/pipermail/openembedded-core/2016-February/116733.html

(Cherry picked from master f674ffa528b06e22d2c70c12f2e17cf308e26173)
Comment 12 Alexandru Georgescu 2016-02-04 09:58:53 UTC
seems that the issue still exists in YP 2.1_M2.rc1
master/3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
Comment 13 Leonardo Sandoval Gonzalez 2016-02-11 16:13:24 UTC
Patch merged into the Jethro branch: 

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=jethro&id=4e74b36458b36ee1202689388da5423ed9ee7b71
Comment 14 Yi Zhao 2016-02-16 09:06:11 UTC
Verified with Yocto 2.0.1 rc6 build.
git rev: jethro/5b12268f6e17574999f91628a60e21711cf62ee4

The do_path can work without errors. But a new issue happened as comment 10 described which found on both jethro and master. I simply guess it was introduced by above patches. I filed a new bug 9120 to report it.
Comment 15 Leonardo Sandoval Gonzalez 2016-02-16 16:46:45 UTC
(In reply to comment #14)
> Verified with Yocto 2.0.1 rc6 build.
> git rev: jethro/5b12268f6e17574999f91628a60e21711cf62ee4
> 
> The do_path can work without errors. But a new issue happened as comment 10
> described which found on both jethro and master. I simply guess it was
> introduced by above patches. I filed a new bug 9120 to report it.

Thanks Yi for looking into that. I will work on the new filed bug before 2.1
Comment 16 Leonardo Sandoval Gonzalez 2016-04-19 22:55:00 UTC
(In reply to comment #4)
> Created attachment 2799 [details]
> kern-tools modifications (partial fix).

Bruce, do you think this fix introduced the problem described on [1]?

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9120#c5
Comment 17 Bruce Ashfield 2016-04-19 23:33:42 UTC
(In reply to comment #16)
> (In reply to comment #4)
> > Created attachment 2799 [details]
> > kern-tools modifications (partial fix).
> 
> Bruce, do you think this fix introduced the problem described on [1]?
> 
> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9120#c5

It shouldn't have been caused by this. I'm lost in the two bugs.

What are the cut and paste yocto-bsp commands to reproduce the problem and I'll have a look at it right now.
Comment 18 Bruce Ashfield 2016-04-20 00:24:33 UTC
(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #4)
> > > Created attachment 2799 [details]
> > > kern-tools modifications (partial fix).
> > 
> > Bruce, do you think this fix introduced the problem described on [1]?
> > 
> > [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9120#c5
> 
> It shouldn't have been caused by this. I'm lost in the two bugs.
> 
> What are the cut and paste yocto-bsp commands to reproduce the problem and
> I'll have a look at it right now.

I found the steps. I'll have a look and update this bug with the results.