Bug 14725 - binutils-cross-canadian build failed
Summary: binutils-cross-canadian build failed
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: unspecified
Hardware: BeagleBone Multiple
: Medium+ normal
Target Milestone: 4.0 M4
Assignee: Sundeep KOKKONDA
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2022-02-18 12:51 UTC by Oleksiy Obitotskyy
Modified: 2022-03-07 03:45 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Oleksiy Obitotskyy 2022-02-18 12:51:39 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/4778/steps/15/logs/stdio
beaglebone ubuntu1604-ty-1


binutils-cross-canadian build failed

NOTE: recipe binutils-cross-canadian-arm-2.38-r0: task do_install: Started
ERROR: binutils-cross-canadian-arm-2.38-r0 do_install: oe_runmake failed
ERROR: binutils-cross-canadian-arm-2.38-r0 do_install: ExecutionError('/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/temp/run.do_install.8274', 1, None, None)
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/temp/log.do_install.8274
Log data follows:ERROR:
...
| /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/hosttools/install: cannot change permissions of '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info/as.info': No such file or directory
Comment 1 Randy MacLeod 2022-02-28 15:26:18 UTC
This race isn't happening for Sundeep yet, if this happens again, please collect:
   tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/temp/log.do_install



Full email from Sundeep:

The Jenkins error log shows:

2782    make[5]: Entering directory '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/git/build.x86_64-pokysdk-linux.arm-poky-linux-gnueabi/gas'
...
2806    |  /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/hosttools/install -c -m 644 ./doc/as.info '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info'
2807    |  /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/hosttools/install -c -m 644 ./doc/as.info '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info'
2808    | /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/hosttools/install: cannot change permissions of '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info/as.info': No such file or directory
...
2817    | make[5]: Leaving directory '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/git/build.x86_64-pokysdk-linux.arm-poky-linux-gnueabi/gas'

 

Here the ‘as.info’ installed twice by install tool with-in one process/thread.

 

But the log file where the build is success shows like below. The as.info installed twice but not with-in the same thread.

532     make[4]: Entering directory '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/git/build.x86_64-pokysdk-linux.arm-poky-linux-gnueabi/gas'
548     /home/bft/skokkonda/beaglebone/poky/build/tmp/hosttools/install -c -m 644 ./doc/as.info '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-      sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info'
549     install-info --info-dir='/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info' '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info/as.info'
554     make[5]: Entering directory '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/git/build.x86_64-pokysdk-linux.arm-poky-linux-gnueabi/gas'
555     /home/bft/skokkonda/beaglebone/poky/build/tmp/hosttools/mkdir -p '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info'
556     /home/bft/skokkonda/beaglebone/poky/build/tmp/hosttools/install -c -m 644 ./doc/as.info '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info'
557     install-info --info-dir='/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info' '/home/bft/skokkonda/beaglebone/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/image/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-pokysdk-linux/usr/share/info/as.info'

So, we suspect the issue might be because the as.info installed twice with in the same thread. But, to confirm and to help our further analysis we need the complete error log (from ‘tmp/work/x86_64-nativesdk-pokysdk-linux/binutils-cross-canadian-arm/2.38-r0/temp/log.do_install’).
Comment 2 pgowda.cve 2022-03-02 13:32:49 UTC
The race condition in binutils/gas folder was introduced with the following patch.
It avoids recursive make into the doc folder. It would install as.info twice which resulted in the race condition sometimes. 
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=bde299e063de090bf36c1fe51874d1e9f4d94c3c

On debugging the code, it was found that the issue was related to install-data-local.
On further analysis, there is already a patch in binutils that removes install-data-local.
On applying the patch as.info is installed once as expected and there’s no possibility of any race condition.
https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=9a84a44d5df4618dd616137fa755bd71b7eacc5f

We will post the patch to OE/master sources after testing
Comment 3 pgowda.cve 2022-03-03 05:58:26 UTC
Patch posted to OE mailing list as:-
https://lists.openembedded.org/g/openembedded-core/message/162634
Comment 4 pgowda.cve 2022-03-07 03:45:29 UTC
Patch committed to master branch as:-
https://git.openembedded.org/openembedded-core/commit/?id=c08a245990eb46906476dc0f6ade0482c7be241d