Bug 14016

Summary: glibc build race in do_compile
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Richard Purdie <richard.purdie>
Component: coreAssignee: Khem Raj <raj.khem>
Status: RESOLVED OBSOLETE QA Contact:
Severity: normal    
Priority: Medium CC: anuj.mittal, meta.mr.watcher, meta.watcher, randy.macleod
Version: 3.2   
Target Milestone: 3.4 M1   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Richard Purdie 2020-08-20 00:18:45 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1250

fedora32-ty-1

Log is huge, tail end is:

/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/build-x86_64-poky-linux/elf/pldd.o.d:232: *** missing separator.  Stop.
| make[4]: Leaving directory '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/git/elf'
| make[3]: *** [../o-iterator.mk:9: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/build-x86_64-poky-linux/elf/rtld-dl-addr-obj.os] Error 2
| make[3]: *** Waiting for unfinished jobs....
| make[4]: Leaving directory '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/git/string'
| make[3]: Leaving directory '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/git/elf'
| make[2]: *** [Makefile:553: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/build-x86_64-poky-linux/elf/rtld-libc.a] Error 2
| make[2]: *** Waiting for unfinished jobs....
| make[2]: Leaving directory '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/git/elf'
| make[1]: *** [Makefile:490: elf/others] Error 2
| make[1]: Leaving directory '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/git'
| make: *** [Makefile:9: all] Error 2
| WARNING: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_comp
ile.3265138:199 exit 1 from 'exit 1'
| WARNING: Backtrace (BB generated script):
| 	#1: bbfatal_log, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 199
| 	#2: die, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 189
| 	#3: oe_runmake, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 175
| 	#4: base_do_compile, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 167
| 	#5: do_compile, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 152
| 	#6: main, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1877933/tmp/work/core2-64-poky-linux/glibc/2.32-r0/temp/run.do_compile.3265138, line 209
| 
| Backtrace (metadata-relative locations):
| 	#1: bbfatal_log, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/classes/logging.bbclass, line 72
| 	#2: die, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/classes/base.bbclass, line 56
| 	#3: oe_runmake, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/classes/base.bbclass, line 65
| 	#4: base_do_compile, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/classes/base.bbclass, line 327
| 	#5: do_compile, /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-core/glibc/glibc_2.32.bb, line 99
NOTE: recipe glibc-2.32-r0: task do_compile: Failed
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-core/glibc/glibc_2.32.bb:do_compile) failed with exit code '1'
NOTE: recipe linux-yocto-5.4.58+gitAUTOINC+83311f062f_7e20784a1d-r0: task do_compile: Succeeded
NOTE: Tasks Summary: Attempted 2301 tasks of which 1711 didn't need to be rerun and 1 failed.
Comment 1 Khem Raj 2020-08-20 22:45:04 UTC 
I wonder if you were able to capture pldd.o.d file, since that might give a bit of insight into the issue, these files are created using -MD gcc option, there are many variables in play here
Comment 2 Randy MacLeod 2020-11-19 15:45:51 UTC 
Lowered to medium, raise to M+ if seen again.
Comment 3 Anuj Mittal 2020-11-23 05:33:19 UTC 
I got this while building gatesgarth:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2760/steps/10/logs/step5b

In the same build configuration, next step also failed with same error:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2760/steps/10/logs/step6b
Comment 4 Khem Raj 2021-03-18 02:56:10 UTC 
one common thing I see is that in all reported cases its failing in elf/ dir

| make[1]: *** [Makefile:490: elf/others] Error 2

lets see if its in same area when it happens next time.
Comment 5 Randy MacLeod 2021-05-13 15:08:04 UTC 
Only seen on versions that are now EOL. Reopen if need on supported versions of glibc.