Created attachment 3036 [details] log.do_compile Git rev: master/8debfea81e69d038bd2d56314b272cb74f5582ed Steps: 1. Set MACHIN = "qemumips" or "qemumips64" 2. Run bitbake core-image-sato -c populate_sdk_ext This issue only happens on Fedora 23 i686 for qemumips and qemumips64 Please see the attached log for details
Is this reproducible on demand? I can't replicate on 32-bit Debian.
This issue only happens on Fedora 23. But I can not reproduce now.
So this was a one-off failure that you can't replicate anymore?
(In reply to comment #3) > So this was a one-off failure that you can't replicate anymore? I found the steps to reproduce it on Fedora 23: 1. Run ". ./oe-init-build-env" to set a build directory, set MACHINE = "qemux86-64". 2. Run bitbake nativesdk-glibc It could be built without problems. 3. Run ". ./oe-init-build-env" to set a new build directory, set MACHINE = "qemumips". Use the same sstate-cache directory generated by step 1. 4. bitbake nativesdk-glibc The issue happens.
CC'ing Khem to see if he has any thoughts. Khem: 32-bit build host fails to build nativesdk-glibc for MIPS targets.
Can you generate config.log for nativesdk-glibc in passing and failing cases.
Created attachment 3104 [details] config.log.pass Here is the passing config.log which build from scratch for mips
Created attachment 3105 [details] config.log.fail Here is the failing config.log which build with sstate-cache for mips
I dont see any differences that would matter between the outputs.
/buildarea/sdk-automation-test/ubs-build/build-mips64/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar: relocation error: /usr/lib/libc.so.6: symbol _dl_starting_up, version GLIBC_PRIVATE not defined in file ld-linux.so.2 with link time reference this says that its using different ld.so and libc.so when invoking ar program. the libc.so is being used from build host where as ld.so is probably coming from crossdk and another think of interest would be if nativesdk-glibc is recompiling when using mips or mips64 as target. This should not happen. on broken build workspace can you run /buildarea/sdk-automation-test/ubs-build/build-mips64/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar -v and see if it succeeds. If it does not then show the output of readelf -a and readelf -d
$ /buildarea/poky/build/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar -v /buildarea/poky/build/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar: relocation error: /usr/lib/libc.so.6: symbol _dl_starting_up, version GLIBC_PRIVATE not defined in file ld-linux.so.2 with link time reference
Created attachment 3106 [details] readelf-a
Created attachment 3107 [details] readelf-d
OK interesting INTERP 0x000174 0x08047174 0x08047174 0x0004a 0x0004a R 0x1 [Requesting program interpreter: /buildarea/poky/build/tmp/sysroots-uninative/i686-linux/lib/ld-linux.so.2] so its using the ld.so from uninative sysroot. but libc.so its using is from build host. So there is a mismatch. Why is it not using libc.so from uninative sysroot ? Can you show output of readelf -d on working version ? also show output of ldd /buildarea/sdk-automation-test/ubs-build/build-mips64/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar on both working and non-working ar binaries
non-working: $ ldd /buildarea/poky/build/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar linux-gate.so.1 (0xb77f3000) libz.so.1 => /usr/lib/libz.so.1 (0xb77d7000) libfl.so.2 => not found libm.so.6 => /usr/lib/libm.so.6 (0xb776f000) libdl.so.2 => /usr/lib/libdl.so.2 (0xb776a000) libc.so.6 => /usr/lib/libc.so.6 (0xb759d000) /buildarea/poky/build/tmp/sysroots-uninative/i686-linux/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x800d2000) working: $ ldd /buildarea/poky/build/tmp/sysroots/i686-linux/usr/bin/i686-pokysdk-linux.gcc-crosssdk-initial-i686/../../libexec/i686-pokysdk-linux.gcc-crosssdk-initial-i686/gcc/i686-pokysdk-linux/5.3.0/ar linux-gate.so.1 (0xb77ea000) libz.so.1 => /usr/lib/libz.so.1 (0xb77ce000) libfl.so.2 => not found libm.so.6 => /usr/lib/libm.so.6 (0xb7766000) libdl.so.2 => /usr/lib/libdl.so.2 (0xb7761000) libc.so.6 => /usr/lib/libc.so.6 (0xb7594000) /lib/ld-linux.so.2 (0x800fb000)
Created attachment 3108 [details] readelf-a-work
Created attachment 3109 [details] readelf-d-work
OK so here I see the difference, in working version INTERP 0x000154 0x08048154 0x08048154 0x00013 0x00013 R 0x1 [Requesting program interpreter: /lib/ld-linux.so.2] which is calling ldso from build host and so is libc.so therefore it works as expected but in failure case its using the ldso from uninative rootfs. Please investigate why is this case. ? Its
Yi Zhao, can you still replicate this? We've had a complete toolchain upgrade since this was filed so I'm wondering if it was fixed in that.
(In reply to comment #19) > Yi Zhao, can you still replicate this? We've had a complete toolchain > upgrade since this was filed so I'm wondering if it was fixed in that. Yes, I can still reproduce it with recent commit: master/6c5d7f1fb276cbe0a461ece6c8f0ca17a478fa8c
This issue still exists in 2.2 M1. Git rev: master/6f0c5537e02c59e1c8f3b08f598dc049ff8ee098
(In reply to comment #21) > This issue still exists in 2.2 M1. > Git rev: master/6f0c5537e02c59e1c8f3b08f598dc049ff8ee098 Did you investigate further on comment #18
This issue still happens on Fedora 24 for 2.2 M2 RC1 build
I can't replicate this on my 32-bit Debian machine and current master. Yi Zhao, can you replicate this still?
Just noticed that I asked this before and it appeared to be only reproducable on Fedora. Can you verify that you've tested on Debian on 32-bit and confirmed that it works and the same OE commit fails on 32-bit Fedora? (which would imply that it's the library path layout differences) I ask to test again as there's been a new uninative since this was filed.
(In reply to comment #25) > Just noticed that I asked this before and it appeared to be only > reproducable on Fedora. Can you verify that you've tested on Debian on > 32-bit and confirmed that it works and the same OE commit fails on 32-bit > Fedora? > > (which would imply that it's the library path layout differences) > > I ask to test again as there's been a new uninative since this was filed. Hi, I did test again for 2.2 M3 RC1 and can confirm this issue only happens on Fedora i686 (The distro is Fedora 24 now). The Debian 8 i686 works fine.
Ok, reopening.
$ uname -a Linux george.burtonini.com 4.7.3-200.fc24.i686+PAE #1 SMP Wed Sep 7 17:47:31 UTC 2016 i686 i686 i386 GNU/Linux $ bitbake nativesdk-glibc Loading cache: 100% |###################################################################################################| Time: 0:00:00 Loaded 1316 entries from dependency cache. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.31.1" BUILD_SYS = "i686-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "mips-poky-linux" MACHINE = "qemumips" DISTRO = "poky" DISTRO_VERSION = "2.1+snapshot-20160920" TUNE_FEATURES = "o32 bigendian fpu-hard mips32r2" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "master:a767c78abeb19a9d730e3c9b1f64c13fbbcade17" Initialising tasks: 100% |##############################################################################################| Time: 0:00:01 NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Tasks Summary: Attempted 356 tasks of which 260 didn't need to be rerun and all succeeded. Works for me. What else are you doing differently?
(In reply to comment #29) > $ uname -a > Linux george.burtonini.com 4.7.3-200.fc24.i686+PAE #1 SMP Wed Sep 7 17:47:31 > UTC 2016 i686 i686 i386 GNU/Linux > > $ bitbake nativesdk-glibc > Loading cache: 100% > |############################################################################ > #######################| Time: 0:00:00 > Loaded 1316 entries from dependency cache. > NOTE: Resolving any missing task queue dependencies > > Build Configuration: > BB_VERSION = "1.31.1" > BUILD_SYS = "i686-linux" > NATIVELSBSTRING = "universal" > TARGET_SYS = "mips-poky-linux" > MACHINE = "qemumips" > DISTRO = "poky" > DISTRO_VERSION = "2.1+snapshot-20160920" > TUNE_FEATURES = "o32 bigendian fpu-hard mips32r2" > TARGET_FPU = "" > meta > meta-poky > meta-yocto-bsp = "master:a767c78abeb19a9d730e3c9b1f64c13fbbcade17" > > Initialising tasks: 100% > |############################################################################ > ##################| Time: 0:00:01 > NOTE: Executing SetScene Tasks > NOTE: Executing RunQueue Tasks > NOTE: Tasks Summary: Attempted 356 tasks of which 260 didn't need to be > rerun and all succeeded. > > Works for me. What else are you doing differently? Hi Ross, The reproduced steps are in comment 4: 1. Run ". ./oe-init-build-env" to set build-1 directory, set MACHINE = "qemux86-64" and set a public sstate-cache directory. 2. Run bitbake nativesdk-glibc It could be built without problems. 3. Run ". ./oe-init-build-env" to set build-2 directory, set MACHINE = "qemumips64". Use the public sstate-cache directory generated by step 1. 4. bitbake nativesdk-glibc The issue happens. The key point is use the same sstate-cache directory.
That can't be the reproducer because it will just take nativesdk-glibc from the sstate.
(In reply to comment #31) > That can't be the reproducer because it will just take nativesdk-glibc from > the sstate. But other archs (arm, arm64, ppc) can use the share sstate-cache directory without problems.
Finally made it reproduce. bitbake nativesdk-libgcc in one build dir, then bitbake nativesdk-libgcc -C configure in another.
Also replicated with qemux86.
You don't even need to have two build dirs, the trigger is unpacking ar from sstate (and so causing the rpath relocations to happen). This is a minimal reproducer: $ bitbake nativesdk-libgcc && rm -rf tmp && bitbake nativesdk-libgcc -C configure
Basically the super-minimal test case is: $ rm -rf tmp; bitbake binutils-crosssdk-x86_64-pokysdk-linux Then running tmp/sysroots/i686-linux/usr/bin/x86_64-pokysdk-linux/x86_64-pokysdk-linux-ar will fail to start up: ross@george ~/Yocto/poky/build/tmp/sysroots/i686-linux/usr/bin/x86_64-pokysdk-linux $ ./x86_64-pokysdk-linux-ar ./x86_64-pokysdk-linux-ar: relocation error: /usr/lib/libc.so.6: symbol _dl_starting_up, version GLIBC_PRIVATE not defined in file ld-linux.so.2 with link time reference I'm guessing the requirement is that host glibc and our glibc are sufficiently different.
See https://sourceware.org/ml/libc-alpha/2014-02/msg00580.html Fedora seems to carry a patch which changes defaults. I wonder if it can just use oe built libc
I think I have a patch. Yi Zhao, would you be able to test poky-contrib:ross/linkage? I'll post it shortly to the list too.
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0c9ba38d929ebe5da2c2e0b2a605e6d12cc57266 and preceding 3 patches.
Really mark as resolved
Verified with 2.2 rc1. git rev: morty/371ab635086e2878ada1f00a59c7ec826481ba0a