| Summary: | sdk: compiler targets glibc, even though rootfs uses musl-libc | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Daniel Klauer <daniel.klauer> |
| Component: | devtools / tool chain | Assignee: | Khem Raj <raj.khem> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium+ | CC: | anon2313, leon, meta.mr.watcher, meta.watcher, raj.khem, randy.macleod, richard.purdie |
| Version: | unspecified | ||
| Target Milestone: | 3.2 M3 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
Robert can you or someone else in your group investigate and discuss with Ricahrd to determine who should work on this bug. I found this oe-core commit: http://git.openembedded.org/openembedded-core/commit/?id=ee16cb0a3f40384f509083f0ef3f36b68f73b7cf which seems to say that this is intentional. But maybe the dynamic linker path issue was overlooked. It was intended that we have one toolchain targeting both and you're right, we probably overlooked the dynamic loader path. We may need to add more commandline options in the toolchain environment files to ensure the binaries work as intended. Is there a fix or workaround for this? When I build Poky with 'TCLIBC = "musl"' added to local.conf for genericx86-64 then the SDK does not produce any usable binaries. I finally stumbled across that the eSDK does produce something that will run. It seems to be possible to specify a custom -Wl,-dynamic-linker,/lib/foo.so in LDFLAGS to override the default. And maybe add it to the default "export LDFLAGS=..." in the environment-* script in the SDK to make it a bit more convenient. The way I ended up working around this is by building the extended SDK and using $CC -dumpspecs to a file called musl.spec. Then I altered the regular and meta-toolchain SDKs to add "-specs=musl.spec" to the environment variable CC in the environment setup script. Now "$CC test.c -o test" produces a working binary. While simple programs work more complicated items can't be linked. I haven't been able to determine any workaround. Basically meta-toolchain or populate_sdk results in non-functional toolchains. With the eSDK not being able to be installed due to xmlcatalog #13732 leaves you with no kind of SDK being available. (In reply to comment #7) > While simple programs work more complicated items can't be linked. I > haven't been able to determine any workaround. Basically meta-toolchain or > populate_sdk results in non-functional toolchains. With the eSDK not being > able to be installed due to xmlcatalog #13732 leaves you with no kind of SDK > being available. Can you try adding -mmusl option to gcc invocation ? this should inform gcc driver about the profile it should assume. If this works then perhaps we should document it for 3.1, if its easy to insert it into build then perhaps we should do it but I guess we have to generate scripts instead of symlinks perhaps with -mmusl added The reason -mmusl is not in the SDK toolchain CC or CFLAGS is because of this: During toolchain-script.bbclass, the libc-musl override is not in OVERRIDES: OVERRIDES=task-generate-content:linux-musl:x86-64:pn-meta-environment-qemux86-64:qemuall:qemux86-64:nodistro:class-cross-canadian:libc-glibc:forcevariable but linux-musl is in OVERRIDES (note: linux-musl vs libc-musl) This append at the top of toolchain-script.bbclass is thus without effect: TARGET_CC_ARCH_append_libc-musl = " -mmusl" Changing it to linux-musl at least gets the -mmusl flag into the SDK environment script: TARGET_CC_ARCH_append_linux-musl = " -mmusl" I did not test this on target yet (but will), nor do I know if this affects more complex library cases. (In reply to comment #9) > The reason -mmusl is not in the SDK toolchain CC or CFLAGS is because of > this: > > During toolchain-script.bbclass, the libc-musl override is not in OVERRIDES: > > OVERRIDES=task-generate-content:linux-musl:x86-64:pn-meta-environment- > qemux86-64:qemuall:qemux86-64:nodistro:class-cross-canadian:libc-glibc: > forcevariable > > but linux-musl is in OVERRIDES (note: linux-musl vs libc-musl) > > > This append at the top of toolchain-script.bbclass is thus without effect: > TARGET_CC_ARCH_append_libc-musl = " -mmusl" > > Changing it to linux-musl at least gets the -mmusl flag into the SDK > environment script: > > TARGET_CC_ARCH_append_linux-musl = " -mmusl" > > I did not test this on target yet (but will), nor do I know if this affects > more complex library cases. what is TCLIBC variable set to ? and I am seeing linux-musl and libc-glibc in overrides which is not common, so how did you end up doing this TCLIBC = "musl" The OVERRIDES contains both linux-musl and libc-glibc, during the canadian cross build only I think, and is the case during toolchain-scripts.bbclass. Just also echo OVERRIDES to the SDK environment script and you can easily verify the above. Full minimal test case here: https://github.com/likewise/oe-musl-sdk-cmake 1. clone 2. make See the SDK build FAIL. Proposed fix is included: toolchain-scripts-musl.patch 1. make clean 2. make The SDK build now PASS. (Which means -mmusl flag and correct loader). As said, I have to test this on target still. patch is on ml confirmed the v2 patch works for our MUSL based x86_64 SDK. |
When using the poky-tiny distro, the rootfs image uses musl libc (TCLIBC = "musl"). But the SDK produced for this image contains a compiler targetting glibc, so binaries built with it don't run on the target image (wrong -dynamic-linker path). Nevertheless, the SDK's target sysroot contains musl libc headers and binaries as expected. To reproduce: - poky master - put in local.conf: DISTRO = "poky-tiny" MACHINE = "qemuarm" - bitbake core-image-minimal -c populate_sdk - install the SDK somewhere For comparison: The tmp/sysroots-components/x86_64/gcc-cross-arm/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-gcc from yocto build directory reports "Target: arm-poky-linux-musleabi". The "arm-poky-linux-gnueabi-gcc" from the SDK installation directory reports "Target: arm-poky-linux-gnueabi", plus there are "arm-poky-linux-musleabi-gcc", "arm-poky-linux-musl-gcc" and "arm-poky-linux-gcc" which are all symlinks to it. I got the same problem building for an aarch64 machine (lx2160ardb from meta-freescale). Part of it seems to be that cross-canadian.bbclass forces TARGET_OS = "linux", instead of leaving it at TARGET_OS = "linux-musl". Tested only with arm/aarch64.