hi, we're currently on meta-rust-bin and would like to move closer to upstream alignment with meta-mixins-lts@scarthgap/rust. today we use TCLIBC="musl" (via multiconfig) and produce a statically linked binary. unfortunately, i've been unable to produce a statically linked rust binary against musl libc, and only get one dynamically linked to musl libc.so, even when trying to use `RUSTFLAGS:prepend= " -C target-feature=+crt-static "` in my recipe. is there a way to support building static binaries with TCLIBC=musl in meta-mixins-lts@scarthgap/rust?
Technically we don't track meta-lts-mixin bugs in the YP bugzilla but... Have you tried this on oe-core /master ? If not please do. If you can provide an example test case and recipe then people will be more willing to investigate.
Nick, this works for me when building master with: conf/local.conf IMAGE_INSTALL:append = " ripgrep fd-find" OE_FRAGMENTS += "core/yocto/root-login-with-empty-password" $ TCLIBC=musl bitbake core-image-minimal root@qemux86-64:~# ldd /bin/ls /lib/ld-musl-x86_64.so.1 (0x7f2623f49000) libc.so => /lib/ld-musl-x86_64.so.1 (0x7f2623f49000) root@qemux86-64:~# rg --help | head -3 head: unrecognized option: 3 BusyBox v1.37.0 () multi-call binary. Usage: head [OPTIONS] [FILE]... root@qemux86-64:~# ldd `which rg` /lib/ld-musl-x86_64.so.1 (0x7fc2eb1f6000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7fc2eac64000) libc.so => /lib/ld-musl-x86_64.so.1 (0x7fc2eb1f6000) Can you try older branches?
(In reply to Randy MacLeod from comment #2) > Nick, this works for me when building master with: > conf/local.conf > > IMAGE_INSTALL:append = " ripgrep fd-find" > OE_FRAGMENTS += "core/yocto/root-login-with-empty-password" > > $ TCLIBC=musl bitbake core-image-minimal > > > root@qemux86-64:~# ldd /bin/ls > /lib/ld-musl-x86_64.so.1 (0x7f2623f49000) > libc.so => /lib/ld-musl-x86_64.so.1 (0x7f2623f49000) > root@qemux86-64:~# rg --help | head -3 > head: unrecognized option: 3 > BusyBox v1.37.0 () multi-call binary. > > Usage: head [OPTIONS] [FILE]... > root@qemux86-64:~# ldd `which rg` > /lib/ld-musl-x86_64.so.1 (0x7fc2eb1f6000) > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7fc2eac64000) > libc.so => /lib/ld-musl-x86_64.so.1 (0x7fc2eb1f6000) > these binaries appear to be dynamically linked to libc. we're looking for statically linked rust binaries when using TCLIBC=musl. > > Can you try older branches?
Ah, okay. Please provide an example recipe and steps to build it.
So, first I'd like to apologize for my earlier misunderstanding and the resultant BZ comment noise. I was dealing with this bug as a result of the YP bug triage meeting and I was trying to wrap it up before heading out to an appointment. I see now, that you can't do what I've been asking you for and that's the bug! One can do create a statically linked binary outside of bitbake using just the flags you listed: " -C target-feature=+crt-static " so with a hw.rs app: $ cargo new --bin hw $ RUSTFLAGS="-C target-feature=+crt-static" cargo build --release ... $ ldd target/release/hw statically linked Of course this is glibc using rust-1.91.1 from rustup. For bitbake, or you already said (!), one can add these options to RUSTFLAGS in a rust recipe, such as meta-oe's recently added ripgrep: ❯ git diff diff --git a/meta-oe/recipes-extended/ripgrep/ripgrep_15.1.0.bb b/meta-oe/recipes-extended/ripgrep/ripgrep_15.1.0.bb index 7bb6be1cb6..3b87c65154 100644 --- a/meta-oe/recipes-extended/ripgrep/ripgrep_15.1.0.bb +++ b/meta-oe/recipes-extended/ripgrep/ripgrep_15.1.0.bb @@ -15,6 +15,7 @@ S = "${CARGO_VENDORING_DIRECTORY}/ripgrep-${PV}" inherit cargo cargo-update-recipe-crates +RUSTFLAGS:append = " -C target-feature=+crt-static" DEPENDS += "libstd-rs" require ${BPN}-crates.inc and append: IMAGE_INSTALL:append = " ripgrep" to conf/local.conf and build for musl: $ TCLIBC=musl bitbake core-image-minimal Looking at the log file (attached) the flags do get passed, but the executable is not statically linked: --- $ runqemu kvm nographic snapshot ... root@qemux86-64:~# uname -r 6.17.10-yocto-standard root@qemux86-64:~# ldd /usr/bin/rg /lib/ld-musl-x86_64.so.1 (0x7f87877b0000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7f878721e000) libc.so => /lib/ld-musl-x86_64.so.1 (0x7f87877b0000) root@qemux86-64:~# ldd --version musl libc (x86_64) Version 1.2.5-git-101-g0ccaf057 ... root@qemux86-64:~# rg foo /usr /usr/lib/ssl-3/openssl.cnf.dist 302:proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo ... ... --- I tested and this is also the case when linking against glibc using bitbake. Phew, so I think we have a reproducer and proof that the problem happens on oe-core master so this is a valid bug for this bugzilla. Sundeep, Please have someone take a look when they have time available.
Created attachment 5160 [details] musl-ripgrep-15.1.0-log.do_compile
The fix here should include an oeqa test case.
See: tspec['dynamic-linking'] = True in the change for: https://lore.kernel.org/openembedded-core/20251222093830.2658141-2-Yash.Shinde@windriver.com/ and my comment.
I’ve been digging into this with a simple "Hello World" recipe and found that rust-target-config.bbclass is currently hardcoded to force dynamic linking, which makes it ignore +crt-static flags. There’s also a tricky circular dependency where libstd keeps pulling in libgcc_s.so for unwinding symbols. I’m currently testing a combination of a class patch and specific linker group flags to satisfy these dependencies statically. I should have a working recipe and the final fix/workaround ready to share soon.
Update on investigation: ******* Using a simple "Hello World" Rust recipe, I've identified two main issues preventing fully static linking with musl libc: 1. **Rust Target Spec Defaults in rust-target-config.bbclass**: The class sets tspec['dynamic-linking'] = True, but crt-static-respected and crt-static-default are false by default. This ignores user overrides like +crt-static. Proposed fix: Explicitly set tspec['crt-static-respected'] = True and tspec['crt-static-default'] = False to honor user requests while supporting dynamic linking capability. Reference: Rust source (compiler/rustc_target/src/spec/mod.rs). 2. **libunwind Linking**: libunwind.a is available (via libstd-rs dependency), but not automatically linked in Rust's musl static targets, leading to _Unwind_Resume errors and fallback to dynamic libgcc_s.so.1. Unlike C cross-builds, Rust needs explicit -lunwind flags. Exploring integration via class patch and linker group flags to resolve statically. Some errors trace back to the linker wrapper. Testing a combined workaround. Will update on Issue 2 soon.
I have successfully built a fully static Rust “Hello World” binary with TCLIBC=musl using the below patch. Although the original reporter did not provide their exact recipe, these changes resolve the underlying oe-core infrastructure issues that previously prevented +crt-static from being applied correctly. 1. rust-target-config.bbclass: Fixed the target specification logic. By setting tspec['crt-static-respected'] = True for musl targets, the Rust compiler now actually listens to the -C target-feature=+crt-static flag instead of ignoring it. 2. rust-common.bbclass (Linker Wrapper): Updated the create_wrapper_rust logic. Added -lunwind to WRAPPER_TARGET_EXTRALD for musl. This resolves the _Unwind_Resume undefined reference errors that previously forced a fallback to the dynamic libgcc_s.so. - Explicitly added the staging library path to the linker wrapper to ensure libunwind.a is found during the static link phase. 3. Dependency Tree (libstd-rs & libcxx): Updated libstd-rs to depend on libcxx for musl targets. This ensures that the necessary unwind headers and libraries are available in the sysroot during the Rust runtime build. libcxx: Enabled the unwind target in the build/install tasks to ensure the static library is generated and deployed. ---------- With this patch applied, a standard Rust recipe using: RUSTFLAGS:append:class-target = " -C target-feature=+crt-static" now produces a "statically linked" binary. Note: While this fixes the infrastructure, complex recipes with many crate dependencies may still require individual tweaks if they explicitly pull in dynamic C libraries via -sys crates. Nick, could you please apply the attached patch and verify if your specific recipes now produce the expected static binaries? Please let us know the results so we can move toward upstreaming this fix." ----------------------- diff --git a/meta/classes-recipe/rust-common.bbclass b/meta/classes-recipe/rust-common.bbclass index 34bb2377cf..7e56c3a9db 100644 --- a/meta/classes-recipe/rust-common.bbclass +++ b/meta/classes-recipe/rust-common.bbclass @@ -165,7 +165,7 @@ WRAPPER_TARGET_EXTRALD = "" # see recipes-devtools/gcc/gcc/0018-Add-ssp_nonshared-to-link-commandline-for-musl-targe.patch # we need to link with ssp_nonshared on musl to avoid "undefined reference to `__stack_chk_fail_local'" # when building MACHINE=qemux86 for musl -WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared" +WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared -lunwind" WRAPPER_TARGET_AR = "${AR}" # compiler is used by gcc-rs @@ -188,7 +188,8 @@ do_rust_create_wrappers () { # Yocto Target / Rust Target C++ compiler create_wrapper_rust "${RUST_TARGET_CXX}" "${WRAPPER_TARGET_EXTRALD}" "${CRATE_CC_FLAGS}" "${WRAPPER_TARGET_CXX}" "${CXXFLAGS}" # Yocto Target / Rust Target linker - create_wrapper_rust "${RUST_TARGET_CCLD}" "${WRAPPER_TARGET_EXTRALD}" "" "${WRAPPER_TARGET_CCLD}" "${WRAPPER_TARGET_LDFLAGS}" + EXTRAS_CCLD="-L${STAGING_DIR_TARGET}${libdir} ${WRAPPER_TARGET_EXTRALD}" + create_wrapper_rust "${RUST_TARGET_CCLD}" "${EXTRAS_CCLD}" "" "${WRAPPER_TARGET_CCLD}" "${WRAPPER_TARGET_LDFLAGS}" # Yocto Target / Rust Target archiver create_wrapper_rust "${RUST_TARGET_AR}" "" "" "${WRAPPER_TARGET_AR}" diff --git a/meta/classes-recipe/rust-target-config.bbclass b/meta/classes-recipe/rust-target-config.bbclass index 2e83cf5aa7..3469de2142 100644 --- a/meta/classes-recipe/rust-target-config.bbclass +++ b/meta/classes-recipe/rust-target-config.bbclass @@ -429,6 +429,8 @@ def rust_gen_target(d, thing, wd, arch): tspec['has-thread-local'] = True tspec['position-independent-executables'] = True tspec['panic-strategy'] = d.getVar("RUST_PANIC_STRATEGY") + if "musl" in tspec['llvm-target']: + tspec['crt-static-respected'] = True # write out the target spec json file with open(wd + rustsys + '.json', 'w') as f: diff --git a/meta/recipes-devtools/clang/libcxx_git.bb b/meta/recipes-devtools/clang/libcxx_git.bb index 42b2c91e43..742cae4183 100644 --- a/meta/recipes-devtools/clang/libcxx_git.bb +++ b/meta/recipes-devtools/clang/libcxx_git.bb @@ -28,8 +28,8 @@ LIC_FILES_CHKSUM = "file://libcxx/LICENSE.TXT;md5=55d89dd7eec8d3b4204b680e27da39 file://libcxxabi/LICENSE.TXT;md5=7b9334635b542c56868400a46b272b1e \ " -OECMAKE_TARGET_COMPILE = "cxxabi cxx" -OECMAKE_TARGET_INSTALL = "install-cxxabi install-cxx" +OECMAKE_TARGET_COMPILE = "unwind cxxabi cxx" +OECMAKE_TARGET_INSTALL = "install-unwind install-cxxabi install-cxx" CC = "${CCACHE}${HOST_PREFIX}clang ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}" CXX = "${CCACHE}${HOST_PREFIX}clang++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}" diff --git a/meta/recipes-devtools/rust/libstd-rs_1.93.0.bb b/meta/recipes-devtools/rust/libstd-rs_1.93.0.bb index 8af93bec57..d15edf4fa8 100644 --- a/meta/recipes-devtools/rust/libstd-rs_1.93.0.bb +++ b/meta/recipes-devtools/rust/libstd-rs_1.93.0.bb @@ -17,7 +17,7 @@ inherit cargo CVE_PRODUCT = "rust" -DEPENDS:append:libc-musl = " libunwind" +DEPENDS:append:libc-musl = " libcxx" # rv32 does not have libunwind ported yet DEPENDS:remove:riscv32 = "libunwind" DEPENDS:remove:riscv64 = "libunwind"
Sunil, Thanks for the update and it will be good to hear back from Nick. Can you add a test similar to the ones in: meta-selftest/recipes-devtools/rust or maybe: meta/lib/oeqa/runtime/cases/rust.py so we can easily monitor for regressions once this change is merged. Ask on the list about where to put the test if you like.
Analysis and Fix: ***************** Two separate problems needed fixing. Problem 1 — +crt-static being silently ignored ---------------------------------------------- The custom Rust target spec JSON for musl targets was missing the crt-static-respected field. Without it, rustc silently ignores +crt-static — the upstream built-in x86_64-unknown-linux-musl has this set internally, our OE JSON did not. Fix: set tspec['crt-static-respected'] = True for musl targets in rust-target-config.bbclass. Once this was in place, rustc correctly switched to static linking mode but the build immediately failed with: ``` cannot find -lunwind: No such file or directory have you installed the static version of the unwind library? ``` Problem 2 — finding a working static unwinder --------------------------------------------- I looked at three options: Option 1 — GNU libunwind (libunwind_1.8.3.bb) -------- The recipe has EXTRA_OECONF = "--enable-static" but no-static-libs.inc (pulled in via defaultsetup.conf <- bitbake.conf) appends --disable-static to EXTRA_OECONF globally. Since autoconf processes flags left-to-right, --disable-static wins. Confirmed in the configure log: ``` NOTE: Running configure --enable-static --disable-tests --disable-static checking whether to build static libraries... no ``` This affects both glibc and musl equally — verified by building libunwind, both produce only .so files. I forced a static build via DISABLE_STATIC:libc-musl = "" in libunwind_1.8.3.bb. libunwind.a was produced and installed, but the link still failed. Checked with nm: ``` nm libunwind.a | grep ' T _Unwind' (no output) ``` GNU libunwind (nongnu.org) does not export _Unwind_* symbols from its static archive on x86_64. Confirmed by nm on all static archives produced by the build: ``` nm libunwind.a | grep ' T _Unwind' → nothing nm libunwind-x86_64.a | grep ' T _Unwind' → nothing nm libunwind-dwarf-local.a | grep ' T _Unwind' → nothing nm libunwind-elf64.a | grep ' T _Unwind' → nothing ``` This is a known issue. Dead end. https://github.com/libunwind/libunwind/issues/761 Option 2 — GCC's libgcc_eh.a -------- libgcc_eh.a is already present in the sysroot at usr/lib/x86_64-oe-linux-musl/15.2.0/ and has all the required symbols: ``` nm libgcc_eh.a | grep ' T _Unwind' _Unwind_Resume, _Unwind_RaiseException, _Unwind_GetIP, _Unwind_Backtrace ... (all present) ``` I added -lgcc_eh via RUSTFLAGS:append:libc-musl in rust-common.bbclass. The _Unwind_* symbols resolved but then hit a new set of errors: ``` undefined reference to `pthread_once' undefined reference to `pthread_mutex_lock' undefined reference to `pthread_mutex_unlock' ``` libgcc_eh.a uses pthreads internally via gthr-posix.h. On musl, pthread symbols only exist in libc.so dynamically. Confirmed: ``` nm libc.a | grep ' T pthread_once' → nothing nm libpthread.a | grep ' T pthread_once' → nothing ``` Tried different orderings — -lpthread -lgcc_eh, (-lgcc_eh -lpthread), and -Wl,-Bstatic,-lpthread,-lgcc_eh,-Bdynamic — all failed for the same reason. This is a fundamental incompatibility between libgcc_eh.a and musl's threading model. Gave up on this option. Option 3 — LLVM's libunwind (from libcxx_git.bb) -------- LLVM's libunwind is already compiled as part of the LLVM runtimes build in libcxx_git.bb via -DLLVM_ENABLE_RUNTIMES='libcxx;libcxxabi;libunwind' but was not being installed. I added the install step scoped to :libc-musl and confirmed: ``` nm libunwind.a | grep ' T _Unwind' _Unwind_Resume, _Unwind_RaiseException, _Unwind_GetIP, _Unwind_Backtrace ... (all present, no pthread dependency) ``` This worked. Collision safety: ---------------- Since GNU libunwind never produces libunwind.a on either glibc or musl (as explained above, no-static-libs.inc always wins), installing LLVM's libunwind.a into ${libdir} has no collision risk on either target. I kept the change scoped to :libc-musl anyway since LLVM libunwind.a is only needed for Rust musl static builds, and there is a known history of conflicts when meta-clang's libcxx and oe-core's libunwind are both in the build. https://github.com/kraj/meta-clang/issues/82 https://github.com/kraj/meta-clang/issues/211 Final fix — 4 files ******************* rust-target-config.bbclass — set crt-static-respected = True for musl targets rust-common.bbclass — add -lunwind to linker wrapper and -L path to find it libcxx_git.bb — install LLVM libunwind.a on musl only libstd-rs_1.94.0.bb — depend on libcxx instead of libunwind for musl Tested with a hello-rust recipe with RUSTFLAGS:append:class-target = " -C target-feature=+crt-static" — binary confirmed statically linked via file and readelf.
Pushed the patch to oe-core-contrib and ran it through the Autobuilder: Commit: https://git.openembedded.org/openembedded-core-contrib/commit/?h=deepesh/rust-static-musl AB build: https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/3518/ All builders passed except musl-qemux86. The failure was a linker error during static Rust binary linking: attempted static link of dynamic object `.../recipe-sysroot/usr/lib/libunwind.so` Root cause: the extras flags appended by the linker wrapper (-L<path> -lssp_nonshared -lunwind) are placed at the end of the full linker command line — after rustc's own -Wl,-Bdynamic switch. As a result, the -lunwind flag is resolved in dynamic mode, causing the linker to pick libunwind.so instead of libunwind.a, which then fails because the overall link is static. This only affects qemux86 (i686) + musl. The x86-64 + musl builder passed cleanly. Working on a fix.
Update on qemux86 musl failure from Comment 14: The problem was that I was explicitly passing -lunwind via WRAPPER_TARGET_EXTRALD. This placed -lunwind after rustc's own -Wl,-Bdynamic in the linker command, so the linker picked libunwind.so instead of libunwind.a and failed. Fix: removed -lunwind from WRAPPER_TARGET_EXTRALD entirely. When +crt-static is active, rustc already emits -lunwind itself in the correct static section of the linker command. I just needed to make libunwind.a findable — done by appending -L${STAGING_DIR_TARGET}${libdir} to WRAPPER_TARGET_LDFLAGS for musl. Also fixed a pre-existing bug in create_wrapper_rust where extras were passed as args.append(extras) — multi-token values like "-lssp_nonshared" were being passed as one broken string to the linker. Fixed to args.extend(extras.split()).
Working on v3 based on review comments. Recent changes posted at: https://git.openembedded.org/openembedded-core-contrib/commit/?h=deepesh/rust-static-testcase_change&id=9e19c927b48313946094667af610347996f8377a Pending changes: 1. Selftest: Exploring SSTATE_SKIP_CREATION:pn-rust-static-musl-test=1 as an alternative to bitbake -C compile to ensure build/target/ directory exists without forcing recompile. 2. Cleanup of now no-op DEPENDS:remove:riscv32/riscv64 = "libunwind" lines. These were originally added in commit 3ed57578 (August 2021) as a precaution alongside riscv32. After switching from libunwind to libcxx, both lines are no-ops. LLVM libunwind (from libcxx) supports riscv64 musl - verified locally with all _Unwind_* symbols present. riscv32 support upstreamed via llvm/llvm-project@b17d464. No DEPENDS:remove needed for libcxx on riscv targets. Ref: https://github.com/llvm/llvm-project/commit/b17d46430fce665d23661e23ade3ca57c3791836
Patch Sent to OE-Core ML: https://lists.openembedded.org/g/openembedded-core/message/235086
Patch merged in Master. Closing this out. https://git.openembedded.org/openembedded-core/commit/?id=75409c60e9e63fdcbb9d4f54130052991362ec08
Following the fix for this issue in commit 75409c60 (“rust: enable fully static linking with TCLIBC=musl”), CI reported a regression due to file collisions involving libunwind (under certain condition). The previous change enabled building LLVM libunwind via libcxx using `install-unwind`, which unintentionally installed shared artifacts (`libunwind.so` and headers). These conflicted with the existing `libunwind` recipe. This has now been addressed in commit 4fd2c4a99edd (“libcxx: fix libunwind collision with musl builds”): * LLVM libunwind is still built for musl targets to support fully static Rust linking. * Installation is restricted to only `libunwind.a` via a custom `do_install` step. * This avoids installing shared libraries and headers, eliminating the collision with the `libunwind` recipe. With this change, static Rust builds with musl continue to work as intended, and CI regressions caused by file conflicts are resolved.
https://git.openembedded.org/openembedded-core/commit/?id=4fd2c4a99edd40dbacb83ea664e35e139d7ebdab
thank you for your work in resolving this! unfortunately i have not found time to test this. also, we are still using scarthgap... we've considering starting to use https://git.yoctoproject.org/meta-lts-mixins/log/?h=scarthgap/rust, and maybe i can hack something together to backport these changes with that.