I had this issue twice last week, while testing patches. Each time this was triggered by a completely unrelated patch. Relaunching builds resulted in the same error, but I was able to pick said patches a bit later without any issue. So it might be something bad happening in the sstate-cache. ERROR: rust-1.90.0-r0 do_test_compile: Execution of '/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/temp/run.do_test_compile.540929' failed with exit code 1 ... | error: rustc interrupted by SIGSEGV, printing backtrace | | /srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-d18035d17505fd2b.so(+0xd2bd75) [0x7f528332bd75] | /srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6(+0x3bba0) [0x7f52822eeba0] | /srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-d18035d17505fd2b.so(+0x73df9e0) [0x7f52899df9e0] | | note: we would appreciate a report at https://github.com/rust-lang/rust | help: you can increase rustc's stack size by setting RUST_MIN_STACK=16777216 Seeing the last line, could that be related to the stack size? Also, we saw a lot of other errors in there, like the following message. At first glance, it might be completely unrelated. | Caused by: | No such file or directory (os error 2) | 0.593062166s INFO prepare_target{force=false package_id=rustc-main v0.0.0 (/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/compiler/rustc) target="build-script-build"}: cargo::core::compiler::fingerprint: fingerprint error for rustc-main v0.0.0 (/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/compiler/rustc)/RunCustomBuild/TargetInner { ..: custom_build_target("build-script-build", "/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/compiler/rustc/build.rs", Edition2024) } | 0.593074916s INFO prepare_target{force=false package_id=rustc-main v0.0.0 (/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/compiler/rustc) target="build-script-build"}: cargo::core::compiler::fingerprint: err: failed to read `/srv/pokybuild/yocto-worker/qemuarm-tc/build/build-st-292660/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/.fingerprint/rustc-main-1dee3ec81ee6b0b4/run-build-script-build-script-build`
oe-selftest-debian debian12-vk-5 mathieu/master-next completed at 2025-11-05 20:17:29+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2660/steps/15/logs/stdio reproducible fedora40-vk-4 mathieu/master-next completed at 2025-11-05 15:35:42+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251105-3znznpba https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251105-3znznpba/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2829/steps/13/logs/stdio reproducible debian12-vk-4 mathieu/master-next completed at 2025-11-05 19:52:07+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251105-r4d5ogjg https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251105-r4d5ogjg/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2830/steps/13/logs/stdio reproducible ubuntu2404-vk-2 master completed at 2025-11-06 12:42:56+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-9rih896v https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-9rih896v/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2833/steps/13/logs/stdio reproducible debian12-vk-9 master completed at 2025-11-06 19:49:32+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-kka60o40 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-kka60o40/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2839/steps/13/logs/stdio reproducible alma9-vk-2 master completed at 2025-11-06 19:20:23+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-hr11esgv https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251106-hr11esgv/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2840/steps/13/logs/stdio reproducible alma8-vk-2 mathieu/master-next completed at 2025-11-07 07:15:32+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251107-ml4tuqo6 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251107-ml4tuqo6/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2842/steps/14/logs/stdio qemuarm-tc rocky9-vk-2 mathieu/master-next completed at 2025-11-07 06:40:23+00:00 https://valkyrie.yocto.io/pub/non-release/20251107-37/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2613/steps/13/logs/stdio qemuarm64-tc debian12-vk-6 mathieu/master-next completed at 2025-11-07 06:59:20+00:00 https://valkyrie.yocto.io/pub/non-release/20251107-37/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2610/steps/13/logs/stdio qemuriscv64-tc ubuntu2204-vk-2 mathieu/master-next completed at 2025-11-07 06:48:23+00:00 https://valkyrie.yocto.io/pub/non-release/20251107-37/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/672/steps/13/logs/stdio reproducible rocky8-vk-1 mathieu/master-next completed at 2025-11-07 23:33:45+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251107-z4tw1eae https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251107-z4tw1eae/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2847/steps/14/logs/stdio beaglebone alma8-vk-2 mathieu/master-next completed at 2025-11-08 14:32:38+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2707/steps/13/logs/stdio beaglebone-alt fedora40-vk-2 mathieu/master-next completed at 2025-11-08 14:33:16+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2694/steps/12/logs/stdio genericarm64 opensuse155-vk-2 mathieu/master-next completed at 2025-11-08 14:50:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2695/steps/13/logs/stdio genericarm64-alt debian12-vk-2 mathieu/master-next completed at 2025-11-08 14:26:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2726/steps/12/logs/stdio genericx86 debian12-vk-4 mathieu/master-next completed at 2025-11-08 14:32:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2695/steps/12/logs/stdio genericx86-64 rocky9-vk-2 mathieu/master-next completed at 2025-11-08 14:43:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/2706/steps/12/logs/stdio genericx86-64-alt stream9-vk-1 mathieu/master-next completed at 2025-11-08 14:29:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2694/steps/12/logs/stdio meta-intel debian12-vk-1 mathieu/master-next completed at 2025-11-08 14:49:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2538/steps/13/logs/stdio musl-qemux86 debian12-vk-8 mathieu/master-next completed at 2025-11-08 14:53:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/2717/steps/12/logs/stdio pkgman-non-rpm debian12-vk-9 mathieu/master-next completed at 2025-11-08 15:02:00+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/2727/steps/18/logs/stdio qemuarm alma8-vk-1 mathieu/master-next completed at 2025-11-08 15:01:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2726/steps/14/logs/stdio qemuarm-tc rocky8-vk-1 mathieu/master-next completed at 2025-11-08 14:57:55+00:00 https://valkyrie.yocto.io/pub/non-release/20251108-85/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2623/steps/14/logs/stdio qemuarm64 debian12-vk-3 mathieu/master-next completed at 2025-11-08 14:59:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2696/steps/13/logs/stdio qemuarm64-alt ubuntu2204-vk-4 mathieu/master-next completed at 2025-11-08 14:31:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2680/steps/13/logs/stdio qemuriscv64 debian12-vk-8 mathieu/master-next completed at 2025-11-08 15:00:28+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/694/steps/13/logs/stdio qemuriscv64-tc ubuntu2404-vk-2 mathieu/master-next completed at 2025-11-08 15:04:53+00:00 https://valkyrie.yocto.io/pub/non-release/20251108-85/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/681/steps/13/logs/stdio qemux86 ubuntu2404-vk-3 mathieu/master-next completed at 2025-11-08 15:06:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2676/steps/13/logs/stdio qemux86-64 alma8-vk-1 mathieu/master-next completed at 2025-11-08 15:02:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2767/steps/17/logs/stdio qemux86-64-alt fedora39-vk-1 mathieu/master-next completed at 2025-11-08 14:29:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2661/steps/13/logs/stdio qemux86-world ubuntu2504-vk-1 mathieu/master-next completed at 2025-11-08 15:23:08+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2705/steps/12/logs/stdio qemux86-world-alt debian11-vk-1 mathieu/master-next completed at 2025-11-08 15:36:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2537/steps/12/logs/stdio
Sundeep, Is this the same /similar issue that we're seeing when testing rust-1.91 on YP AB ?
The first error in the build log is this: Building stage1 tool llvm-bitcode-linker (x86_64-unknown-linux-gnu) running: cd "BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src" && env -u MAKEFLAGS -u MFLAGS AR_x86_64_unknown_linux_gnu="ar" CARGO_INCREMENTAL="0" CARGO_LOG="cargo::core::compiler::fingerprint=info" CARGO_PROFILE_RELEASE_DEBUG="0" CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS="false" CARGO_PROFILE_RELEASE_OVERFLOW_CHECKS="false" CARGO_PROFILE_RELEASE_STRIP="false" CARGO_TARGET_DIR="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/bootstrap-tools" CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/wrapper/build-rust-ccld" CC_x86_64_unknown_linux_gnu="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/wrapper/build-rust-cc" CFG_COMPILER_BUILD_TRIPLE="x86_64-unknown-linux-gnu" CFG_COMPILER_HOST_TRIPLE="x86_64-unknown-linux-gnu" CFG_RELEASE="1.90.0" CFG_RELEASE_CHANNEL="stable" CFG_RELEASE_NUM="1.90.0" CFG_VERSION="1.90.0 (1159e78c4 2025-09-14) (built from a source tarball)" CFG_VER_DATE="2025-09-14" CFG_VER_DESCRIPTION="built from a source tarball" CFG_VER_HASH="1159e78c4747b02ef996e55082b704c09b970588" CFLAGS_x86_64_unknown_linux_gnu="" CXXFLAGS_x86_64_unknown_linux_gnu="" CXX_x86_64_unknown_linux_gnu="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/wrapper/build-rust-cxx" DOC_RUST_LANG_ORG_CHANNEL="https://doc.rust-lang.org/1.90.0" FORCE_ON_BROKEN_PIPE_KILL="-Zon-broken-pipe=kill" LIBC_CHECK_CFG="1" LZMA_API_STATIC="1" RANLIB_x86_64_unknown_linux_gnu="ar s" REAL_LIBRARY_PATH="BUILD/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib:BUILD/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64" REAL_LIBRARY_PATH_VAR="LD_LIBRARY_PATH" RUSTBUILD_NATIVE_DIR="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/native" RUSTC="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/bootstrap/debug/rustc" RUSTC_ALLOW_FEATURES="binary-dep-depinfo,proc_macro_span,proc_macro_span_shrink,proc_macro_diagnostic" RUSTC_BOOTSTRAP="1" RUSTC_BREAK_ON_ICE="1" RUSTC_CARGO_REGISTRY_SRC_TO_REMAP="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/vendor=/rust/deps" RUSTC_DEBUGINFO_MAP="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src=/rustc/1159e78c4747b02ef996e55082b704c09b970588" RUSTC_ERROR_METADATA_DST="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/tmp/extended-error-metadata" RUSTC_HOST_FLAGS="-Clinker=BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/wrapper/build-rust-ccld" RUSTC_INSTALL_BINDIR="bin" RUSTC_LIBDIR="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/recipe-sysroot-native/usr/lib" RUSTC_LINK_STD_INTO_RUSTC_DRIVER="1" RUSTC_LINT_FLAGS="-Wrust_2018_idioms -Wunused_lifetimes -Dwarnings" RUSTC_REAL="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/recipe-sysroot-native/usr/bin/rustc" RUSTC_SNAPSHOT="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/recipe-sysroot-native/usr/bin/rustc" RUSTC_SNAPSHOT_LIBDIR="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/recipe-sysroot-native/usr/lib" RUSTC_STAGE="1" RUSTC_SYSROOT="/srv/pokybuild/yocto-worker/musl-qemuarm64build/build/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot" RUSTC_TLS_MODEL_INITIAL_EXEC="1" RUSTC_VERBOSE="1" RUSTC_WRAPPER="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/bootstrap/debug/rustc" RUSTDOC="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/bootstrap/debug/rustdoc" RUSTDOCFLAGS="--remap-path-prefix=BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0=/usr/src/debug/rust/1.90.0 -Clink-arg=-lz -Clink-arg=-lzstd --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options -Dwarnings -Wrustdoc::invalid_codeblock_attributes --crate-version 1.90.0\t(1159e78c4\t2025-09-14)\t(built\tfrom\ta\tsource\ttarball) -Clinker=BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/wrapper/build-rust-ccld" RUSTDOC_REAL="/path/to/nowhere/rustdoc/not/required" RUSTFLAGS="--remap-path-prefix=BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0=/usr/src/debug/rust/1.90.0 -Clink-arg=-lz -Clink-arg=-lzstd --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options -Zmacro-backtrace -Csplit-debuginfo=off -Clink-args=-Wl,-z,origin -Clink-args=-Wl,-rpath,$ORIGIN/../lib -Alinker-messages -Zunstable-options -Clto=off" RUST_TEST_THREADS="16" SYSROOT="BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot" __CARGO_DEFAULT_LIB_METADATA="bootstrap1.90.0" "BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/recipe-sysroot-native/usr/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "16" "-Zroot-dir=BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src" "-v" "--frozen" "--release" "--manifest-path" "BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/tools/llvm-bitcode-linker/Cargo.toml" "--message-format" "json-render-diagnostics" (failure_mode=Exit) INFO prepare_target{force=false package_id=llvm-bitcode-linker v0.0.1 (BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/tools/llvm-bitcode-linker) target="llvm_bitcode_linker"}: cargo::core::compiler::fingerprint: fingerprint error for llvm-bitcode-linker v0.0.1 (BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/tools/llvm-bitcode-linker)/Build/TargetInner { name_inferred: true, ..: lib_target("llvm_bitcode_linker", ["lib"], "BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/tools/llvm-bitcode-linker/src/lib.rs", Edition2021) } INFO prepare_target{force=false package_id=llvm-bitcode-linker v0.0.1 (BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/tools/llvm-bitcode-linker) target="llvm_bitcode_linker"}: cargo::core::compiler::fingerprint: err: failed to read `BUILD/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/bootstrap-tools/x86_64-unknown-linux-gnu/release/.fingerprint/llvm-bitcode-linker-8931e15fef6bfc92/lib-llvm_bitcode_linker` I replaced the build paths with BUILD and noticed that RUSTC_SYSROOT is set incorrectly: RUSTC_SYSROOT="/srv/pokybuild/yocto-worker/musl-qemuarm64build/build/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot" This is probably not the cause, but still wrong.
I investigated this a bit on Friday, I did not find much. But I note increasing the stack size does not fix the issue: all it does is, every-time you increase it, the error message suggest to increase it even more.
A variety of workers so not specific to a host distro. My hunch/worst nightmare is that something horrible ended up in a test build that has poisoned some sstate.
reproducible debian12-vk-3 master completed at 2025-11-28 03:24:51+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251128-r9y5emh6 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251128-r9y5emh6/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251128-r9y5emh6/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2954/steps/13/logs/stdio genericarm64 fedora42-vk-1 master completed at 2025-11-28 02:15:31+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2791/steps/12/logs/stdio genericarm64-alt ubuntu2404-vk-1 master completed at 2025-11-28 01:48:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2822/steps/12/logs/stdio genericx86 fedora41-vk-1 master completed at 2025-11-28 01:32:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2795/steps/12/logs/stdio genericx86-64 debian12-vk-9 master completed at 2025-11-28 01:47:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/2804/steps/12/logs/stdio genericx86-64-alt ubuntu2204-vk-2 master completed at 2025-11-28 01:53:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2790/steps/12/logs/stdio genericx86-alt debian12-vk-5 master completed at 2025-11-28 01:35:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2784/steps/12/logs/stdio meta-intel fedora40-vk-2 master completed at 2025-11-28 02:25:56+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2634/steps/13/logs/stdio multilib ubuntu2404-vk-3 master completed at 2025-11-28 02:20:09+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2783/steps/24/logs/stdio musl-qemuarm64 ubuntu2504-vk-1 master completed at 2025-11-28 02:14:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1533/steps/12/logs/stdio musl-qemux86-64 debian12-vk-6 master completed at 2025-11-28 02:13:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2848/steps/12/logs/stdio no-x11 debian13-vk-1 master completed at 2025-11-28 03:05:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2788/steps/12/logs/stdio qemuarm-alt debian12-vk-4 master completed at 2025-11-28 01:46:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2776/steps/13/logs/stdio qemuarm-tc debian12-vk-7 master completed at 2025-11-28 01:58:30+00:00 https://valkyrie.yocto.io/pub/non-release/20251128-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2720/steps/13/logs/stdio qemuarm64 debian11-vk-3 master completed at 2025-11-28 02:25:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2796/steps/13/logs/stdio qemuarm64-alt ubuntu2404-vk-2 master completed at 2025-11-28 01:43:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2776/steps/13/logs/stdio qemuarm64-tc fedora40-vk-1 master completed at 2025-11-28 02:47:13+00:00 https://valkyrie.yocto.io/pub/non-release/20251128-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2718/steps/13/logs/stdio qemuarmv5 debian12-vk-7 master completed at 2025-11-28 01:55:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2633/steps/13/logs/stdio qemuriscv64 ubuntu2204-vk-2 master completed at 2025-11-28 02:10:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/761/steps/13/logs/stdio qemuriscv64-tc opensuse155-vk-2 master completed at 2025-11-28 01:30:15+00:00 https://valkyrie.yocto.io/pub/non-release/20251128-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/750/steps/14/logs/stdio qemux86-64 opensuse156-vk-1 master completed at 2025-11-28 02:02:30+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2864/steps/14/logs/stdio qemux86-64-alt ubuntu2404-vk-1 master completed at 2025-11-28 01:51:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2757/steps/13/logs/stdio qemux86-64-tc debian11-vk-3 master completed at 2025-11-28 03:38:43+00:00 https://valkyrie.yocto.io/pub/non-release/20251128-8/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2733/steps/13/logs/stdio qemux86-alt debian12-vk-2 master completed at 2025-11-28 01:38:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2765/steps/13/logs/stdio qemux86-tc debian11-vk-1 master completed at 2025-11-28 03:15:46+00:00 https://valkyrie.yocto.io/pub/non-release/20251128-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2705/steps/13/logs/stdio qemux86-world fedora40-vk-2 master completed at 2025-11-28 03:02:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2800/steps/12/logs/stdio qemux86-64 fedora39-vk-1 master completed at 2025-11-28 08:08:53+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2866/steps/16/logs/stdio beaglebone debian12-vk-4 master completed at 2025-11-28 08:09:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2806/steps/12/logs/stdio qemux86-64 ubuntu2204-vk-1 master completed at 2025-11-28 09:08:23+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2868/steps/16/logs/stdio qemux86-64 debian13-vk-1 master completed at 2025-11-28 10:02:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2870/steps/16/logs/stdio qemux86-64 fedora39-vk-1 master completed at 2025-11-28 10:51:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2872/steps/13/logs/stdio beaglebone debian12-vk-2 master completed at 2025-11-28 10:47:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2809/steps/12/logs/stdio beaglebone-alt fedora40-vk-1 master completed at 2025-11-28 10:40:38+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2794/steps/12/logs/stdio qemux86-64 debian11-vk-3 master completed at 2025-11-28 11:24:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2875/steps/13/logs/stdio reproducible debian12-vk-8 master completed at 2025-11-29 02:23:29+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251129-msi1kbrt https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251129-msi1kbrt/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2959/steps/13/logs/stdio beaglebone fedora40-vk-2 master completed at 2025-11-29 01:23:08+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2811/steps/12/logs/stdio beaglebone-alt ubuntu2404-vk-3 master completed at 2025-11-29 01:10:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2796/steps/12/logs/stdio genericx86-64 debian12-vk-3 master completed at 2025-11-29 01:22:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/2809/steps/12/logs/stdio genericx86-64-alt debian11-vk-3 master completed at 2025-11-29 01:11:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2795/steps/12/logs/stdio genericx86-alt fedora40-vk-1 master completed at 2025-11-29 01:11:53+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2789/steps/12/logs/stdio meta-intel debian12-vk-7 master completed at 2025-11-29 01:24:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2639/steps/13/logs/stdio multilib fedora39-vk-1 master completed at 2025-11-29 01:23:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2788/steps/24/logs/stdio musl-qemuarm64 ubuntu2204-vk-2 master completed at 2025-11-29 01:24:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1538/steps/12/logs/stdio musl-qemux86 fedora41-vk-1 master completed at 2025-11-29 01:45:30+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/2818/steps/12/logs/stdio musl-qemux86-64 fedora40-vk-4 master completed at 2025-11-29 01:21:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2853/steps/12/logs/stdio no-x11 ubuntu2204-vk-3 master completed at 2025-11-29 01:20:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2793/steps/15/logs/stdio qemuarm-tc debian12-vk-6 master completed at 2025-11-29 01:52:09+00:00 https://valkyrie.yocto.io/pub/non-release/20251129-12/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2725/steps/13/logs/stdio qemuarm64 ubuntu2204-vk-1 master completed at 2025-11-29 01:39:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2801/steps/17/logs/stdio qemuarm64-alt ubuntu2204-vk-4 master completed at 2025-11-29 01:15:12+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2781/steps/14/logs/stdio qemuarm64-tc ubuntu2404-vk-2 master completed at 2025-11-29 02:13:47+00:00 https://valkyrie.yocto.io/pub/non-release/20251129-12/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2723/steps/13/logs/stdio qemuriscv64 ubuntu2204-vk-2 master completed at 2025-11-29 01:35:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/765/steps/14/logs/stdio qemuriscv64-tc fedora40-vk-3 master completed at 2025-11-29 02:04:58+00:00 https://valkyrie.yocto.io/pub/non-release/20251129-12/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/754/steps/13/logs/stdio qemux86-64 fedora39-vk-1 master completed at 2025-11-29 01:39:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2877/steps/17/logs/stdio qemux86-64-alt debian12-vk-8 master completed at 2025-11-29 01:21:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2762/steps/14/logs/stdio qemux86-64-tc debian12-vk-4 master completed at 2025-11-29 03:05:08+00:00 https://valkyrie.yocto.io/pub/non-release/20251129-12/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2738/steps/13/logs/stdio qemux86-alt fedora40-vk-3 master completed at 2025-11-29 01:19:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2770/steps/14/logs/stdio qemux86-world debian12-vk-1 master completed at 2025-11-29 01:18:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2805/steps/12/logs/stdio qemux86-world-alt ubuntu2504-vk-1 master completed at 2025-11-29 02:10:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2639/steps/12/logs/stdio beaglebone debian12-vk-3 master completed at 2025-11-30 01:22:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2814/steps/12/logs/stdio genericarm64 debian12-vk-1 master completed at 2025-11-30 01:25:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2799/steps/12/logs/stdio genericarm64-alt fedora40-vk-4 master completed at 2025-11-30 01:13:29+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2830/steps/12/logs/stdio genericx86-64-alt debian11-vk-1 master completed at 2025-11-30 01:13:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2798/steps/12/logs/stdio meta-intel ubuntu2204-vk-3 master completed at 2025-11-30 01:25:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2642/steps/13/logs/stdio multilib debian12-vk-6 master completed at 2025-11-30 01:22:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2791/steps/24/logs/stdio musl-qemuarm64 ubuntu2404-vk-3 master completed at 2025-11-30 01:20:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1541/steps/12/logs/stdio musl-qemux86 fedora41-vk-1 master completed at 2025-11-30 01:17:16+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/2821/steps/12/logs/stdio musl-qemux86-64 ubuntu2404-vk-1 master completed at 2025-11-30 01:21:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2856/steps/12/logs/stdio no-x11 debian12-vk-9 master completed at 2025-11-30 01:20:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2796/steps/15/logs/stdio qemuarm ubuntu2204-vk-4 master completed at 2025-11-30 01:43:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2831/steps/14/logs/stdio qemuarm-alt debian12-vk-2 master completed at 2025-11-30 01:18:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2784/steps/14/logs/stdio qemuarm-tc debian12-vk-8 master completed at 2025-11-30 01:52:02+00:00 https://valkyrie.yocto.io/pub/non-release/20251130-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2728/steps/13/logs/stdio qemuarm64 fedora40-vk-2 master completed at 2025-11-30 01:45:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2804/steps/17/logs/stdio qemuarm64-alt debian11-vk-1 master completed at 2025-11-30 01:16:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2784/steps/14/logs/stdio qemuarm64-tc debian12-vk-1 master completed at 2025-11-30 02:13:21+00:00 https://valkyrie.yocto.io/pub/non-release/20251130-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2726/steps/13/logs/stdio qemuriscv64 alma8-vk-2 master completed at 2025-11-30 01:33:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/768/steps/15/logs/stdio qemuriscv64-tc ubuntu2204-vk-2 master completed at 2025-11-30 01:59:50+00:00 https://valkyrie.yocto.io/pub/non-release/20251130-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/757/steps/13/logs/stdio qemux86 debian12-vk-4 master completed at 2025-11-30 01:37:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2780/steps/17/logs/stdio qemux86-64 ubuntu2404-vk-2 master completed at 2025-11-30 01:40:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2880/steps/17/logs/stdio qemux86-64-alt debian13-vk-1 master completed at 2025-11-30 01:16:00+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2765/steps/14/logs/stdio qemux86-64-tc fedora40-vk-1 master completed at 2025-11-30 02:54:13+00:00 https://valkyrie.yocto.io/pub/non-release/20251130-8/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2741/steps/13/logs/stdio qemux86-alt ubuntu2204-vk-4 master completed at 2025-11-30 01:20:09+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2773/steps/14/logs/stdio qemux86-tc ubuntu2404-vk-1 master completed at 2025-11-30 03:38:27+00:00 https://valkyrie.yocto.io/pub/non-release/20251130-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2713/steps/13/logs/stdio qemux86-world rocky9-vk-2 master completed at 2025-11-30 01:16:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2808/steps/12/logs/stdio qemux86-world-alt debian12-vk-5 master completed at 2025-11-30 01:20:15+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2642/steps/12/logs/stdio reproducible ubuntu2204-vk-1 master completed at 2025-12-01 02:24:08+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251201-_1yxx_4h https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251201-_1yxx_4h/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2963/steps/13/logs/stdio beaglebone debian12-vk-2 master completed at 2025-12-01 01:22:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2815/steps/12/logs/stdio beaglebone-alt ubuntu2404-vk-3 master completed at 2025-12-01 01:11:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2800/steps/12/logs/stdio genericarm64 fedora41-vk-1 master completed at 2025-12-01 01:21:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2800/steps/12/logs/stdio genericarm64-alt fedora42-vk-1 master completed at 2025-12-01 01:18:09+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2831/steps/12/logs/stdio genericx86-64-alt debian12-vk-8 master completed at 2025-12-01 01:12:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2799/steps/12/logs/stdio genericx86-alt fedora40-vk-4 master completed at 2025-12-01 01:14:29+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2793/steps/12/logs/stdio meta-intel debian13-vk-1 master completed at 2025-12-01 01:24:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2643/steps/13/logs/stdio multilib fedora40-vk-1 master completed at 2025-12-01 01:21:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2792/steps/24/logs/stdio musl-qemux86-64 debian12-vk-7 master completed at 2025-12-01 01:16:24+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2857/steps/12/logs/stdio no-x11 debian11-vk-3 master completed at 2025-12-01 01:21:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2797/steps/15/logs/stdio qemuarm debian12-vk-5 master completed at 2025-12-01 01:37:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2832/steps/14/logs/stdio qemuarm-alt debian12-vk-6 master completed at 2025-12-01 01:18:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2785/steps/14/logs/stdio qemuarm-tc rocky8-vk-1 master completed at 2025-12-01 01:20:32+00:00 https://valkyrie.yocto.io/pub/non-release/20251201-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2729/steps/14/logs/stdio qemuarm64-alt debian12-vk-7 master completed at 2025-12-01 01:14:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2785/steps/14/logs/stdio qemuarm64-tc fedora40-vk-4 master completed at 2025-12-01 02:16:49+00:00 https://valkyrie.yocto.io/pub/non-release/20251201-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2727/steps/13/logs/stdio qemuriscv64 opensuse156-vk-1 master completed at 2025-12-01 01:34:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/769/steps/15/logs/stdio qemuriscv64-tc fedora40-vk-2 master completed at 2025-12-01 02:05:10+00:00 https://valkyrie.yocto.io/pub/non-release/20251201-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/758/steps/13/logs/stdio qemux86 debian12-vk-3 master completed at 2025-12-01 01:38:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2781/steps/17/logs/stdio qemux86-64 debian12-vk-4 master completed at 2025-12-01 01:37:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2881/steps/17/logs/stdio qemux86-alt ubuntu2204-vk-4 master completed at 2025-12-01 01:20:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2774/steps/14/logs/stdio qemux86-tc debian12-vk-5 master completed at 2025-12-01 03:19:36+00:00 https://valkyrie.yocto.io/pub/non-release/20251201-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2714/steps/13/logs/stdio qemux86-world fedora39-vk-1 master completed at 2025-12-01 01:18:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2809/steps/12/logs/stdio qemux86-world-alt ubuntu2404-vk-2 master completed at 2025-12-01 01:18:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2643/steps/12/logs/stdio reproducible fedora40-vk-4 master completed at 2025-12-02 02:26:38+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251202-5u4spwks https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251202-5u4spwks/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2967/steps/13/logs/stdio beaglebone debian12-vk-8 master completed at 2025-12-02 01:20:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2818/steps/12/logs/stdio beaglebone-alt debian12-vk-6 master completed at 2025-12-02 01:10:01+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2803/steps/12/logs/stdio genericarm64 debian11-vk-3 master completed at 2025-12-02 01:21:56+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2803/steps/12/logs/stdio genericarm64-alt fedora40-vk-3 master completed at 2025-12-02 01:13:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2834/steps/12/logs/stdio genericx86-64 debian12-vk-2 master completed at 2025-12-02 01:22:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/2816/steps/12/logs/stdio genericx86-64-alt ubuntu2504-vk-1 master completed at 2025-12-02 01:10:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2802/steps/12/logs/stdio genericx86-alt debian13-vk-1 master completed at 2025-12-02 01:11:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2796/steps/12/logs/stdio meta-intel fedora39-vk-1 master completed at 2025-12-02 01:26:54+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2646/steps/13/logs/stdio multilib fedora40-vk-1 master completed at 2025-12-02 01:21:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2795/steps/24/logs/stdio musl-qemuarm64 fedora42-vk-1 master completed at 2025-12-02 01:30:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1545/steps/12/logs/stdio musl-qemux86 ubuntu2404-vk-2 master completed at 2025-12-02 01:18:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/2825/steps/12/logs/stdio musl-qemux86-64 ubuntu2204-vk-1 master completed at 2025-12-02 01:18:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2860/steps/12/logs/stdio no-x11 fedora40-vk-2 master completed at 2025-12-02 01:22:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2800/steps/15/logs/stdio qemuarm debian12-vk-3 master completed at 2025-12-02 01:33:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/2835/steps/14/logs/stdio qemuarm-tc debian12-vk-4 master completed at 2025-12-02 01:52:05+00:00 https://valkyrie.yocto.io/pub/non-release/20251202-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2732/steps/13/logs/stdio qemuarm64-alt ubuntu2504-vk-1 master completed at 2025-12-02 01:16:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2788/steps/14/logs/stdio qemuarm64-tc ubuntu2204-vk-4 master completed at 2025-12-02 02:11:39+00:00 https://valkyrie.yocto.io/pub/non-release/20251202-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2731/steps/13/logs/stdio qemuriscv64 debian12-vk-6 master completed at 2025-12-02 01:34:20+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/770/steps/14/logs/stdio qemuriscv64-tc fedora42-vk-1 master completed at 2025-12-02 02:40:23+00:00 https://valkyrie.yocto.io/pub/non-release/20251202-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/759/steps/13/logs/stdio qemux86 debian13-vk-1 master completed at 2025-12-02 01:35:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2784/steps/17/logs/stdio qemux86-64-alt fedora40-vk-3 master completed at 2025-12-02 01:14:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2769/steps/14/logs/stdio qemux86-alt ubuntu2204-vk-1 master completed at 2025-12-02 01:15:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2777/steps/14/logs/stdio qemux86-tc ubuntu2204-vk-3 master completed at 2025-12-02 03:06:20+00:00 https://valkyrie.yocto.io/pub/non-release/20251202-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2717/steps/13/logs/stdio qemux86-world debian12-vk-3 master completed at 2025-12-02 01:19:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2812/steps/12/logs/stdio qemux86-world-alt fedora41-vk-1 master completed at 2025-12-02 01:17:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2646/steps/12/logs/stdio
oe-selftest-debian ubuntu2404-vk-3 master completed at 2025-12-03 03:08:13+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2785/steps/15/logs/stdio beaglebone debian12-vk-8 master completed at 2025-12-03 01:21:12+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/2821/steps/12/logs/stdio beaglebone-alt debian12-vk-6 master completed at 2025-12-03 01:09:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2806/steps/12/logs/stdio build-appliance stream9-vk-1 master completed at 2025-12-03 01:18:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/2840/steps/14/logs/stdio genericarm64 debian12-vk-3 master completed at 2025-12-03 01:16:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2806/steps/12/logs/stdio genericarm64-alt opensuse156-vk-1 master completed at 2025-12-03 01:17:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/2837/steps/13/logs/stdio genericx86 debian12-vk-9 master completed at 2025-12-03 01:10:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2810/steps/12/logs/stdio genericx86-64 rocky8-vk-1 master completed at 2025-12-03 01:22:16+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/2819/steps/13/logs/stdio genericx86-64-alt rocky9-vk-3 master completed at 2025-12-03 01:10:51+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2805/steps/12/logs/stdio genericx86-alt ubuntu2504-vk-1 master completed at 2025-12-03 01:16:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2799/steps/12/logs/stdio meta-arm ubuntu2404-vk-1 master completed at 2025-12-03 01:15:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2690/steps/13/logs/stdio meta-intel ubuntu2204-vk-1 master completed at 2025-12-03 01:18:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2649/steps/13/logs/stdio multilib debian12-vk-1 master completed at 2025-12-03 01:25:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2798/steps/24/logs/stdio musl-qemuarm64 debian12-vk-4 master completed at 2025-12-03 01:20:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1548/steps/12/logs/stdio musl-qemux86 ubuntu2404-vk-2 master completed at 2025-12-03 01:19:01+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/2828/steps/12/logs/stdio musl-qemux86-64 ubuntu2204-vk-2 master completed at 2025-12-03 01:24:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/2863/steps/12/logs/stdio pkgman-non-rpm debian12-vk-2 master completed at 2025-12-03 01:20:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/2839/steps/12/logs/stdio qa-extras debian13-vk-1 master completed at 2025-12-03 01:31:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/99/builds/2796/steps/20/logs/stdio qa-extras2 fedora42-vk-1 master completed at 2025-12-03 01:34:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2789/steps/12/logs/stdio qemuarm-tc rocky9-vk-2 master completed at 2025-12-03 01:19:31+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2735/steps/13/logs/stdio qemuarm64 ubuntu2204-vk-3 master completed at 2025-12-03 01:39:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2811/steps/17/logs/stdio qemuarm64-alt rocky9-vk-3 master completed at 2025-12-03 01:13:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2791/steps/14/logs/stdio qemuarm64-tc fedora39-vk-1 master completed at 2025-12-03 02:11:12+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2734/steps/13/logs/stdio qemuarmv5 rocky8-vk-1 master completed at 2025-12-03 01:31:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2648/steps/15/logs/stdio qemuriscv64-tc rocky9-vk-1 master completed at 2025-12-03 02:40:48+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/761/steps/13/logs/stdio qemux86 ubuntu2204-vk-3 master completed at 2025-12-03 01:37:56+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2787/steps/17/logs/stdio qemux86-64 debian11-vk-1 master completed at 2025-12-03 01:38:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/2887/steps/14/logs/stdio qemux86-64-ptest ubuntu2204-vk-4 master completed at 2025-12-03 01:29:01+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2707/steps/12/logs/stdio qemux86-64-tc debian13-vk-1 master completed at 2025-12-03 02:57:09+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2767/steps/13/logs/stdio qemux86-alt opensuse156-vk-1 master completed at 2025-12-03 01:25:00+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2780/steps/15/logs/stdio qemux86-tc fedora40-vk-4 master completed at 2025-12-03 03:37:31+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2720/steps/13/logs/stdio qemux86-world ubuntu2204-vk-1 master completed at 2025-12-03 01:17:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/2815/steps/12/logs/stdio qemux86-world-alt debian12-vk-5 master completed at 2025-12-03 01:23:53+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2649/steps/12/logs/stdio wic fedora42-vk-1 master completed at 2025-12-03 01:26:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/15/builds/2764/steps/12/logs/stdio reproducible fedora39-vk-1 mathieu/master-next completed at 2025-12-03 11:14:44+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251203-34b_sfcx https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251203-34b_sfcx/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20251203-34b_sfcx/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/2971/steps/13/logs/stdio build-appliance debian11-vk-1 mathieu/master-next completed at 2025-12-03 08:33:08+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/2841/steps/14/logs/stdio
I wasn't able to reproduce the "do_test_compile" failure locally i.e. without using sstate-cache and the builds passed successfully. However, once I enabled the use of poky sstate-cache through the new bitbake-setup steps, the issue was reproduced locally. This suggests that the sstate-cache is corrupted due to some commit/config in it or some config is messing it up in bitbake. CC: Alex
Yes, I'm also trying to narrow down the offending sstate item. The primary candidate is llvm-native.
@Alex, thanks for working alone to narrow down the issue. How is sstate-cache populated in AB? (I am not sure how it works there) What is your method of narrowing it down? Should we git bisect the commits and check the builds at every bisect to segregate good and bad commits?
Typo correction: @Alex, thanks for working along* to narrow down the issue.
Basically I'm running two local builds side by side, one with poky sstate and one without (e.g. one that would build everything from scratch, and generate sstate locally). One should fail, the other should not. Then I'll take sstate pieces from the working build into the failing one, until it stops failing. Or in the other direction. I don't think a bisect would help, as the issue is not triggered by any particular commit, but rather by a particular autobuilder build that somehow poisoned sstate. It could be the same Fedora 43 problem, or it could be just a random fail, it's very difficult to trace it back now, because sstate items do not contain pointers to builds that generated them. Sometimes you can identify gcc version or build paths from the binaries inside the sstate archive.
Thanks for the pointers. I will update here if I find anything further.
I can confirm, the offending sstate item is: universal/2e/e4/sstate:llvm-native:x86_64-linux:21.1.6:r1:x86_64:14:2ee49ee8e33b6040e8bab4cc89b182ff5a09b124320ceff39e332915d41d973d_populate_sysroot.tar.zst The timestamps inside the object are Nov 27 13:36 and the files refer to 'running gcc 15.2.1 20250808'. It all does seem to come from that same private Fedora 43 build. I need to step away from keyboard now for a few hours!
I moved the broken sstate item into /srv/autobuilder/valkyrie.yocto.io/pub/shared-failure-data/fedora43-sstate/ I also re-started the nightly a-full to see how far it gets: https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2817
I stopped that build as more issues started popping up. The fedora43 sstate contamination is now tracked in its own bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=16087
qemux86-tc fedora40-vk-2 master completed at 2025-12-03 18:08:52+00:00 https://valkyrie.yocto.io/pub/non-release/20251203-88/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2724/steps/13/logs/stdio
build-appliance alma8-vk-1 mathieu/master-next completed at 2026-01-06 10:02:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3010/steps/15/logs/stdio oe-selftest-debian ubuntu2404-vk-3 mathieu/master-next completed at 2026-01-06 14:39:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2970/steps/15/logs/stdio oe-selftest-fedora fedora42-vk-1 mathieu/master-next completed at 2026-01-06 14:59:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2859/steps/15/logs/stdio reproducible fedora40-vk-4 mathieu/master-next completed at 2026-01-06 14:28:04+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-by_tmwxh https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-by_tmwxh/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3149/steps/13/logs/stdio qemuarm-tc debian11-vk-1 mathieu/master-next completed at 2026-01-06 13:50:17+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-74/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2910/steps/13/logs/stdio qemuarm64-tc debian12-vk-2 mathieu/master-next completed at 2026-01-06 14:06:17+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-74/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2903/steps/13/logs/stdio qemuriscv64-tc ubuntu2204-vk-4 mathieu/master-next completed at 2026-01-06 14:00:37+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-74/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/882/steps/13/logs/stdio qemux86-alt fedora41-vk-1 mathieu/master-next-test1&master&mathieu/master-next completed at 2026-01-06 14:53:29+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2949/steps/13/logs/stdio qemux86-alt ubuntu2404-vk-2 mathieu/master-next-test2&master&mathieu/master-next completed at 2026-01-06 14:53:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2950/steps/13/logs/stdio qemux86-alt rocky9-vk-2 master&mathieu/master-next&mathieu/master-next-test3 completed at 2026-01-06 14:51:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2951/steps/13/logs/stdio I had all of these failures on my branch, probably something bad hit the autobuilder sstate cache at some point. I managed to get out of it by reverting some unrelated commit in my branch to force rebuild. Hopefully I can remove this revert later. no-x11 debian11-vk-1 mathieu/master-next-test1&mathieu/master-next completed at 2026-01-06 16:38:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2985/steps/12/logs/stdio no-x11 fedora40-vk-3 mathieu/master-next&mathieu/master-next-test3 completed at 2026-01-06 16:40:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/25/builds/2986/steps/12/logs/stdio qemux86-alt fedora40-vk-1 master&mathieu/master-next&mathieu/master-next-test4 completed at 2026-01-06 15:40:17+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2952/steps/13/logs/stdio oe-selftest-fedora fedora40-vk-2 mathieu/master-next completed at 2026-01-06 17:53:09+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2860/steps/15/logs/stdio reproducible rocky8-vk-1 mathieu/master-next completed at 2026-01-06 17:39:11+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-_3miyym7 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-_3miyym7/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3150/steps/14/logs/stdio qemuarm-tc rocky9-vk-3 mathieu/master-next completed at 2026-01-06 17:01:18+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2911/steps/13/logs/stdio qemuarm64-tc fedora39-vk-1 mathieu/master-next completed at 2026-01-06 17:20:18+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2904/steps/13/logs/stdio qemuriscv64-tc debian12-vk-7 mathieu/master-next completed at 2026-01-06 17:11:18+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/883/steps/13/logs/stdio qemux86-64-tc debian13-vk-1 mathieu/master-next completed at 2026-01-07 11:37:04+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2952/steps/13/logs/stdio qemux86-tc fedora40-vk-1 mathieu/master-next completed at 2026-01-07 10:59:44+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2912/steps/13/logs/stdio qemuriscv64-tc debian12-vk-3 mathieu/master-next completed at 2026-01-06 18:38:20+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/884/steps/13/logs/stdio qemuarm-tc fedora41-vk-1 mathieu/master-next completed at 2026-01-06 18:30:29+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2912/steps/13/logs/stdio reproducible fedora42-vk-1 mathieu/master-next completed at 2026-01-06 19:51:23+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-5n1s0uve https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260106-5n1s0uve/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3151/steps/13/logs/stdio qemuarm64-tc alma8-vk-1 mathieu/master-next completed at 2026-01-06 18:48:20+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-97/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2905/steps/14/logs/stdio oe-selftest-debian debian11-vk-1 mathieu/master-next completed at 2026-01-06 21:50:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2972/steps/15/logs/stdio oe-selftest-fedora fedora42-vk-1 mathieu/master-next completed at 2026-01-06 22:26:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2861/steps/15/logs/stdio qemuarm64-tc debian12-vk-7 mathieu/master-next-test1&master&mathieu/master-next completed at 2026-01-06 21:40:17+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-116/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2906/steps/13/logs/stdio qemuarm64-tc rocky9-vk-2 mathieu/master-next-test2&master&mathieu/master-next completed at 2026-01-07 03:36:51+00:00 https://valkyrie.yocto.io/pub/non-release/20260106-119/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2907/steps/13/logs/stdio qemuarm64-tc debian12-vk-9 mathieu/master-next-test1&master&mathieu/master-next completed at 2026-01-07 08:02:36+00:00 https://valkyrie.yocto.io/pub/non-release/20260107-44/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2913/steps/13/logs/stdio qemuarm64-tc opensuse156-vk-1 mathieu/master-next-test2&master&mathieu/master-next completed at 2026-01-07 08:02:38+00:00 https://valkyrie.yocto.io/pub/non-release/20260107-47/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2914/steps/14/logs/stdio
So it's hitting master now :( Hopefully it will go away the next time some patches are merged. beaglebone alma8-vk-1 master completed at 2026-01-09 01:56:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/3015/steps/13/logs/stdio beaglebone-alt debian12-vk-2 master completed at 2026-01-09 01:49:20+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/2999/steps/12/logs/stdio This one is a bit different, but I suspect it's still the same thing. build-appliance ubuntu2404-vk-2 master completed at 2026-01-09 02:03:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3031/steps/14/logs/stdio buildtools opensuse155-vk-2 master completed at 2026-01-09 01:56:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3039/steps/13/logs/stdio genericarm64 debian12-vk-5 master completed at 2026-01-09 01:53:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2999/steps/12/logs/stdio genericarm64-alt fedora40-vk-1 master completed at 2026-01-09 02:40:01+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/3029/steps/12/logs/stdio genericx86 debian12-vk-7 master completed at 2026-01-09 01:53:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/3004/steps/12/logs/stdio genericx86-64 debian12-vk-4 master completed at 2026-01-09 01:41:35+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/3011/steps/12/logs/stdio genericx86-64-alt rocky8-vk-1 master completed at 2026-01-09 01:47:38+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/2997/steps/13/logs/stdio genericx86-alt alma9-vk-2 master completed at 2026-01-09 01:28:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2991/steps/12/logs/stdio meta-arm fedora40-vk-4 master completed at 2026-01-09 02:13:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2886/steps/13/logs/stdio meta-intel fedora40-vk-2 master completed at 2026-01-09 01:52:09+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2842/steps/13/logs/stdio multilib opensuse156-vk-1 master completed at 2026-01-09 02:33:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2990/steps/19/logs/stdio musl-qemuarm64 ubuntu2204-vk-4 master completed at 2026-01-09 01:59:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1740/steps/12/logs/stdio musl-qemux86 alma8-vk-2 master completed at 2026-01-09 01:49:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/3022/steps/13/logs/stdio musl-qemux86-64 debian11-vk-1 master completed at 2026-01-09 02:21:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/3056/steps/12/logs/stdio pkgman-non-rpm debian12-vk-1 master completed at 2026-01-09 02:05:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/3037/steps/12/logs/stdio qa-extras debian12-vk-3 master completed at 2026-01-09 02:07:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/99/builds/2991/steps/20/logs/stdio qa-extras2 debian12-vk-6 master completed at 2026-01-09 02:30:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2983/steps/12/logs/stdio qemuarm opensuse156-vk-1 master completed at 2026-01-09 02:23:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/3033/steps/15/logs/stdio qemuarm-alt fedora39-vk-1 master completed at 2026-01-09 01:42:35+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2980/steps/14/logs/stdio qemuarm-tc debian13-vk-2 master completed at 2026-01-09 01:53:26+00:00 https://valkyrie.yocto.io/pub/non-release/20260109-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2931/steps/13/logs/stdio qemuarm64 alma9-vk-2 master completed at 2026-01-09 02:05:17+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/3000/steps/14/logs/stdio qemuarm64-alt debian12-vk-5 master completed at 2026-01-09 01:54:19+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2979/steps/14/logs/stdio qemuarm64-tc ubuntu2404-vk-1 master completed at 2026-01-09 02:32:51+00:00 https://valkyrie.yocto.io/pub/non-release/20260109-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2937/steps/13/logs/stdio qemuarmv5 fedora42-vk-1 master completed at 2026-01-09 02:25:40+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2837/steps/14/logs/stdio qemuriscv64 ubuntu2204-vk-1 master completed at 2026-01-09 02:12:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/908/steps/14/logs/stdio qemuriscv64-ptest debian12-vk-1 master completed at 2026-01-09 02:49:04+00:00 https://valkyrie.yocto.io/pub/non-release/20260109-8/testresults/qemuriscv64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/932/steps/12/logs/stdio qemuriscv64-tc alma8-vk-1 master completed at 2026-01-09 02:14:08+00:00 https://valkyrie.yocto.io/pub/non-release/20260109-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/898/steps/14/logs/stdio qemux86 rocky9-vk-1 master completed at 2026-01-09 02:23:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2976/steps/14/logs/stdio qemux86-64 fedora40-vk-2 master completed at 2026-01-09 02:12:23+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3076/steps/14/logs/stdio qemux86-64-alt debian13-vk-1 master completed at 2026-01-09 01:23:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2961/steps/14/logs/stdio qemux86-64-ltp debian12-vk-8 master completed at 2026-01-09 01:41:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/63/builds/2820/steps/12/logs/stdio qemux86-64-ptest ubuntu2204-vk-3 master completed at 2026-01-09 02:04:35+00:00 https://valkyrie.yocto.io/pub/non-release/20260109-8/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2895/steps/12/logs/stdio qemux86-alt rocky9-vk-3 master completed at 2026-01-09 01:46:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2973/steps/14/logs/stdio qemux86-world debian11-vk-1 master completed at 2026-01-09 02:48:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3003/steps/12/logs/stdio qemux86-world-alt opensuse155-vk-2 master completed at 2026-01-09 02:48:57+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2840/steps/13/logs/stdio wic fedora40-vk-4 master completed at 2026-01-09 01:56:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/15/builds/2952/steps/12/logs/stdio oe-selftest-debian ubuntu2404-vk-1 master completed at 2026-01-09 03:07:19+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2991/steps/15/logs/stdio oe-selftest-fedora rocky9-vk-3 master completed at 2026-01-09 03:08:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2882/steps/15/logs/stdio reproducible fedora40-vk-3 master completed at 2026-01-09 02:26:56+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260109-qxx0kulk https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260109-qxx0kulk/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260109-qxx0kulk/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3170/steps/13/logs/stdio
I was getting worried that we have another fedora 43 sstate contamination, but this looks different, as the issue happens even in a reproducible test in the B step (building without sstate).
The bad news of course is that adding commits to invalidate sstate is probably not going to help :(
(In reply to Alexander Kanavin from comment #21) > The bad news of course is that adding commits to invalidate sstate is > probably not going to help :( Now that you say it, it's a bit strange. I had following two builds and then I did manage to have a working build by reverting some commit in my branch. https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3151/steps/13/logs/stdio https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3150/steps/14/logs/stdio Yet I do agree with you that it should not have had any impact, as sstate is not used during the B step of the reproducibility test. The said commit was "xz: upgrade 5.8.1 -> 5.8.2": https://git.openembedded.org/openembedded-core/commit/?id=4fa9f8b37e9f75f75e008aca03f8a8543e11de3e . I was able to remove this revert a few days later and I still have successful builds. This commit was merged in master yesterday.
It's also slightly odd that the logs do not print the actual error, only that the task failed. Or am I missing it?
This doesn't block M1 it seems to move to M2. Yash has anyone been able to reproduce the error in a personal YP AB build or on our server ? Do you have any comments on Alex's question about logs?
(In reply to Randy MacLeod from comment #24) > This doesn't block M1 it seems to move to M2. > Yash has anyone been able to reproduce the error in a personal YP AB build > or on our server ? Last week, my colleagues triggered multiple *-tc builds to investigate and reproduce the glibc test hanging issue on AB. During these tests, the rust oe-selftest was also executed and no segfaults were observed in the rust tests. Today, after rebasing and pulling the latest master (commit 235e6d49e5, origin/master, origin/HEAD, libpcap: upgrade 1.10.5 → 1.10.6 which includes xz upgrade), I was able to reproduce the segfault issue on our server. As mentioned by Mathieu, reverting the commit “xz: upgrade 5.8.1 → 5.8.2” resulted in successful builds with no errors. This suggests that the issue may be intermittent or something related to that commit. > Do you have any comments on Alex's question about logs? The recent AB failure links (last 4-5) shared by Mathieu were generated from reproducibility tests, which only detect and report package or binary differences. The actual error here is the segfault which occurs during the rust build itself and can be directly reproduced using "bitbake rust -c test_compile". The actual and complete logs will be available when executing "do_test_compile" instead of reproducibility test. This is more efficient than executing the full reproducibility tests or the rust oe-selftest.
A errors link that filters by cargo-native failures: https://errors.yoctoproject.org/Errors/Search/?filter=cargo-native:%20do_compile&type=failure Thinking this should be a high...
Grabbed a recent build and re-ran it with -vv -j1 to get a neater output, the failure is still: Compiling libc v0.2.176 Running `CARGO=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/cargo-1.91.0-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/sources/rustc-1.92.0-src/vendor/libc-0.2.176 CARGO_MANIFEST_PATH=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/sources/rustc-1.92.0-src/vendor/libc-0.2.176/Cargo.toml CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_RUST_VERSION=1.63 CARGO_PKG_VERSION=0.2.176 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=176 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/build/target/release/deps rustc --crate-name build_script_build --edition=2021 /srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/sources/rustc-1.92.0-src/vendor/libc-0.2.176/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=150 --crate-type bin --emit=dep-info,link -C embed-bitcode=no '--allow=clippy::used_underscore_binding' --allow=unused_qualifications '--warn=clippy::unnecessary_semicolon' '--allow=clippy::unnecessary_cast' '--allow=clippy::uninlined_format_args' '--warn=clippy::ptr_as_ptr' '--allow=clippy::non_minimal_cfg' '--allow=clippy::missing_safety_doc' '--warn=clippy::map_unwrap_or' '--warn=clippy::manual_assert' '--allow=clippy::identity_op' '--warn=clippy::explicit_iter_loop' '--allow=clippy::expl_impl_clone_on_copy' -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="std"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("align", "const-extern-fn", "default", "extra_traits", "rustc-dep-of-std", "rustc-std-workspace-core", "std", "use_std"))' -C metadata=cbee25a640e401c4 -C extra-filename=-07e06bad1ad999fc --out-dir /srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/build/target/release/build/libc-07e06bad1ad999fc -C linker=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/wrapper/target-rust-ccld -C strip=debuginfo -L dependency=/srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/build/target/release/deps --cap-lints warn` error: rustc interrupted by SIGSEGV, printing backtrace /srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so(+0xd22ba5) [0x7f6443322ba5] /srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6(+0x3bba0) [0x7f64422f3ba0] /srv/pokybuild/yocto-worker/qemuriscv64-ptest/build/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so(+0x6be8cc0) [0x7f64491e8cc0] note: we would appreciate a report at https://github.com/rust-lang/rust help: you can increase rustc's stack size by setting RUST_MIN_STACK=67108864 error: could not compile `libc` (build script) (increasing RUST_MIN_STACK didn't give any more information)
The previous round of fails was traced back to llvm-native produced by fedora 43. I just reproduced this fail with yocto sstate, and llvm-native in it comes from ubuntu 25.10. My working hypothesis is that both these distros have a native toolchain that produces sstate that is problematic on other hosts. Investigation ongoing.
Alex, could you point me at the sstate object you're talking about? I'd like to look at it myself. Also, does it work if you rebuild on Ubuntu 25.10 using the same sstate? From what you've said I would expect it to succeed there and fail on older distros.
It's not llvm-native. The offending sstate item is universal/e6/2c/sstate:rust-native:x86_64-linux:1.92.0:r0:x86_64:14:e62cdcb7dcc52596ebfe8dcbe964e75ac4ce0d32d3120b9727e5e9ca4e233e84_populate_sysroot.tar.zst and it was produced here: https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3039 (if you unpack the sstate archive, the binaries have buildtools build paths in them and timestamps that match) Note that this build takes llvm-native from sstate, then builds rust-native from scratch and then fails in cargo-native. So I would try building from scratch on opensuse 155 next, and if that succeeds try again, but take llvm-native from sstate instead. I have to stop now, it's getting late :)
Progress! I reproduced the broken rust-native item locally: 1. Take the ubuntu 25.10 produced llvm-native from sstate: universal/ff/a2/sstate:llvm-native:x86_64-linux:21.1.8:r1:x86_64:14:ffa2b99e1ac7557231113d8060ff131e4ff93e440bb0de4c03cdc5ad805c1c58_populate_sysroot.tar.zst 2. Build rust-native (it will succeed) 2. Build cargo-native -> kaboom. The next step is to build llvm-native separately on ubuntu 25.10, and try again with that on a different distro. That will confirm that distro as problematic.
Hi Alex, great news that we're narrowing things down! What happens if you take the Ubuntu 22.10 llvm-native sstate, then use that to build rust-native & cargo-native on Ubuntu 22.10? I would hope that it works when we use it on the distro it's built on at least, but it'd be good to confirm.
25.10. Not 22.10. I ran a cargo-native build from scratch on 25.10, and it succeeded. Then I took the llvm-native sstate item from that build into a different distro (debian 13, but any older one should do), built rust-native there, then cargo-native, and cargo-native failed. So this confirms that llvm-native from ubuntu 25.10 is not suitable for building rust-native on older distros. What we should do: - pause 25.10 workers - move problematic sstate (llvm-native, cargo-native) into a 'problematic sstate' location as was done for fedora 43 contaminated items: https://valkyrie.yocto.io/pub/shared-failure-data/
I took the liberty of doing both these actions: - ubuntu 25.10 paused - problematic sstate moved to https://valkyrie.yocto.io/pub/shared-failure-data/ubuntu2510-sstate/ Fingers crossed that this addresses the ongoing fails. But exactly what the problem is in the binaries still needs investigating.
I ran two more experiments: 1. build llvm-native on 25.10 under buildtools and transfer llvm sstate to another distro and build cargo-native there -> no fails 2. build llvm-native on 25.10 without buildtools and run another cargo-native build on 25.10 using that llvm-native from sstate -> no fails. So the issue is specifically with the host toolchain doing something incompatible with older distros, or exposing a bug somewhere.
Alex, can add export RUST_BACKTRACE="full" to cargo's do_compile:prepend() to get rust to dump more information? Slightly longer backtraces might have a little more context. My spider sense is triggering with it being ubuntu2510 and llvm-native related, there were fun problems with static binaries + uninative + 2510 in the clang/llvm split that got resolved by not doing static linking at all. I wonder if another static binary has leaked into the build and it's crashing instead of writing a file, or something.
Not a lot more: error: rustc interrupted by SIGSEGV, printing backtrace /srv/storage/alex/yocto/build-64-test/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so(+0xd22f95) [0x7 f3822b22f95] /srv/storage/alex/yocto/build-64-test/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6(+0x3bba0) [0x7f3821af3ba0] /srv/storage/alex/yocto/build-64-test/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so(+0x6d35e80) [0x 7f3828b35e80]
oe-selftest-debian debian12-vk-5 master completed at 2026-01-12 09:39:20+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2996/steps/15/logs/stdio oe-selftest-fedora fedora40-vk-4 master completed at 2026-01-12 09:44:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2887/steps/15/logs/stdio oe-selftest-debian debian12-vk-7 master completed at 2026-01-12 09:39:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2998/steps/15/logs/stdio beaglebone opensuse155-vk-2 master completed at 2026-01-12 08:12:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/3020/steps/13/logs/stdio genericarm64 ubuntu2204-vk-4 master completed at 2026-01-12 08:07:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/3004/steps/12/logs/stdio genericx86 fedora39-vk-1 master completed at 2026-01-12 08:19:08+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/3009/steps/12/logs/stdio genericx86-64 rocky8-vk-1 master completed at 2026-01-12 08:09:28+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/3016/steps/13/logs/stdio meta-intel debian11-vk-3 master completed at 2026-01-12 08:14:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2847/steps/13/logs/stdio multilib debian12-vk-3 master completed at 2026-01-12 08:27:17+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2995/steps/18/logs/stdio pkgman-non-rpm debian12-vk-1 master completed at 2026-01-12 08:11:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/3042/steps/12/logs/stdio qemuarm fedora40-vk-1 master completed at 2026-01-12 08:47:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/3038/steps/14/logs/stdio qemuarm-tc debian12-vk-2 master completed at 2026-01-12 08:42:14+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-28/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2936/steps/13/logs/stdio qemuarm64-alt rocky8-vk-1 master completed at 2026-01-12 08:16:40+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2984/steps/15/logs/stdio qemuarm64-tc debian12-vk-7 master completed at 2026-01-12 09:04:42+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-28/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2942/steps/13/logs/stdio qemuarmv5 debian12-vk-4 master completed at 2026-01-12 08:29:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2842/steps/14/logs/stdio qemuriscv64 fedora39-vk-1 master completed at 2026-01-12 08:47:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/914/steps/14/logs/stdio qemuriscv64-ptest debian12-vk-8 master completed at 2026-01-12 09:21:01+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-28/testresults/qemuriscv64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/937/steps/12/logs/stdio qemuriscv64-tc debian12-vk-8 master completed at 2026-01-12 08:51:59+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-28/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/903/steps/13/logs/stdio qemux86 fedora40-vk-2 master completed at 2026-01-12 08:35:08+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2981/steps/14/logs/stdio qemux86-64 opensuse155-vk-2 master completed at 2026-01-12 08:35:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3081/steps/15/logs/stdio qemux86-world fedora41-vk-1 master completed at 2026-01-12 08:28:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3008/steps/12/logs/stdio qemux86-world debian12-vk-3 master completed at 2026-01-12 08:39:04+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3009/steps/12/logs/stdio beaglebone debian11-vk-3 master completed at 2026-01-12 08:18:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/3023/steps/12/logs/stdio genericarm64 ubuntu2404-vk-3 master completed at 2026-01-12 08:31:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/3006/steps/12/logs/stdio genericx86 ubuntu2510-vk-1 master completed at 2026-01-12 08:35:30+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/3010/steps/12/logs/stdio genericx86-64 fedora41-vk-1 master completed at 2026-01-12 08:18:42+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/3018/steps/12/logs/stdio meta-intel debian13-vk-2 master completed at 2026-01-12 08:27:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2848/steps/13/logs/stdio multilib fedora42-vk-1 master completed at 2026-01-12 08:30:15+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/2996/steps/18/logs/stdio pkgman-non-rpm ubuntu2404-vk-2 master completed at 2026-01-12 08:29:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/3044/steps/12/logs/stdio qemuriscv64-ptest debian13-vk-1 master completed at 2026-01-12 09:40:00+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-30/testresults/qemuriscv64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/938/steps/12/logs/stdio qemuarm ubuntu2204-vk-2 master completed at 2026-01-12 08:43:58+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/3039/steps/14/logs/stdio qemuriscv64 debian11-vk-3 master completed at 2026-01-12 08:40:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/916/steps/14/logs/stdio qemuarm64 debian13-vk-1 master completed at 2026-01-12 08:43:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/3007/steps/14/logs/stdio qemuarm64-alt debian12-vk-1 master completed at 2026-01-12 08:32:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2985/steps/14/logs/stdio qemuarm64-tc debian13-vk-1 master completed at 2026-01-12 09:26:52+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-30/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2944/steps/13/logs/stdio qemuarmv5 fedora40-vk-2 master completed at 2026-01-12 08:46:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2844/steps/14/logs/stdio qemuriscv64-tc opensuse156-vk-1 master completed at 2026-01-12 09:32:46+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-30/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/905/steps/14/logs/stdio qemux86 debian11-vk-1 master completed at 2026-01-12 08:56:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2982/steps/14/logs/stdio qemux86-64 ubuntu2404-vk-1 master completed at 2026-01-12 08:48:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3083/steps/14/logs/stdio qemux86-world ubuntu2404-vk-1 master completed at 2026-01-12 08:36:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3011/steps/12/logs/stdio oe-selftest-debian debian12-vk-7 master completed at 2026-01-13 02:25:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3006/steps/15/logs/stdio beaglebone ubuntu2404-vk-2 master completed at 2026-01-13 01:23:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/2/builds/3031/steps/12/logs/stdio genericarm64 debian12-vk-5 master completed at 2026-01-13 01:14:15+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/3013/steps/12/logs/stdio genericx86 opensuse156-vk-1 master completed at 2026-01-13 01:19:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/3018/steps/13/logs/stdio genericx86-64 debian12-vk-8 master completed at 2026-01-13 01:21:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/3025/steps/12/logs/stdio meta-intel fedora40-vk-2 master completed at 2026-01-13 01:23:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2856/steps/13/logs/stdio multilib fedora42-vk-1 master completed at 2026-01-13 01:24:53+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3004/steps/18/logs/stdio pkgman-non-rpm ubuntu2204-vk-4 master completed at 2026-01-13 01:35:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/3052/steps/12/logs/stdio qemuarm fedora40-vk-1 master completed at 2026-01-13 01:35:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/3048/steps/14/logs/stdio qemuarm64 ubuntu2204-vk-2 master completed at 2026-01-13 01:31:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/3015/steps/14/logs/stdio qemuarm64-alt fedora40-vk-4 master completed at 2026-01-13 01:14:38+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/2994/steps/14/logs/stdio qemuarm64-tc debian13-vk-2 master completed at 2026-01-13 02:08:35+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemuarm64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2952/steps/13/logs/stdio qemuarmv5 fedora40-vk-2 master completed at 2026-01-13 01:36:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/80/builds/2853/steps/14/logs/stdio qemuriscv64 debian12-vk-3 master completed at 2026-01-13 01:39:40+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/921/steps/14/logs/stdio qemuriscv64-ptest opensuse156-vk-1 master completed at 2026-01-13 02:34:13+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemuriscv64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/944/steps/13/logs/stdio qemuriscv64-tc debian12-vk-2 master completed at 2026-01-13 02:00:46+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/910/steps/13/logs/stdio qemux86 debian13-vk-2 master completed at 2026-01-13 01:36:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2991/steps/14/logs/stdio qemux86-64 fedora40-vk-3 master completed at 2026-01-13 01:32:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3092/steps/14/logs/stdio qemux86-world fedora41-vk-1 master completed at 2026-01-13 01:20:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3020/steps/12/logs/stdio beaglebone-alt debian12-vk-4 master completed at 2026-01-12 07:54:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/3004/steps/12/logs/stdio buildtools fedora40-vk-4 master completed at 2026-01-12 08:34:13+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3044/steps/12/logs/stdio genericarm64-alt debian11-vk-1 master completed at 2026-01-12 08:16:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/3034/steps/12/logs/stdio genericx86-64-alt fedora40-vk-3 master completed at 2026-01-12 07:56:31+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/3002/steps/12/logs/stdio genericx86-alt debian12-vk-7 master completed at 2026-01-12 07:57:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2996/steps/12/logs/stdio meta-arm debian12-vk-6 master completed at 2026-01-12 08:09:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2891/steps/13/logs/stdio musl-qemuarm64 debian13-vk-2 master completed at 2026-01-12 08:04:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1745/steps/12/logs/stdio musl-qemux86 fedora40-vk-2 master completed at 2026-01-12 08:09:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/3027/steps/12/logs/stdio musl-qemux86-64 debian12-vk-5 master completed at 2026-01-12 08:10:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/3061/steps/12/logs/stdio qemuarm-alt opensuse156-vk-1 master completed at 2026-01-12 08:11:10+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2985/steps/15/logs/stdio qemux86-alt debian12-vk-6 master completed at 2026-01-12 08:07:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2978/steps/14/logs/stdio qemux86-alt debian13-vk-1 master completed at 2026-01-12 08:05:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2979/steps/14/logs/stdio qemux86-world-alt debian12-vk-9 master completed at 2026-01-12 08:22:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2847/steps/12/logs/stdio wic fedora42-vk-1 master completed at 2026-01-12 08:30:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/15/builds/2958/steps/12/logs/stdio beaglebone-alt ubuntu2204-vk-4 master completed at 2026-01-12 08:10:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/3005/steps/12/logs/stdio buildtools opensuse156-vk-1 master completed at 2026-01-12 08:31:28+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3045/steps/13/logs/stdio buildtools ubuntu2204-vk-2 master completed at 2026-01-12 08:39:51+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3046/steps/12/logs/stdio genericarm64-alt debian12-vk-4 master completed at 2026-01-12 08:14:01+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/3035/steps/12/logs/stdio genericx86-64-alt debian13-vk-1 master completed at 2026-01-12 08:15:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/3003/steps/12/logs/stdio beaglebone-alt ubuntu2404-vk-2 master completed at 2026-01-12 08:10:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/3007/steps/12/logs/stdio buildtools fedora41-vk-1 master completed at 2026-01-12 08:26:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3049/steps/12/logs/stdio genericx86-alt alma8-vk-1 master completed at 2026-01-12 08:09:30+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/2997/steps/13/logs/stdio meta-arm ubuntu2510-vk-1 master completed at 2026-01-12 08:21:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2893/steps/13/logs/stdio musl-qemuarm64 ubuntu2404-vk-2 master completed at 2026-01-12 08:22:23+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1747/steps/12/logs/stdio musl-qemux86 debian12-vk-2 master completed at 2026-01-12 08:25:17+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/3029/steps/12/logs/stdio musl-qemux86-64 debian12-vk-6 master completed at 2026-01-12 08:40:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/3062/steps/12/logs/stdio qemuarm-tc opensuse155-vk-2 master completed at 2026-01-12 09:02:45+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-30/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2938/steps/14/logs/stdio qemux86-alt rocky8-vk-1 master completed at 2026-01-12 08:36:39+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2981/steps/15/logs/stdio qemux86-world-alt rocky8-vk-1 master completed at 2026-01-12 08:37:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2849/steps/13/logs/stdio wic opensuse155-vk-2 master completed at 2026-01-12 08:36:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/15/builds/2959/steps/13/logs/stdio beaglebone-alt fedora40-vk-1 master completed at 2026-01-13 01:10:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/10/builds/3014/steps/12/logs/stdio buildtools debian12-vk-3 master completed at 2026-01-13 01:26:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/43/builds/3056/steps/12/logs/stdio genericarm64-alt debian11-vk-3 master completed at 2026-01-13 01:25:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/22/builds/3043/steps/12/logs/stdio genericx86-64-alt debian12-vk-6 master completed at 2026-01-13 01:21:19+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/3013/steps/12/logs/stdio genericx86-alt ubuntu2204-vk-3 master completed at 2026-01-13 01:09:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/65/builds/3005/steps/12/logs/stdio meta-arm debian12-vk-2 master completed at 2026-01-13 01:16:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2902/steps/13/logs/stdio musl-qemuarm64 opensuse155-vk-2 master completed at 2026-01-13 01:29:53+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/111/builds/1755/steps/13/logs/stdio musl-qemux86 ubuntu2404-vk-3 master completed at 2026-01-13 01:23:47+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/6/builds/3037/steps/12/logs/stdio musl-qemux86-64 ubuntu2404-vk-1 master completed at 2026-01-13 01:13:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/3/builds/3071/steps/12/logs/stdio qemuarm-alt debian12-vk-6 master completed at 2026-01-13 01:21:07+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/74/builds/2995/steps/14/logs/stdio qemuarm-tc fedora39-vk-1 master completed at 2026-01-13 01:52:24+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemuarm-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/2946/steps/13/logs/stdio qemux86-alt ubuntu2404-vk-3 master completed at 2026-01-13 01:13:41+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2989/steps/14/logs/stdio qemux86-world-alt debian12-vk-8 master completed at 2026-01-13 01:21:33+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2856/steps/12/logs/stdio wic debian11-vk-3 master completed at 2026-01-13 01:25:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/15/builds/2967/steps/12/logs/stdio build-appliance ubuntu2204-vk-2 master completed at 2026-01-12 07:56:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3036/steps/14/logs/stdio qa-extras ubuntu2404-vk-1 master completed at 2026-01-12 08:12:16+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/99/builds/2996/steps/20/logs/stdio qa-extras2 alma9-vk-2 master completed at 2026-01-12 08:13:51+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2988/steps/12/logs/stdio qemux86-64-alt ubuntu2510-vk-1 master completed at 2026-01-12 08:04:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2966/steps/14/logs/stdio qemux86-64-ltp ubuntu2404-vk-2 master completed at 2026-01-12 07:56:21+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/63/builds/2825/steps/12/logs/stdio qemux86-64-ptest ubuntu2404-vk-3 master completed at 2026-01-12 08:18:30+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-28/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2900/steps/12/logs/stdio qemux86-64-alt debian11-vk-1 master completed at 2026-01-12 08:05:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2967/steps/14/logs/stdio qemux86-64-ptest debian12-vk-5 master completed at 2026-01-12 08:23:29+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-27/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2901/steps/12/logs/stdio build-appliance debian13-vk-2 master completed at 2026-01-12 08:15:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3037/steps/14/logs/stdio build-appliance rocky9-vk-2 master completed at 2026-01-12 08:10:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3038/steps/14/logs/stdio qemux86-64-ptest debian11-vk-1 master completed at 2026-01-12 08:36:49+00:00 https://valkyrie.yocto.io/pub/non-release/20260112-30/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2902/steps/12/logs/stdio qemux86-64-alt opensuse156-vk-1 master completed at 2026-01-12 08:16:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2968/steps/15/logs/stdio qa-extras rocky9-vk-2 master completed at 2026-01-12 08:31:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/99/builds/2997/steps/20/logs/stdio qa-extras2 rocky9-vk-3 master completed at 2026-01-12 08:40:56+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2990/steps/12/logs/stdio qemux86-64-ltp ubuntu2404-vk-3 master completed at 2026-01-12 08:20:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/63/builds/2828/steps/12/logs/stdio build-appliance alma8-vk-2 master completed at 2026-01-13 01:09:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/3046/steps/15/logs/stdio qa-extras debian12-vk-9 master completed at 2026-01-13 01:18:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/99/builds/3006/steps/20/logs/stdio qa-extras2 ubuntu2204-vk-3 master completed at 2026-01-13 01:31:48+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2998/steps/12/logs/stdio qemux86-64-alt fedora42-vk-1 master completed at 2026-01-13 01:12:19+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2977/steps/14/logs/stdio qemux86-64-ltp debian12-vk-7 master completed at 2026-01-13 01:09:18+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/63/builds/2836/steps/12/logs/stdio qemux86-64-ptest rocky8-vk-1 master completed at 2026-01-13 01:27:40+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemux86-64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2911/steps/13/logs/stdio oe-selftest-fedora rocky9-vk-1 master completed at 2026-01-12 10:16:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2888/steps/15/logs/stdio oe-selftest-fedora fedora40-vk-4 master completed at 2026-01-13 02:34:51+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2897/steps/15/logs/stdio
reproducible opensuse156-vk-1 master completed at 2026-01-12 08:57:39+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-zvqdfm8z https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-zvqdfm8z/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-zvqdfm8z/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3176/steps/14/logs/stdio reproducible fedora39-vk-1 master completed at 2026-01-12 08:44:09+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-vracrl9_ https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-vracrl9_/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260112-vracrl9_/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3175/steps/13/logs/stdio reproducible rocky8-vk-1 master completed at 2026-01-13 01:52:47+00:00 https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260113-keuqvwhc https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260113-keuqvwhc/bitbake-reproducibleA.log https://valkyrie.yocto.io/pub/repro-fail/oe-reproducible-20260113-keuqvwhc/bitbake-reproducibleB-extended.log https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/3184/steps/14/logs/stdio qemux86-64-tc debian11-vk-1 master completed at 2026-01-13 12:03:52+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemux86-64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/3011/steps/13/logs/stdio qemux86-tc debian12-vk-4 master completed at 2026-01-13 12:03:52+00:00 https://valkyrie.yocto.io/pub/non-release/20260113-8/testresults/qemux86-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2950/steps/13/logs/stdio qemuriscv64 debian12-vk-9 master completed at 2026-01-14 02:14:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/929/steps/14/logs/stdio qemuriscv64-ptest fedora41-vk-1 master completed at 2026-01-14 02:57:06+00:00 https://valkyrie.yocto.io/pub/non-release/20260114-14/testresults/qemuriscv64-ptest/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/952/steps/12/logs/stdio qemuriscv64-tc debian13-vk-1 master completed at 2026-01-14 02:18:19+00:00 https://valkyrie.yocto.io/pub/non-release/20260114-14/testresults/qemuriscv64-tc/ https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/918/steps/13/logs/stdio
I spent some time analyzing the segfaults and the backtraces indicate failures across multiple crate builds (such as regex_syntax, memchr, jiff, etc.) similar to AB logs, which suggests possible corruption in the sstate cache. The sstate cache affects the compilation stage of rust build each time. After the removal of contaminated sstate cache (rust-native, llvm-native from ubuntu 25.10), the builds are now passing successfully. However, when comparing last week’s sstate cache with the current/updated one, there is a significant difference. The sstate contents were reduced from approximately 5893 files and 257 directories (~14 GB) to 400 files and 164 directories (~1.6 GB). This change may be also related to sstate cleanup performed to address other AB INT issues but it is unclear whether this behavior is expected. How do we proceed further?
Hi Yash, please also see recent comments on https://bugzilla.yoctoproject.org/show_bug.cgi?id=16087. It's possible that both issues have the same underlying cause with sstate artifacts built on newer workers (Fedora 43 & Ubuntu 25.10) not being compatible with older workers (CentOS 8/9 derivatives). Have you seen failures on distros other than AlmaLinux 8/9, Rocky Linux 8/9, CentOS Stream 9?
@Yash, the issue is directly reproducible without Yocto sstate: - build llvm-native on ubuntu 25.10 - transfer the llvm-native populate_sysroot tar.zst to an older distro - build rust-native and then cargo-native using that sstate piece - cargo-native build will have rustc crashes If you can't easily do step 1, you can download the needed archive from https://valkyrie.yocto.io/pub/shared-failure-data/ubuntu2510-sstate/ (there's also the rust-native archive built with that llvm-native) We do need to understand why the crash is happening, otherwise we cannot enable ubuntu 25.10, or possibly any other distro more recent than that. Unfortunately I do not have any specific ideas on how to start debugging, but getting a more useful backtrace than the one in comment 37 would be a good start.
@Alex, thanks for sharing the steps to reproduce without Yocto's sstate. I followed the steps and set up a docker environment based on CentOS Stream 9 and pulled the llvm-native and rust-native sstate archives built on Ubuntu 25.10 from the shared location. However, I was not able to reproduce the segfault after transferring the Ubuntu 25.10 sstate or when building everything locally from scratch without sstate. I am not sure if using a Docker env could be affecting it? Am I missing something here?
Hi Yash, one possibility is that binaries built on the newer distros can't be loaded by older kernels which are missing https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0da1d5002745cdc721bc018b582a8a9704d56c42. If that is the root cause (still just a guess right now), then running under Docker may not reproduce the issue if the host kernel is v5.15.y or later. Comparing behaviour of CentOS Stream 9 in docker (with host kernel v5.15.y or later) vs native CentOs Stream 9 (with kernel v5.14.y) would help to narrow down whether this is a userspace or kernel issue.
@Yash The issue is not going to happen if you build from scratch. You specifically need to ensure that llvm-native from ubuntu 25.10 is used. Also, just to be sure: it's not enough to just copy the broken sstate archive into the sstate directory: it needs to have the file name such that it is actually used in a build (it's not possible to have the exact same sstate signature in the build compared to when this sstate was produced, so you almost certainly need to rename the file to match what bitbake is looking for in sstate). You can probably start by building llvm-native from scratch, then copy/rename the broken llvm archive such that it overwrites the one in sstate. Another option is that you first set up a ubuntu 25.10 build, produce the broken item there, then set up the same build on a different distro, and transfer it. Then you do not need to rename anything. @Paul, I do not think it's a kernel elf loader issue. Unlike the fedora43 issues, this one happens across all distros, including some very recent ones. Can't say for sure, the more important thing is that someone else than me can reproduce it locally.
(In reply to Alexander Kanavin from comment #45) > @Yash The issue is not going to happen if you build from scratch. You > specifically need to ensure that llvm-native from ubuntu 25.10 is used. I would like to correct myself in case my earlier comment was misleading or misunderstood. I did used the llvm-native sstate generated on Ubuntu 25.10 (from the shared archive link) in CentOS Stream 9 docker env and tested it against two different build setups: 1. using the Yocto sstate cache and 2. building without using Yocto sstate However, in both the cases the issue didn't reproduce. > Also, just to be sure: it's not enough to just copy the broken sstate > archive into the sstate directory: it needs to have the file name such that > it is actually used in a build (it's not possible to have the exact same > sstate signature in the build compared to when this sstate was produced, so > you almost certainly need to rename the file to match what bitbake is > looking for in sstate). You can probably start by building llvm-native from > scratch, then copy/rename the broken llvm archive such that it overwrites > the one in sstate. > > Another option is that you first set up a ubuntu 25.10 build, produce the > broken item there, then set up the same build on a different distro, and > transfer it. Then you do not need to rename anything. As you suggested, I also ensured that the llvm-native sstate archive was renamed to match the signature expected by bitBake (by first building llvm-native locally and then overwriting the corresponding sstate artifact with the Ubuntu 25.10–generated one). Even with this in place, the builds still completed successfully and I was not able to trigger the segfault. I checked with the same builds on both Ubuntu 25.10 and CentOS Stream 9 every time.
Thanks Yash. I'll try to make much more specific steps to reproduce this, as it's possible some additional minor detail has to happen to trigger this.
I tried to reproduce the issue on a machine having linux kernel version before 5.15 as suggested by Paul, 5.4.0-190-generic to be precise, with CentOs Stream 9 docker image, added the llvm-native & rust-native sstate from Ubuntu 25.10 and renamed to match signature and still no luck. The builds passed and unable to reproduce the segfault.
@Yash: the issue still reliably reproduces for me, which I just confirmed: 1. Build llvm-native on ubuntu 2510 (set up poky master with bitbake-setup for example). 2. Pack up the resulting sstate-cache/ directory into a tarball and transfer it to e.g. debian 12. 3. Set up the same poky-master build with bitbake-setup on debian 12 4. Unpack sstate-cache and run 'bitbake -c install cargo-native' 5. The build should fully reuse the unpacked sstate from ubuntu, and proceed directly to building rust-native (building it will take some minutes), then it will crash in cargo-native:do_compile. I have used Yocto's autobuilder worker machines to do this.
Here are the detailed steps that I followed from comment 49 and did multiple attempts, still the builds passed in debian-12-vk-1 worker: 1. Did poky-master bitbake setup and built "bitbake llvm-native" on ubuntu25.10 docker (we don't the access on ubuntu2510-vk-1.yocto.io yet, the worker is paused as of now but others can still ssh, access and do builds) and tar the sstate-cache dir. 2. Did the same above setup in Debian 12 AB worker and unpacked sstate-cache in build dir. Also, explicitly set SSTATE_DIR in local.conf to ensure the unpacked sstate is used. 3. Tested "bitbake -c install cargo-native" and "bitbake rust -c do_test_compile". It's a bit frustrating that issue doesn't reproduce. :( Next step would be to get native ubuntu 25.10 access and try with the sstate from it.
Update: Finally, the segfault is reproduced! Until now, we checked with Ubuntu 25.10 docker's sstate-cache with which builds passed. Michael helped us with Ubuntu 25.10 AB worker access today which we didn't have until now. Thus, it's only reproducible with native Ubuntu 25.10 sstate-cache. Next, we are trying to isolate and debug the segfault and check if toolchain updates/commits cause the issue.
Congratulations! One approach you could also try is carefully compare the build directories of llvm-native on 25.10 docker, and on 25.10 AB baremetal. Particularly what gets installed in ${WORKDIR}/image/ and possible differences there, and the configure/compile logs that tell what is enabled or not. I think some important clues might turn up that way.
Bug 16087 fix we tried and that fixed this segfault issue as well. The -no-rosegment flag is added in bitbake.conf and built llvm-native in Ubuntu 25.10. By sharing this sstate the cargo-native is built in debian12, Ubuntu22.10 & alma9 and all are success.
@Sundeep unfortunately this is not true, and the fix does not help. The crash happens on Debian 12 regardless of whether llvm-native from ubuntu 25.10 was produced with the rosegment tweak or not. Please pay attention when you share sstate: the build configurations need to match exactly (which is not possible because the tweak requires recent binutils to work), or you need to carefully rename sstate items to ensure they are indeed reused, and no rebuilds of e.g. llvm-native happen.
I added DEBUG_BUILD="1" to both builds to try and get useful backtraces, and indeed, it's then possible to see exactly where the crash happens by simply running rustc with a trivial .rs file: akanavin@debian12-vk-5:~/bitbake-builds/poky-master/build$ gdb --args tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/rustc ../layers/openembedded-core/meta/lib/oeqa/files/test.rs ... 0x00007ffff3de1ca0 in llvm::X86ReadAdvanceTable () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so (gdb) bt #0 0x00007ffff3de1ca0 in llvm::X86ReadAdvanceTable () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #1 0x00007fffefeb42f3 in llvm::X86GenSubtargetInfo::X86GenSubtargetInfo(llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::StringRef) () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #2 0x00007fffefeb4c44 in llvm::X86Subtarget::X86Subtarget(llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::X86TargetMachine const&, llvm::MaybeAlign, unsigned int, unsigned int) () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #3 0x00007fffee487bf4 in llvm::X86TargetMachine::getSubtargetImpl(llvm::Function const&) const () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #4 0x00007fffef169960 in (anonymous namespace)::ExpandLargeDivRemLegacyPass::runOnFunction(llvm::Function&) () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #5 0x00007fffefb64e21 in llvm::FPPassManager::runOnFunction(llvm::Function&) [clone .localalias] () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #6 0x00007fffefb65079 in llvm::FPPassManager::runOnModule(llvm::Module&) [clone .localalias] () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #7 0x00007fffefb655ed in llvm::legacy::PassManagerImpl::run(llvm::Module&) [clone .localalias] () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #8 0x00007fffee4287f3 in LLVMRustWriteOutputFile () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #9 0x00007fffee29dd2f in rustc_codegen_llvm[91f854d6bb7e0623]::back::write::write_output_file () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #10 0x00007fffee2a0d7e in rustc_codegen_llvm[91f854d6bb7e0623]::back::write::codegen () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #11 0x00007fffee290458 in rustc_codegen_ssa[17bf207abf45fc64]::back::write::execute_optimize_work_item::<rustc_codegen_llvm[91f854d6bb7e0623]::LlvmCodegenBackend> () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #12 0x00007fffee375d1b in std[c3234882613b4850]::sys::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_llvm[91f854d6bb7e0623]::LlvmCodegenBackend as rustc_codegen_ssa[17bf207abf45fc64]::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa[17bf207abf45fc64]::back::write::start_executing_work<rustc_codegen_llvm[91f854d6bb7e0623]::LlvmCodegenBackend>::{closure#4}, core[f0ef83cf092eb77a]::result::Result<rustc_codegen_ssa[17bf207abf45fc64]::back::write::CompiledModules, ()>>::{closure#0}, core[f0ef83cf092eb77a]::result::Result<rustc_codegen_ssa[17bf207abf45fc64]::back::write::CompiledModules, ()>> () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #13 0x00007fffee4003e2 in <<std[c3234882613b4850]::thread::Builder>::spawn_unchecked_<<rustc_codegen_llvm[91f854d6bb7e0623]::LlvmCodegenBackend as rustc_codegen_ssa[17bf207abf45fc64]::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa[17bf207abf45fc64]::back::write::start_executing_work<rustc_codegen_llvm[91f854d6bb7e0623]::LlvmCodegenBackend>::{closure#4}, core[f0ef83cf092eb77a]::result::Result<rustc_codegen_ssa[17bf207abf45fc64]::back::write::CompiledModules, ()>>::{closure#0}, core[f0ef83cf092eb77a]::result::Result<rustc_codegen_ssa[17bf207abf45fc64]::back::write::CompiledModules, ()>>::{closure#1} as core[f0ef83cf092eb77a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-1402f12c7ca0944c.so #14 0x00007fffed1536d7 in ?? () from /home/akanavin/bitbake-builds/poky-master/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/libstd-c65b7339946dac4c.so #15 0x00007fffecf4755d in start_thread () from /home/akanavin/bitbake-builds/poky-master/build/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6 #16 0x00007fffecfc378c in __clone3 () from /home/akanavin/bitbake-builds/poky-master/build/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6 So bits of llvm from ubuntu 25.10 are being *statically* linked into rust on debian 12, and I suspect this is where the root of the problem is. Ross has already pointed this out in comment 36.
I have reproduced the issue without involving ubuntu 25.10 at all, just using debian 12 and buildtools tarball: 1. Use oe-core master to build buildtools-tarball-extended. 2. Install that and add it to environment, build llvm-native with it (this would be done with gcc 15/binutils 2.45 from buildtools). 3. Rename tmp/ to tmp.buildtools/ and re-initialize the environment to go back to debian's host compiler and linker (gcc 12, binutils 2.40). 4. Build cargo-native. Rustc then crashes in exactly same way. So I think this almost definitively confirms that the issue is gcc 15 generating code in a way that is not properly understood by older linkers. Which means we can't share .a or .o files between hosts, only shared libraries or executables.
> Ross has already pointed this out in comment 36. From my memory the underlying llvm problem was a link-time problem involving static linking, because uninative works by letting code build with the host /usr/include but link against uninative libc.so. We pass --allow-shlib-undefined to make this work, but that doesn't work if components have been statically linked. It was mostly a build failure so is an interesting thread to pull, but I'm not sure it's the smoking gun.
(In reply to Alexander Kanavin from comment #55) > I added DEBUG_BUILD="1" to both builds to try and get useful backtraces, and -- I rebuilt llvm-native and rust/cargo with full debug enabled (RelWithDebInfo, -g3, rust.debug = true, opt-level=0) to obtain a more detailed backtrace. With this setup, the same reproducer still crashes in llvm::X86ReadAdvanceTable(), and GDB is able to unwind further through LLVM, showing a deeper backtrace (31 frames total). Attaching the backtrace below for reference. **************** 0x00007ffff2cdcb00 in llvm::X86ReadAdvanceTable () from /srv/pokybuild/sundeep/5-debug_verify-llv/bitbake-builds/debian-debug-build-gdb/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-2c5af5bbf5b8ffb5.so (gdb) bt #0 0x00007ffff2cdcb00 in llvm::X86ReadAdvanceTable () from /srv/pokybuild/sundeep/5-debug_verify-llv/bitbake-builds/debian-debug-build-gdb/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-2c5af5bbf5b8ffb5.so #1 0x00007fffe97e37c8 in llvm::X86GenSubtargetInfo::X86GenSubtargetInfo (this=this@entry=0x7fffb802aba0, TT=..., CPU=..., TuneCPU=..., FS=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work/x86_64-linux/llvm-native/21.1.8/build/lib/Target/X86/X86GenSubtargetInfo.inc:66997 #2 0x00007fffe97e4c85 in llvm::X86Subtarget::X86Subtarget (this=this@entry=0x7fffb802aba0, TT=..., CPU=..., TuneCPU=<error reading variable: DWARF-2 expression error: DW_OP_GNU_uninit must always be the very last op.>, FS=..., TM=..., StackAlignOverride=..., PreferVectorWidthOverride=0, RequiredVectorWidth=4294967295) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/Target/X86/X86Subtarget.cpp:341 #3 0x00007fffe6b357fb in std::make_unique<llvm::X86Subtarget, llvm::Triple const&, llvm::StringRef&, llvm::StringRef&, llvm::StringRef&, llvm::X86TargetMachine const&, llvm::MaybeAlign, unsigned int&, unsigned int&> () at /usr/include/c++/15/bits/unique_ptr.h:1084 #4 llvm::X86TargetMachine::getSubtargetImpl (this=0x7fffd09445a0, F=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/Target/X86/X86TargetMachine.cpp:344 #5 0x00007fffe80886ff in (anonymous namespace)::ExpandLargeDivRemLegacyPass::runOnFunction (this=<optimized out>, F=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/CodeGen/ExpandLargeDivRem.cpp:155 #6 0x00007fffe924f5a0 in llvm::FPPassManager::runOnFunction (this=0x7fffb80107b0, F=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/IR/LegacyPassManager.cpp:1398 #7 0x00007fffe924f843 in llvm::FPPassManager::runOnModule (this=0x7fffb80107b0, M=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/IR/LegacyPassManager.cpp:1444 #8 0x00007fffe924ff14 in (anonymous namespace)::MPPassManager::runOnModule (this=0x7fffb8001370, M=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/IR/LegacyPassManager.cpp:1513 #9 llvm::legacy::PassManagerImpl::run (this=0x7fffb8000f20, M=...) at /srv/pokybuild/sundeep/5-debug-info-llvm/bitbake-builds/debug-buildd/build/tmp/work-shared/llvm-project-source-21.1.8-r1/sources/llvm-project-21.1.8.src/llvm/lib/IR/LegacyPassManager.cpp:531 #10 0x00007fffe6ae3625 in LLVMRustWriteOutputFile (Target=0x7fffd09445a0, PMR=0x7fffb8000f00, M=0x7fffd09406f0, Path=<optimized out>, DwoPath=0x0, RustFileType=<optimized out>, VerifyIR=false) at llvm-wrapper/PassWrapper.cpp:430 #11 0x00007fffe67ccec1 in rustc_codegen_llvm::back::write::write_output_file (dcx=..., target=0x7fffd09445a0, no_builtins=false, m=0x7fffd09406f0, output=..., dwo_output=..., file_type=rustc_codegen_llvm::llvm::ffi::FileType::ObjectFile, self_profiler_ref=0x7fffc73f78d0, verify_llvm_ir=false) at compiler/rustc_codegen_llvm/src/back/write.rs:80 #12 0x00007fffe67d382c in rustc_codegen_llvm::back::write::codegen (cgcx=0x7fffc73f7830, module=..., config=0x7fffd099e220) at compiler/rustc_codegen_llvm/src/back/write.rs:960 #13 0x00007fffe67e655a in rustc_codegen_llvm::{impl#3}::codegen (cgcx=0x7fffc73f7830, module=<error reading variable: Cannot access memory at address 0x99>, config=0x7fffd099e220) at compiler/rustc_codegen_llvm/src/lib.rs:219 #14 0x00007fffe6938504 in rustc_codegen_ssa::back::write::execute_optimize_work_item<rustc_codegen_llvm::LlvmCodegenBackend> (cgcx=0x7fffc73f7830, module=...) at compiler/rustc_codegen_ssa/src/back/write.rs:866 #15 0x00007fffe694a1b7 in rustc_codegen_ssa::back::write::start_executing_work::{closure#4}::{closure#1}<rustc_codegen_llvm::LlvmCodegenBackend> ( --Type <RET> for more, q to quit, c to continue without paging-- allocator_module=<error reading variable: Cannot access memory at address 0x99>) at compiler/rustc_codegen_ssa/src/back/write.rs:1504 #16 0x00007fffe69f084b in core::option::Option<rustc_codegen_ssa::ModuleCodegen<rustc_codegen_llvm::ModuleLlvm>>::and_then<rustc_codegen_ssa::ModuleCodegen<rustc_codegen_llvm::ModuleLlvm>, rustc_codegen_ssa::CompiledModule, rustc_codegen_ssa::back::write::start_executing_work::{closure#4}::{closure_env#1}<rustc_codegen_llvm::LlvmCodegenBackend>> (self=..., f=...) at library/core/src/option.rs:1545 #17 0x00007fffe6943406 in rustc_codegen_ssa::back::write::start_executing_work::{closure#4}<rustc_codegen_llvm::LlvmCodegenBackend> () at compiler/rustc_codegen_ssa/src/back/write.rs:1503 #18 0x00007fffe67a7e47 in rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> () at compiler/rustc_codegen_llvm/src/lib.rs:150 #19 0x00007fffe6a34f22 in std::sys::backtrace::__rust_begin_short_backtrace<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> (f=...) at library/std/src/sys/backtrace.rs:158 #20 0x00007fffe6a3a366 in std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure#0}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> () at library/std/src/thread/mod.rs:562 #21 0x00007fffe69eaa8e in core::panic::unwind_safe::{impl#23}::call_once<core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>, std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>> (self=...) at library/core/src/panic/unwind_safe.rs:274 #22 0x00007fffe68359f8 in std::panicking::catch_unwind::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> (data=0x7fffc73f8d50) at library/std/src/panicking.rs:590 #23 0x00007fffe6a826eb in __rust_try () from /srv/pokybuild/sundeep/5-debug_verify-llv/bitbake-builds/debian-debug-build-gdb/build/tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/../lib/librustc_driver-2c5af5bbf5b8ffb5.so #24 0x00007fffe6a393d5 in std::panicking::catch_unwind<core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>, core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>>> (f=...) at library/std/src/panicking.rs:553 #25 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> (f=...) at library/std/src/panic.rs:359 #26 std::thread::{impl#0}::spawn_unchecked_::{closure#1}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>> () at library/std/src/thread/mod.rs:560 #27 0x00007fffe678415f in core::ops::function::FnOnce::call_once<std::thread::{impl#0}::spawn_unchecked_::{closure_env#1}<rustc_codegen_llvm::{impl#2}::spawn_named_thread::{closure_env#0}<rustc_codegen_ssa::back::write::start_executing_work::{closure_env#4}<rustc_codegen_llvm::LlvmCodegenBackend>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, core::result::Result<rustc_codegen_ssa::back::write::CompiledModules, ()>>, ()> () at library/core/src/ops/function.rs:250 #28 0x00007fffde1ba126 in alloc::boxed::{impl#29}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> (self=..., args=()) --Type <RET> for more, q to quit, c to continue without paging-- at library/alloc/src/boxed.rs:2005 #29 0x00007fffde17d498 in std::sys::thread::unix::{impl#2}::new::thread_start (data=0x7fffd09a0710) at library/std/src/sys/thread/unix.rs:126 #30 0x00007fffdde9e55d in start_thread () from /srv/pokybuild/sundeep/5-debug_verify-llv/bitbake-builds/debian-debug-build-gdb/build/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6 #31 0x00007fffddf1a78c in __clone3 () from /srv/pokybuild/sundeep/5-debug_verify-llv/bitbake-builds/debian-debug-build-gdb/build/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6 (gdb) q
(In reply to Alexander Kanavin from comment #56) > I have reproduced the issue without involving ubuntu 25.10 at all, just > using debian 12 and buildtools tarball: > -- Thanks Alex. I tried to reproduce the same scenario on my local setups, but I was not able to observe the crash. I tested on two different distributions: Debian 12 and AlmaLinux 9. My test environments: ******************** AlmaLinux 9 (GCC: 11.5.0, Binutils: 2.35.2, glibc: 2.34) Path: /srv/pokybuild/sundeep/alexsteps/bitbake-builds/verification-gcc/build Debian 12 (GCC: 12.2.0, Binutils: 2.40, glibc: 2.36) Path: /srv/pokybuild/sundeep/reverify/bitbake-builds/revrfcation/build Steps I followed: - Built buildtools-extended-tarball using oe-core master. - Installed and sourced the buildtools environment. - Verified that GCC 15 from buildtools was active. - Built llvm-native. - Renamed tmp/ to tmp.buildtools/. - Reset the shell environment and re-initialized it without sourcing buildtools (so the host GCC/binutils were active again). - Verified that GCC reverted to the distro version. - Ran bitbake cargo-native. In my case, cargo-native completed successfully. Only setscene tasks were executed and sstate-cache was reused. I did not see a crash on either system. Is there any additional step, cleanup or environment difference that I should apply to reproduce the issue more closely?
@Sunil I think there's a key step missing between these two steps, sorry I didn't describe what I did more precisely: - Verified that GCC 15 from buildtools was active. <----- at this point you also need to rename tmp/, so that links in tmp/hosttools/ are recreated to point to the buildtools, otherwise they will remain as they are, e.g. pointing to host gcc. - Built llvm-native.
For extra confidence, before building llvm-native you can also rename sstate-cache/ , so that anything built in the first step (e.g. for buildtools) does not get reused, and everything is rebuilt with gcc15 from buildtools.
(In reply to Alexander Kanavin from comment #61) > For extra confidence, before building llvm-native you can also rename > sstate-cache/ , so that anything built in the first step (e.g. for > buildtools) does not get reused, and everything is rebuilt with gcc15 from > buildtools. -- Thanks for the clarification. Sure — I will try the following steps next: 1. Build buildtools-extended-tarball 2. Install and source buildtools (with GCC 15 active) 3. Rename tmp/ 4. Rename sstate-cache/ (for safe side) 5. Build llvm-native (everything rebuilt with GCC 15) 6. Rename tmp/ again (optional, to keep results separate) 7. Reset the shell environment (no buildtools sourced) 8. Build cargo-native Along with this, I wanted to share a update from my earlier testing for context. I tested llvm-native sstate produced on Ubuntu 25.10 in two different environments and then reused that sstate on other older host systems. Producer environments 1. Ubuntu 25.10 (bare metal, laptop) Kernel: 6.17, GCC: 15, Binutils: 2.45 2. Ubuntu 25.10 (Docker container) Kernel (host): 6.14, GCC: 15, Binutils: 2.45 Results using sstate from Ubuntu 25.10 bare metal ************************************************* The llvm-native sstate built on Ubuntu 25.10 bare metal reproduces the crash when reused on older Distros. Results using sstate from Ubuntu 25.10 Docker ********************************************* The llvm-native sstate built inside an Ubuntu 25.10 Docker container does not reproduce the crash when reused on: In these cases, cargo-native completes successfully. ----- Both producer environments use GCC 15 and binutils 2.45. However, only the llvm-native sstate produced on Ubuntu 25.10 bare metal(laptop) triggers the crash. The sstate produced inside Ubuntu 25.10 Docker does not. As a next isolation step, I am considering testing the bare-metal producer with a slightly older kernel (for example, 6.16 instead of 6.17), while keeping the rest of the userspace unchanged. Please let me know if this sounds like a worthwhile test, or if you would recommend a different approach.
Step 6 isn't optional. The links in tmp/hosttols/ is what Yocto is actually using to build native recipes. The binaries in PATH that you see on command line are only used to create those links, and the only way to do it is remove/rename tmp/. As we are going from host toolchain to buildtools toolchain, and then back to host toolchain, tmp/ needs to be removed before both transitions. TBH, I think docker/kernel route is a dead end. The issue seems to trigger with a particular combination: building llvm-native with a newer gcc/binutils and then building rust using that llvm-native, linked statically into librustc_driver, using older gcc/binutils. I'm not sure why you can't see it with docker ubuntu, but as we now have a way to do everything on debian 12, I think that's the best way to continue.
Thanks Alex. I can now reliably reproduce the crash on Debian 12 by following the updated steps, especially making sure that tmp/ is renamed at each transition so that tmp/hosttools/ links are recreated correctly. With this setup: - llvm-native is built using GCC 15 from buildtools - cargo-native / rust is later built using the host toolchain I consistently see the crash as reported. ------ As you mentioned, enabling the link-shared configuration in the rust recipe avoids the crash, which further confirms that the issue is related to mixing native artifacts built with a newer toolchain and rust being built with an older host toolchain. At this point, the reproduction is reliable on Debian 12. I’ll continue testing as needed, and please let me know if there is any additional information or validation that would be helpful.
I'd suggest you build with DEBUG_BUILD = "1" and reproduce the crash in gdb as suggested in one of my earlier comments: akanavin@debian12-vk-5:~/bitbake-builds/poky-master/build$ gdb --args tmp/work/x86_64-linux/cargo-native/1.92.0/recipe-sysroot-native/usr/bin/rustc ../layers/openembedded-core/meta/lib/oeqa/files/test.rs I think it would be a useful exercise to see exactly where and how it happens. Specifically, use 'disas /r' command to print a disassembly of the crashing function with associated hex codes (gdb has tab-autocompletion to fill in the function name), and what you'll see is that it looks like data that is attempted to be executed as code. Then use disassembly command on the parent function above (one that calls the bogus 'function') and save the output. Then do the same with working rustc (e.g. one that is built entirely with gcc 15, both llvm-native and rust-native). You'll see a critical difference in disassembly of that parent function, and that difference is where the older linker messed things up.
I kept a breakpoint on llvm::X86GenSubtargetInfo::X86GenSubtargetInfo and stepped into TargetSubtargetInfo with everything built using GCC 15. In that case, all arguments (TT, CPU, TuneCPU, WPR, WL, RA, etc.) appear valid. In particular, RA (the Read Advance Table) points to llvm::X86ReadAdvanceTable, and the memory addresses look correct. Execution proceeds normally through the constructor. However, when building LLVM with GCC 15 but running Cargo with GCC 12, stepping into the same constructor immediately results in a segmentation fault. It seems the pointer to RA is either invalid or points to unmapped memory. For reference, the instruction pointers are: ------------ PASS case (working): rip = 0x7fffe7588916 <llvm::TargetSubtargetInfo::TargetSubtargetInfo(...)> FAIL case (segfault): rip = 0x7ffff19041e0 <llvm::X86ReadAdvanceTable> I’m not entirely sure how RA ends up pointing to invalid memory in the fail case — it seems the table is either incorrectly linked or not mapped as expected at runtime.
Patch sent: https://lists.openembedded.org/g/openembedded-core/message/231048
V2 Sent: https://lists.openembedded.org/g/openembedded-core/message/231254
The fix has landed in oe-core master, so I have unpaused the 25.10 worker: https://autobuilder.yoctoproject.org/valkyrie/#/workers/63 The bug can be closed.
Two similar issues, both seen on fedora40, with the patch mentioned above being merged. Not reopening right away, but still tracking the issue. qemuarm-oecore fedora40-vk-2 mathieu/master-next completed at 2026-02-21 21:21:03+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/40/builds/3199/steps/12/logs/stdio qemuarm-oecore fedora40-vk-3 mathieu/master-next completed at 2026-02-22 16:20:22+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/40/builds/3204/steps/12/logs/stdio
I am looking at the logs.
I am able to reproduce the issue locally. The problem appears to be with nativesdk-rust. I am currently validating the fix.
Reopeining, as fix for nativesdk/target still hasn't landed.
The hopefully final fix is now in master, so i think this bug can be closed: https://git.openembedded.org/openembedded-core/commit/?id=d0671c3dad87a063b3a41dd07cde89b5684e692c
Hi Alex, Yes, Changes merged in Master. https://git.openembedded.org/openembedded-core/commit/?id=d0671c3dad87a063b3a41dd07cde89b5684e692c As you know, This change was merged as a temporary workaround for a Rust toolchain issue (as discussed earlier with Randy). The plan is to revisit it once the upstream situation is clearer. (https://lists.openembedded.org/g/openembedded-core/message/233260) I’ll try to create a simpler reproducer for reporting this upstream to GCC/Binutils. Because of the mixed toolchain setup, it may not be easy to reduce, but I’ll share an update if I can file a bug. We should also keep a tracking bug to make sure we come back to this and remove the workaround later in future. This doesn’t need to keep ypbz#16058 open, so i think, we can close it. I’ll follow up if there are updates on the upstream side.
Tracking Bug created: https://bugzilla.yoctoproject.org/show_bug.cgi?id=16207
WE have a fix / work-around and the root cause tracking bug so closing this one. Sunil, add the commit link if it's missing (no time to check now).
Just had a segfault in scarthgap/librustc_driver: scarthgap qemuarm64-tc debian12-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/3395 It looks a lot like this bug but: * We do not have a Ubuntu 25.10 used with scarthgap * Rust version are quite different 1.75/1.94 Maybe I need to look into backporting: * rust: Enable dynamic linking with llvm * https://git.openembedded.org/openembedded-core/commit/?id=74ba238ff1ba1e9b612aece1989b828f3a8f8770 * This one does apply directly * rust: enable dynamic LLVM linking by default * https://git.openembedded.org/openembedded-core/commit/?id=d0671c3dad87a063b3a41dd07cde89b5684e692c * This one has conflicts ... and/or start by using the reproducer to confirm that scarthgap does indeed have this bug.
(In reply to Yoann Congal from comment #78) > Just had a segfault in scarthgap/librustc_driver: > scarthgap qemuarm64-tc debian12-vk-2 > https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/3395 > > It looks a lot like this bug but: > * We do not have a Ubuntu 25.10 used with scarthgap It isn't clear where the list of workers that are used for scarthgap builds is defined, but I'd say you need to backport the master fixes, yes.
(In reply to Alexander Kanavin from comment #79) > It isn't clear where the list of workers that are used for scarthgap builds > is defined, but I'd say you need to backport the master fixes, yes. It's defined here: https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n202: "alma8", "alma9", "debian11", "debian12", "fedora38", "fedora39", "fedora40", "fedora41", "opensuse154", "rocky8", "rocky9", "stream8", "stream9", "ubuntu1804", "ubuntu2004", "ubuntu2204", "ubuntu2304", "ubuntu2404", "perf-" # == "perf-debian12-vk", "perf-alma8-vk" Note that not everything in this list has active workers. See https://autobuilder.yoctoproject.org/valkyrie/#/workers for active workers.
(In reply to Yoann Congal from comment #80) > It's defined here: > https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n202: > "alma8", > "alma9", > "debian11", > "debian12", > "fedora38", > "fedora39", > "fedora40", > "fedora41", > "opensuse154", > "rocky8", > "rocky9", > "stream8", > "stream9", > "ubuntu1804", > "ubuntu2004", > "ubuntu2204", > "ubuntu2304", > "ubuntu2404", > "perf-" # == "perf-debian12-vk", "perf-alma8-vk" One possibility is that it's not caused by a 'new' distro, but by an old distro with buildtools, which have a new toolchain. Do you know which gcc/binutils versions are in it?
(In reply to Alexander Kanavin from comment #81) > One possibility is that it's not caused by a 'new' distro, but by an old > distro with buildtools, which have a new toolchain. Do you know which > gcc/binutils versions are in it? https://git.yoctoproject.org/yocto-autobuilder-helper/tree/config.json?h=scarthgap#n1930 "buildtools" : { "ubuntu1804-arm-*" : "${BUILDTOOLS_ARM_URL}", "alma8*" : "${BUILDTOOLS_URL}", "rocky8*" : "${BUILDTOOLS_URL}", "stream8*" : "${BUILDTOOLS_URL}", "opensuse154*" : "${BUILDTOOLS_URL}", "opensuse155*" : "${BUILDTOOLS_URL}", "perf-alma8*" : "${BUILDTOOLS_URL}", "perf-ubuntu1804-ty-*" : "${BUILDTOOLS_URL}" } Those worker will use > "BUILDTOOLS_URL" : "https://downloads.yoctoproject.org/releases/yocto/yocto-5.0.4/buildtools/x86_64-buildtools-extended-nativesdk-standalone-5.0.4.sh", And get: $ gcc --version gcc (GCC) 13.4.0 $ ld --version GNU ld (GNU Binutils) 2.42.0.20240723
Hi Alex, I attempted to reproduce the scarthgap segfault using both methods that were effective for master: - Copying llvm-native sstate from an Ubuntu 25.10 host and reusing it on an older distro - Building llvm-native with the buildtools-extended-tarball (GCC 15 toolchain),then rebuilding cargo-native/rust with the host toolchain (mixed toolchain scenario) In both cases, I was unable to reproduce the segfault on the scarthgap branch. The builds completed successfully without any crash. Hi Yoann, Could you help with a bit more context: When was the segfault last seen on the AB? Is it still occurring in recent builds, or was it a one-time occurrence (comment #78)? Do you have the exact builder/build URL of any recent recurrence?
(In reply to Sermsity Sunil Kumar Dora from comment #83) > Hi Yoann, > Could you help with a bit more context: > When was the segfault last seen on the AB? > Is it still occurring in recent builds, or was it a one-time occurrence (comment #78)? > Do you have the exact builder/build URL of any recent recurrence? Comment #78 is the only occurence (yet), I did not have another one: * URL: ...#/builders/5/builds/3395 (not putting the full URL to not confuse the AB-INT parsing tool) * date: 2026-03-24 14:55:22+01:00
qemuarm-oecore rocky8-vk-1 scarthgap completed at 2026-06-24 11:38:20+00:00 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/40/builds/3981 Note: We added Ubuntu 26.04 into the worker pool recently.
Root cause found, reproduced and verified on master and scarthgap. The crash is caused by a binutils linker bug, PR ld/32991: https://sourceware.org/bugzilla/show_bug.cgi?id=32991 Starting with binutils 2.45 (commit 11c2852449825), gas emits the relaxable relocation R_X86_64_GOTPCRELX for "push foo@GOTPCREL(%rip)", a pattern gcc 15/16 generates for stack arguments. Older ld versions without the PR32991 fix misinterpret this relocation and silently rewrite the push into a call. In librustc_driver.so the rewritten call jumps into read-only data (llvm::X86ReadAdvanceTable), so rustc segfaults at runtime. The link itself completes successfully - the problem only appears when the newly built rustc is first executed (cargo-native do_compile). Binutils upstream backported the fix to all 2.38-2.44 branches in May 2025. I confirmed the analysis with binutils upstream. https://sourceware.org/pipermail/binutils/2026-July/150097.html When does it happen =================== * LLVM objects assembled by gas >= 2.45, and * linked into librustc_driver.so by an ld without the PR32991 fix. Matches the master AB failures and scarthgap. Reproducer: check any ld in few seconds ======================================= $ cat repro.s .text .globl func func: push tbl@GOTPCREL(%rip) add $8, %rsp ret .section .rodata .globl tbl .hidden tbl tbl: .zero 576 $ as -o repro.o repro.s # as from binutils >= 2.45 $ readelf -rW repro.o | grep tbl # -> R_X86_64_GOTPCRELX $ ld -shared -o out.so repro.o # ld under test $ objdump -d out.so | grep -A1 '<func>:' unfixed ld: 67 e8 ... addr32 call <tbl> (was a push) fixed ld: ff 35 ... push (through GOT) gas < 2.45 emits plain GOTPCREL for this source, so old-only environments were never affected. Full-scale verification ======================= oe-core master (Ubuntu 22.04 host, ld 2.38, dynamic-linking workaround reverted): llvm-native built with buildtools gcc16/gas2.46, then cargo-native built with the host ld via shared sstate -> rustc SIGSEGV. Same sstate with an ld built from binutils-2_38-branch tip (= the PR32991 backport) -> clean. scarthgap: rust-llvm-native built in an ubuntu:26.04 container (gcc 15.2/gas 2.46; note scarthgap's rust uses rust-llvm-native, not llvm-native), sstate shared to an Ubuntu 22.04 host -> rust-native links with host ld 2.38 -> rustc SIGSEGV in cargo-native. Same sstate with a PR32991-fixed ld -> clean. Impact ====== * No rust changes are required; the fix is on the linker side. * Every ld that links rust needs the PR32991 fix. * rust-native sstate produced by an affected linker should be invalidated/rebuilt - the incorrect code is already embedded in the cached librustc_driver.so, so a fixed worker restoring old sstate still ships a broken rustc.