Created attachment 5171 [details] logs from kirkstone qemuarm64 fedora40-vk-4 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/3032/steps/14/logs/stdio wget https://ftpmirror.gnu.org/gnu/cpio/cpio-2.13.tar.gz fails on "502 Bad Gateway". Note that this is the load balancer that return 502 not one of the mirrors behind. Locally, or even in a shell on AB I could not reproduce this. Impacted builds: kirkstone meta-arm ubuntu2204-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/2919/steps/19/logs/stdio kirkstone meta-intel ubuntu2204-vk-4 https://autobuilder.yoctoproject.org/valkyrie/#/builders/41/builds/2873/steps/14/logs/stdio kirkstone qemuarm64 fedora40-vk-4 https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/3032/steps/14/logs/stdio kirkstone qemuarm-oecore ubuntu2204-vk-4 https://autobuilder.yoctoproject.org/valkyrie/#/builders/40/builds/3015/steps/19/logs/stdio kirkstone qemux86-64 ubuntu2404-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/68/builds/3109/steps/16/logs/stdio kirkstone qemux86 ubuntu2404-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/3008/steps/16/logs/stdio Log extract: NOTE: Running task 1 of 1 (/srv/pokybuild/yocto-worker/meta-arm/build/meta/recipes-sato/images/core-image-sato.bb:do_testsdk) NOTE: recipe core-image-sato-1.0-r0: task do_testsdk: Started SDK testing environment: armv8a-poky-linux Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/meta-arm/build/meta/lib/oeqa/sdk/cases/buildcpio.py", line 20, in test_cpio tarball = self.fetch(testdir, self.td["DL_DIR"], "https://ftpmirror.gnu.org/gnu/cpio/cpio-2.13.tar.gz") File "/srv/pokybuild/yocto-worker/meta-arm/build/meta/lib/oeqa/sdk/case.py", line 29, in fetch subprocess.check_output(["wget", "-O", tarball, url], stderr=subprocess.STDOUT) File "/usr/lib/python3.10/subprocess.py", line 421, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '['wget', '-O', '/srv/pokybuild/yocto-worker/meta-arm/build/build/tmp/work/generic_arm64-poky-linux/core-image-sato/1.0-r0/testimage-sdk/cpio-x35qc3sm/cpio-2.13.tar.gz', 'https://ftpmirror.gnu.org/gnu/cpio/cpio-2.13.tar.gz']' returned non-zero exit status 8 Standard Output: --2026-01-15 11:44:54-- https://ftpmirror.gnu.org/gnu/cpio/cpio-2.13.tar.gz Resolving ftpmirror.gnu.org (ftpmirror.gnu.org)... 2001:470:142:5::200, 209.51.188.200 Connecting to ftpmirror.gnu.org (ftpmirror.gnu.org)|2001:470:142:5::200|:443... connected. HTTP request sent, awaiting response... 502 Bad Gateway 2026-01-15 11:44:54 ERROR 502: Bad Gateway. RESULTS: RESULTS - assimp.BuildAssimp.test_assimp: PASSED (46.06s) RESULTS - buildepoxy.EpoxyTest.test_epoxy: PASSED (11.39s) RESULTS - buildgalculator.GalculatorTest.test_galculator: PASSED (8.74s) RESULTS - buildlzip.BuildLzipTest.test_lzip: PASSED (1.31s) RESULTS - gcc.GccCompileTest.test_gcc_compile: PASSED (0.06s) RESULTS - gcc.GccCompileTest.test_gpp2_compile: PASSED (0.06s) RESULTS - gcc.GccCompileTest.test_gpp_compile: PASSED (0.17s) RESULTS - gcc.GccCompileTest.test_make: PASSED (0.06s) RESULTS - perl.PerlTest.test_perl: PASSED (0.01s) RESULTS - python.Python3Test.test_python3: PASSED (0.02s) RESULTS - buildcpio.BuildCpioTest.test_cpio: ERROR (0.53s) SUMMARY: core-image-sato sdk (poky-glibc-i686-core-image-sato-armv8a-generic-arm64-toolchain-4.0.32.sh:environment-setup-armv8a-poky-linux) - Ran 11 tests in 68.418s core-image-sato sdk - FAIL - Required tests failed (successes=10, skipped=0, failures=0, errors=1) ERROR: core-image-sato-1.0-r0 do_testsdk: core-image-sato - FAILED - check the task log and the commands log ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/meta-arm/build/build/tmp/work/generic_arm64-poky-linux/core-image-sato/1.0-r0/temp/log.do_testsdk.1370821 NOTE: recipe core-image-sato-1.0-r0: task do_testsdk: Failed ERROR: Task (/srv/pokybuild/yocto-worker/meta-arm/build/meta/recipes-sato/images/core-image-sato.bb:do_testsdk) failed with exit code '1'
This is intermittent: kirkstone qemumips-alt ubuntu2404-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/13/builds/1417 => failed But, rebuild as kirkstone qemumips-alt alma8-vk-2 https://autobuilder.yoctoproject.org/valkyrie/#/builders/13/builds/1418 => success!
That URL is resolved to mirrors, so the actual site is likely random. 502 Bad Gateway is usually something like nginx reverse proxy not being able to load the (often containerized) backend application.
Yoann points out this is the mirror URL itself failing before the redirect.
Note from bug triage team: * There is code to use DL_DIR instead of downloading it every time on master (and maybe in kirkstone) but I need to check if kirkstone is not missing backports. * Also, master uses downloads.yoctoproject.org instead of upstream servers: oeqa/runtime/buildcpio: Use our own mirror for source · openembedded/openembedded-core@038d727 https://github.com/openembedded/openembedded-core/commit/038d7270aef0cccf87d50a117160c58261beb9b9 (RP can help with more context if needed)
code is trying to download https://ftpmirror.gnu.org/gnu/cpio/cpio-2.13.tar.gz but it's missing in DL_DIR: ycongal@ubuntu2404-vk-1:~$ ls /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-* -1 /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.14.tar.gz /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.14.tar.gz.done /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.15.tar.gz /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.15.tar.gz.done /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.8.tar.gz /srv/autobuilder/valkyrie.yocto.io/current_sources/cpio-2.8.tar.gz.done We could either or both: * Maybe switch to cpio-2.14.tar.gz (which is the cpio version of kirkstone, and so, in DL_DIR) to avoid hitting the network * Switch to https://downloads.yoctoproject.org/mirror/sources/cpio-2.13.tar.gz to avoid the unreliable ftpmirror.gnu.org server
Note: master does not use downloads.yoctoproject.org everywhere: meta/lib/oeqa/sdk/cases/autotools.py: tarball = self.fetch(testdir, self.td["DL_DIR"], "https://ftpmirror.gnu.org/gnu/cpio/cpio-2.15.tar.gz") meta/lib/oeqa/selftest/cases/meta_ide.py: "https://ftpmirror.gnu.org/gnu/cpio/cpio-2.15.tar.gz", I conclude that it does not hit this bug because cpio-2.15.tar.gz is always in DL_DIR. So, I'll send a patch for kirkstone to align the test archive with the cpio recipe to ensure it is in DL_DIR.
Backported fix merged on kirkstone: oeqa: Use 2.14 release of cpio instead of 2.13 https://git.openembedded.org/openembedded-core/commit/?h=kirkstone&id=64d56cf416b31ae92438deefe4028402120ed998 => Closing Note: If that ever happen again, we can still switch to yp.org mirrors.