14690 2022-01-14 19:16:01 +0000 setscene errors accessing YP sstate servers from multiple threads 2022-02-24 16:11:46 +0000 1 1 5 Infrastructure Infrastructure General infrastructure-general 4.0 x86 Multiple RESOLVED FIXED High normal 4.0 M3 1 michael.opdenacker mhalstead quaresma.jose randy.macleod richard.purdie --- --- No (bug/feature does not impact docs) oldest_to_newest 92326 0 michael.opdenacker 2022-01-14 19:16:01 +0000 I'm building the latest Poky ("core-image-minimal" plus a few extra packages), on Ubuntu 20.04. I'm always getting the below setscene errors after upgrading my Poky sources: ... <more similar errors before> WARNING: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache WARNING: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: No suitable staging package found ERROR: Logfile of failure stored in: /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libpcre2-native/10.39-r0/temp/log.do_populate_sysroot_setscene.315121 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/libpcre/libpcre2_10.39.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead Currently 4 running tasks (1838 of 1838/467 of 4630) 10% |################ The errors are not critical but paint my terminal with scary red messages. Here are my settings: BB_HASHSERVE = "auto" BB_SIGNATURE_HANDLER = "OEEquivHash" SSTATE_MIRRORS ?= "file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH" BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687" The errors go away after setting: BB_NUMBER_THREADS="1" (my system normally has 4 threads) Thanks to Jose Quaresma for the workaround! There may be an issue on the infrastructure, not support simultaneous requests from the same client... I'll be happy to make further tests if needed. 92356 1 quaresma.jose 2022-01-19 23:30:11 +0000 I have tested the state mirror locally with my own http mirror and it works perfectly. There are no problems with the concurrent connections. I'm more sure that the root cause is some bandwidth/connection limitation on the http YP mirror. 92395 2 michael.opdenacker 2022-01-26 10:00:28 +0000 The setscene issues are now warnings (better!) but there are fetcher errors now, that don't seem to be critical either: ... WARNING: gpgme-native-1.16.0-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/gpgme-native/1.16.0-r0/temp/log.do_populate_sysroot_setscene.863220 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/gpgme/gpgme_1.16.0.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead WARNING: docbook-xml-dtd4-native-4.5-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/docbook-xml-dtd4-native/4.5-r0/temp/log.do_populate_sysroot_setscene.863223 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-devtools/docbook-xml/docbook-xml-dtd4_4.5.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead WARNING: itstool-native-2.0.7-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/itstool-native/2.0.7-r0/temp/log.do_populate_sysroot_setscene.863224 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/itstool/itstool_2.0.7.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead WARNING: libxau-native-1_1.0.9-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead. WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libxau-native/1_1.0.9-r0/temp/log.do_populate_sysroot_setscene.863229 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-graphics/xorg-lib/libxau_1.0.9.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead WARNING: libxml2-native-2.9.12-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst;downloadfilename=universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst, attempting MIRRORS if available ERROR: libxml2-native-2.9.12-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst;downloadfilename=universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache ... 92660 3 randy.macleod 2022-02-24 16:11:46 +0000 Michael Halstead changed the setup of the sstate server.