| Summary: | setscene errors accessing YP sstate servers from multiple threads | ||
|---|---|---|---|
| Product: | [Infrastructure] Infrastructure General | Reporter: | Michael Opdenacker <michael.opdenacker> |
| Component: | infrastructure-general | Assignee: | Michael Halstead <mhalstead> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | High | CC: | quaresma.jose, randy.macleod, richard.purdie |
| Version: | 4.0 | ||
| Target Milestone: | 4.0 M3 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
I have tested the state mirror locally with my own http mirror and it works perfectly. There are no problems with the concurrent connections. I'm more sure that the root cause is some bandwidth/connection limitation on the http YP mirror. The setscene issues are now warnings (better!) but there are fetcher errors now, that don't seem to be critical either:
...
WARNING: gpgme-native-1.16.0-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/gpgme-native/1.16.0-r0/temp/log.do_populate_sysroot_setscene.863220
WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/gpgme/gpgme_1.16.0.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
WARNING: docbook-xml-dtd4-native-4.5-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/docbook-xml-dtd4-native/4.5-r0/temp/log.do_populate_sysroot_setscene.863223
WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-devtools/docbook-xml/docbook-xml-dtd4_4.5.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
WARNING: itstool-native-2.0.7-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/itstool-native/2.0.7-r0/temp/log.do_populate_sysroot_setscene.863224
WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/itstool/itstool_2.0.7.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
WARNING: libxau-native-1_1.0.9-r0 do_populate_sysroot_setscene: No sstate archive obtainable, will run full task instead.
WARNING: Logfile for failed setscene task is /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libxau-native/1_1.0.9-r0/temp/log.do_populate_sysroot_setscene.863229
WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-graphics/xorg-lib/libxau_1.0.9.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
WARNING: libxml2-native-2.9.12-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst;downloadfilename=universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst, attempting MIRRORS if available
ERROR: libxml2-native-2.9.12-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst;downloadfilename=universal/a6/72/sstate:libxml2-native:x86_64-linux:2.9.12:r0:x86_64:7:a67256090f630006c779e3ae43fde80224e33887736a79c9cc7f071ffa935325_populate_sysroot.tar.zst anywhere. The paths that were searched were:
/media/mike/ssd/yocto/poky/build/sstate-cache
/media/mike/ssd/yocto/poky/build/sstate-cache
...
Michael Halstead changed the setup of the sstate server. |
I'm building the latest Poky ("core-image-minimal" plus a few extra packages), on Ubuntu 20.04. I'm always getting the below setscene errors after upgrading my Poky sources: ... <more similar errors before> WARNING: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache WARNING: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: No suitable staging package found ERROR: Logfile of failure stored in: /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libpcre2-native/10.39-r0/temp/log.do_populate_sysroot_setscene.315121 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/libpcre/libpcre2_10.39.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead Currently 4 running tasks (1838 of 1838/467 of 4630) 10% |################ The errors are not critical but paint my terminal with scary red messages. Here are my settings: BB_HASHSERVE = "auto" BB_SIGNATURE_HANDLER = "OEEquivHash" SSTATE_MIRRORS ?= "file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH" BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687" The errors go away after setting: BB_NUMBER_THREADS="1" (my system normally has 4 threads) Thanks to Jose Quaresma for the workaround! There may be an issue on the infrastructure, not support simultaneous requests from the same client... I'll be happy to make further tests if needed.