14310 2021-03-19 16:26:29 +0000 xz / liblzma memory allocation failure during reproducible build testing 2021-05-20 15:06:27 +0000 1 1 10 QA/Testing Build Testing general unspecified x86 Multiple RESOLVED FIXED AB-INT Medium+ normal 3.4 M1 1 saul.wold changqing.li changqing.li chee.yang.lee oobitots peter randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 89664 0 saul.wold 2021-03-19 16:26:29 +0000 During the reproducible build testing the following failure occured on the debian10-ty-1 machine (56cpu/128G) ram. There was a patch submitted about a year ago for a similar issue: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=27ff81bfd880280607c79dce2f724c8bfafce02d RPM build errors: Deprecated external dependency generator is used! Deprecated external dependency generator is used! Deprecated external dependency generator is used! Deprecated external dependency generator is used! Deprecated external dependency generator is used! liblzma: Memory allocation failed liblzma: Memory allocation failed liblzma: Memory allocation failed liblzma: Memory allocation failed liblzma: Memory allocation failedWARNING: exit code 1 from a shell command. And NOTE: Running task 10969 of 14696 (/home/pokybuild/yocto-worker/reproducible/build/meta/recipes-graphics/xorg-lib/libxv_1.0.11.bb:do_package_write_rpm) ERROR: libxkbfile-1_1.1.0-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/scripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot/usr/bin/crossscripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/reproducible/build/bitbake/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/hosttools" opkg-build -Z xz -a "--memlimit=50% --threads=56" libxkbfile-dbg /home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:xz: (stdin): Cannot allocate memory Command 'PATH="/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/scripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot/usr/bin/crossscripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/reproducible/build/bitbake/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/hosttools" opkg-build -Z xz -a "--memlimit=50% --threads=56" libxkbfile-src /home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:xz: (stdin): Cannot allocate memory Command 'PATH="/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/scripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot/usr/bin/crossscripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/reproducible/build/bitbake/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/hosttools" opkg-build -Z xz -a "--memlimit=50% --threads=56" libxkbfile /home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:xz: (stdin): Cannot allocate memory Command 'PATH="/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/scripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot/usr/bin/crossscripts:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/reproducible/build/bitbake/bin:/home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/hosttools" opkg-build -Z xz -a "--memlimit=50% --threads=56" libxkbfile-dev /home/pokybuild/yocto-worker/reproducible/build/build-st-13724/reproducibleB/tmp/work/core2-64-poky-linux/libxkbfile/1_1.1.0-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:xz: (stdin): Cannot allocate memory 89747 1 randy.macleod 2021-03-25 14:41:26 +0000 We have tunes xz down in the past to lower memory usage so perhaps we need to reduce it again. 89753 2 randy.macleod 2021-03-25 14:57:20 +0000 *** Bug 14319 has been marked as a duplicate of this bug. *** 89879 3 peter 2021-04-07 06:46:56 +0000 FWIW I also see similarities with another issue I had which I fixed in http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=6d14fe119c6e7d1d26ef33dfaf3b44ceff113546 90351 4 chee.yang.lee 2021-05-12 11:18:02 +0000 happen during build qemumips64 at ubuntu1804-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3412/steps/12/logs/stdio qemux86-world at ubuntu1804-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/52/builds/3401/steps/11/logs/stdio 90357 5 chee.yang.lee 2021-05-13 09:57:53 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/3407/steps/12/logs/stdio qemuppc ubuntu1804-ty-1 90428 6 randy.macleod 2021-05-20 15:06:27 +0000 http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/commit/?id=19b4e74b3960174238acc79fd112f55706bc7385 config.json: Set XZ limits to more reasonable values on autobuilderHEADmaster The autobuilders have 128GB memory, we don't want them using 50% which is the default, 5% should be enough. Also limit the number of threads down from 48 to something reasonable. This may be partly causing some of our performance issues? Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>