10159 2016-08-18 12:30:31 +0000 [rm_work] building VDI image twice fails with rm_work enabled 2016-08-24 11:09:12 +0000 1 1 7 Build System, Metadata & Runtime OE-Core deployment 2.1 x86 x86 RESOLVED FIXED Medium normal 2.2 1 net147 eduard.bartosh jatedev liezhi.yang richard.purdie ross.burton sgw stephano --- Regression (Used to work) No (bug/feature does not impact docs) oldest_to_newest 65295 0 net147 2016-08-18 12:30:31 +0000 Commands to reproduce: git clone git://git.yoctoproject.org/poky cd poky . oe-init-build-env cat >> conf/local.conf << 'EOF' RM_OLD_IMAGE = "1" PRSERV_HOST = "localhost:0" INHERIT += "buildhistory" BUILDHISTORY_COMMIT = "1" INHERIT += "rm_work" IMAGE_FSTYPES = "vdi" EOF bitbake core-image-minimal bitbake core-image-minimal NOTE: Started PRServer with DBfile: [...]/poky/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 40437, PID: 31804 WARNING: Host distribution "ArchLinux" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution. Parsing recipes: 100% |#######################################################################################################################| Time: 0:00:12 Parsing of 864 .bb files complete (0 cached, 864 parsed). 1313 targets, 47 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.31.1" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "i586-poky-linux" MACHINE = "qemux86" DISTRO = "poky" DISTRO_VERSION = "2.1+snapshot-20160818" TUNE_FEATURES = "m32 i586" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "master:6c24a6446a2e5310088bd13c0deaa2d2ac3484c9" Initialising tasks: 100% |####################################################################################################################| Time: 0:00:03 NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks ERROR: core-image-minimal-1.0-r0 do_bootdirectdisk: Function failed: Unable to open [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/syslinux_vm.cfg ERROR: Logfile of failure stored in: [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_bootdirectdisk.495 Log data follows: | DEBUG: Executing python function do_bootdirectdisk | DEBUG: Executing python function build_syslinux_cfg | DEBUG: Python function build_syslinux_cfg finished | DEBUG: Python function do_bootdirectdisk finished | ERROR: Function failed: Unable to open [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/syslinux_vm.cfg ERROR: Task [...]/poky/meta/recipes-core/images/core-image-minimal.bb:do_bootdirectdisk ([...]/poky/meta/recipes-core/images/core-image-minimal.bb:do_bootdirectdisk) failed with exit code '1' NOTE: Tasks Summary: Attempted 2337 tasks of which 2336 didn't need to be rerun and 1 failed. NOTE: Writing buildhistory Summary: 1 task failed: [...]/poky/meta/recipes-core/images/core-image-minimal.bb:do_bootdirectdisk Summary: There was 1 WARNING message shown. Summary: There was 1 ERROR message shown, returning a non-zero exit code. 65297 1 net147 2016-08-18 12:48:52 +0000 This seems to be a regression caused by http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=efc07c293695c21329442d69fe34411027416188. The issue disappears after reverting the commit. 65343 2 eduard.bartosh 2016-08-19 12:06:20 +0000 I can reproduce the issue and confirm that it's caused by commit efc07c293695c21329442d69fe34411027416188 The fix for syslinux failure is quite straightforward. see attached patch. However, I suspect other bootloaders, buildhistory, wic-related and possibly other tasks to expect rootfs directory and files to be present. So, the question is should we revert efc07c293695c21329442d69fe34411027416188 or should we get ready to fix similar issues when they appear? 65344 3 3369 eduard.bartosh 2016-08-19 12:07:20 +0000 Created attachment 3369 fix v.1 65345 4 net147 2016-08-19 12:12:18 +0000 (In reply to comment #2) > I can reproduce the issue and confirm that it's caused by commit > efc07c293695c21329442d69fe34411027416188 > > The fix for syslinux failure is quite straightforward. see attached patch. > > However, I suspect other bootloaders, buildhistory, wic-related and possibly > other tasks to expect rootfs directory and files to be present. > > So, the question is should we revert > efc07c293695c21329442d69fe34411027416188 or should we get ready to fix > similar issues when they appear? I get the following after applying your fix and running "bitbake -c cleanall core-image-minimal; bitbake core-image-minimal; bitbake core-image-minimal": ERROR: core-image-minimal-1.0-r0 do_image_complete: Function failed: buildhistory_get_imageinfo (log file is located at [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_image_complete.28557) ERROR: Logfile of failure stored in: [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_image_complete.28557 Log data follows: | DEBUG: Executing python function do_image_complete | NOTE: Executing write_deploy_manifest ... | DEBUG: Executing python function write_deploy_manifest | DEBUG: Python function write_deploy_manifest finished | NOTE: Executing buildhistory_get_imageinfo ... | DEBUG: Executing shell function buildhistory_get_imageinfo | [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/run.buildhistory_get_imageinfo.28557: line 171: cd: [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs: No such file or directory | WARNING: [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/run.buildhistory_get_imageinfo.28557:1 exit 1 from '[ "$3" = "fakeroot" ]' | DEBUG: Python function do_image_complete finished | ERROR: Function failed: buildhistory_get_imageinfo (log file is located at [...]/poky/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_image_complete.28557) ERROR: Task [...]/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_complete ([...]/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_complete) failed with exit code '1' 65346 5 eduard.bartosh 2016-08-19 12:14:02 +0000 yep, this is exactly what I was talking about. buildhistory expects roots directory and files to exist. 65348 6 richard.purdie 2016-08-19 12:22:26 +0000 Firstly, functions should be creating directories they write too, they shouldn't assume something else did it for them. Rather that os.makedirs, you can use the function [dirs] flag. Secondly, I think that commit being pointed too is missing a *do_image_* which is likely needed after the split of do_rootfs into separate tasks. If we do that, I suspect it would solve a lot of the issues mentioned here? 65372 7 net147 2016-08-20 08:37:07 +0000 I think the images shouldn't be rebuilt in the first place as there are no changes on the second run of "bitbake core-image-minimal". 65373 8 net147 2016-08-20 10:52:10 +0000 (In reply to comment #7) > I think the images shouldn't be rebuilt in the first place as there are no > changes on the second run of "bitbake core-image-minimal". A closer looks shows that rm_work.bbclass is removing the stamps for the following tasks: * do_bootimg (image-live.bbclass) * do_bootdirectdisk (image-vm.bbclass) * do_vmimg (image-vm.bbclass) The stamps being removed causes the tasks to always execute even when there are no changes to the image when rm_work is enabled. A quick workaround would be to exclude those from do_rm_work() in rm_work.bbclass just like we do for do_image. A better solution would be if we can refactor image-live.bbclass and image-vm.bbclass to use IMAGE_CMD and avoid manually adding tasks (e.g. http://git.yoctoproject.org/cgit/cgit.cgi/meta-raspberrypi/tree/classes/sdcard_image-rpi.bbclass). 65383 9 eduard.bartosh 2016-08-22 08:46:13 +0000 first fix has been merged to master: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=f019bb933f8016d33937d755887700f9a446408e 65384 10 eduard.bartosh 2016-08-22 08:48:11 +0000 > A quick workaround would be to exclude those from do_rm_work() in > rm_work.bbclass just like we do for do_image. A better solution would be if > we can refactor image-live.bbclass and image-vm.bbclass to use IMAGE_CMD and > avoid manually adding tasks (e.g. > http://git.yoctoproject.org/cgit/cgit.cgi/meta-raspberrypi/tree/classes/ > sdcard_image-rpi.bbclass). Thank you for the investigation! I'm for a quick fix for now. Feel free to create new bugs if you want refactoring work to be done. 65386 11 eduard.bartosh 2016-08-22 09:32:01 +0000 Quick fix has been sent for review: http://lists.openembedded.org/pipermail/openembedded-core/2016-August/125456.html 65481 12 eduard.bartosh 2016-08-24 11:09:12 +0000 the fix has been merged to poky master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1a620968b1775f5fde27fa513ba7ef0eed501785 3369 2016-08-19 12:07:20 +0000 2016-08-19 12:07:20 +0000 fix v.1 file_10159.txt text/plain 493 eduard.bartosh ZGlmZiAtLWdpdCBhL21ldGEvY2xhc3Nlcy9zeXNsaW51eC5iYmNsYXNzIGIvbWV0YS9jbGFzc2Vz L3N5c2xpbnV4LmJiY2xhc3MNCmluZGV4IGZhM2UzZWMuLmJlYTE4YTYgMTAwNjQ0DQotLS0gYS9t ZXRhL2NsYXNzZXMvc3lzbGludXguYmJjbGFzcw0KKysrIGIvbWV0YS9jbGFzc2VzL3N5c2xpbnV4 LmJiY2xhc3MNCkBAIC0xMDIsNiArMTAyLDEwIEBAIHB5dGhvbiBidWlsZF9zeXNsaW51eF9jZmcg KCkgew0KICAgICBpZiBub3QgY2ZpbGU6DQogICAgICAgICByYWlzZSBiYi5idWlsZC5GdW5jRmFp bGVkKCdVbmFibGUgdG8gcmVhZCBTWVNMSU5VWF9DRkcnKQ0KIA0KKyAgICBjZGlyID0gb3MucGF0 aC5kaXJuYW1lKGNmaWxlKQ0KKyAgICBpZiBub3Qgb3MucGF0aC5pc2RpcihjZGlyKToNCisgICAg ICAgIG9zLm1ha2VkaXJzKGNkaXIpDQorDQogICAgIHRyeToNCiAgICAgICAgIGNmZ2ZpbGUgPSBv cGVuKGNmaWxlLCAndycpDQogICAgIGV4Y2VwdCBPU0Vycm9yOg==