3315 2012-10-20 18:46:46 +0000 Confusing error message if build/tmp/deploy is deleted between runs 2013-02-18 05:20:19 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple RESOLVED FIXED Low normal 1.4 1 evadeflow richard.purdie poky.bs.watcher poky.watcher sgw --- --- --- oldest_to_newest 26783 0 evadeflow 2012-10-20 18:46:46 +0000 If I delete the tmp/deploy folder from the build directory and re-run bitbake, it fails with a ton of scary output (see below). Steps to reproduce: > % source oe-init-build-env > % bitbake -c rootfs core-image-minimal > % rm -rf tmp/deploy > % bitbake -c rootfs core-image-minimal Setting aside the question of why I did this (ANSWER: I'm a n00b and don't know any better), I'm wondering: 1. Can bitbake really not recover from this? If not: 2. Can the error message below be improved? ---------- > % bitbake -c rootfs core-image-minimal > Loading cache: 100% > |############################################################################| > ETA: 00:00:00 > Loaded 1120 entries from dependency cache. > > Build Configuration: > BB_VERSION = "1.16.0" > TARGET_ARCH = "arm" > TARGET_OS = "linux-gnueabi" > MACHINE = "tegra" > DISTRO = "poky" > DISTRO_VERSION = "1.3+snapshot-20121020" > TUNE_FEATURES = "armv7a vfp callconvention-hard cortexa9" > TARGET_FPU = "vfp" > meta = "master:caba9cbfce09f19eb27f4c6615c0c5c48e1a2952" > meta-tegra = "master:dc9d4624c95961c330ddbaee6cbab13fb9411c4a" > meta-yocto > meta-yocto-bsp = "master:caba9cbfce09f19eb27f4c6615c0c5c48e1a2952" > > NOTE: Resolving any missing task queue dependencies > NOTE: preferred version 2.6% of linux-yocto not available (for item > virtual/kernel) > NOTE: versions of linux-yocto available: > 3.4.11+git1+a201268353c030d9fafe00f2041976f7437d9386_1+449f7f520350700858f21a5554b81cc8ad23267d > NOTE: preferred version 2.6% of linux-yocto not available (for item > kernel-base) > NOTE: versions of linux-yocto available: > 3.4.11+git1+a201268353c030d9fafe00f2041976f7437d9386_1+449f7f520350700858f21a5554b81cc8ad23267d > NOTE: preferred version 2.6% of linux-yocto not available (for item > kernel-image) > NOTE: versions of linux-yocto available: > 3.4.11+git1+a201268353c030d9fafe00f2041976f7437d9386_1+449f7f520350700858f21a5554b81cc8ad23267d > NOTE: Preparing runqueue > NOTE: Executing SetScene Tasks > NOTE: Executing RunQueue Tasks > ERROR: Function failed: do_rootfs (see > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/temp/log.do_rootfs.8152 > for further information) > ERROR: Logfile of failure stored in: > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/temp/log.do_rootfs.8152 > Log data follows: > | DEBUG: Executing shell function do_rootfs > | + rm -f > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/opkg.conf > | + touch > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/opkg.conf > | + package_update_index_ipk > | + set -x > | + ipkgarchs='all any noarch armv5hf-vfp armv5ehf-vfp armv6hf-vfp > armv7ahf-vfp tegra all any noarch i686-nativesdk' > | + '[' '!' -z '' ']' > | + packagedirs=/home/evadeflow/projects/poky/build/tmp/deploy/ipk > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch' > | + for arch in '$ipkgarchs' > | + packagedirs='/home/evadeflow/projects/poky/build/tmp/deploy/ipk > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/i686-nativesdk' > | + multilib_archs= > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch/ ']' > | + for pkgdir in '$packagedirs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/i686-nativesdk/ ']' > | + package_generate_ipkg_conf > | + package_generate_archlist > | + ipkgarchs='all any noarch i686-nativesdk' > | + priority=1 > | + for arch in '$ipkgarchs' > | + echo 'arch all 1' > | ++ expr 1 + 5 > | + priority=6 > | + for arch in '$ipkgarchs' > | + echo 'arch any 6' > | ++ expr 6 + 5 > | + priority=11 > | + for arch in '$ipkgarchs' > | + echo 'arch noarch 11' > | ++ expr 11 + 5 > | + priority=16 > | + for arch in '$ipkgarchs' > | + echo 'arch i686-nativesdk 16' > | ++ expr 16 + 5 > | + priority=21 > | + ipkgarchs='all any noarch armv5hf-vfp armv5ehf-vfp armv6hf-vfp > armv7ahf-vfp tegra' > | + priority=1 > | + for arch in '$ipkgarchs' > | + echo 'arch all 1' > | ++ expr 1 + 5 > | + priority=6 > | + for arch in '$ipkgarchs' > | + echo 'arch any 6' > | ++ expr 6 + 5 > | + priority=11 > | + for arch in '$ipkgarchs' > | + echo 'arch noarch 11' > | ++ expr 11 + 5 > | + priority=16 > | + for arch in '$ipkgarchs' > | + echo 'arch armv5hf-vfp 16' > | ++ expr 16 + 5 > | + priority=21 > | + for arch in '$ipkgarchs' > | + echo 'arch armv5ehf-vfp 21' > | ++ expr 21 + 5 > | + priority=26 > | + for arch in '$ipkgarchs' > | + echo 'arch armv6hf-vfp 26' > | ++ expr 26 + 5 > | + priority=31 > | + for arch in '$ipkgarchs' > | + echo 'arch armv7ahf-vfp 31' > | ++ expr 31 + 5 > | + priority=36 > | + for arch in '$ipkgarchs' > | + echo 'arch tegra 36' > | ++ expr 36 + 5 > | + priority=41 > | + echo 'src oe file:/home/evadeflow/projects/poky/build/tmp/deploy/ipk' > | + ipkgarchs='all any noarch i686-nativesdk' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/i686-nativesdk/Packages ']' > | + echo 'src oe file:/home/evadeflow/projects/poky/build/tmp/deploy/ipk' > | + ipkgarchs='all any noarch armv5hf-vfp armv5ehf-vfp armv6hf-vfp > armv7ahf-vfp tegra' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/all/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/any/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/noarch/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5hf-vfp/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv5ehf-vfp/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv6hf-vfp/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/armv7ahf-vfp/Packages ']' > | + for arch in '$ipkgarchs' > | + '[' -e /home/evadeflow/projects/poky/build/tmp/deploy/ipk/tegra/Packages ']' > | + mkdir -p > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/temp/ > | + > STATUS=/home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/rootfs/var/lib/opkg/status > | + mkdir -p > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/rootfs/var/lib/opkg > | + opkg-cl -f > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/opkg.conf > -o > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/rootfs > --force-overwrite --prefer-arch-to-version update > | Downloading file:/home/evadeflow/projects/poky/build/tmp/deploy/ipk/Packages. > | Collected errors: > | * copy_file: /home/evadeflow/projects/poky/build/tmp/deploy/ipk/Packages: No > such file or directory. > | * file_copy: Failed to copy file > /home/evadeflow/projects/poky/build/tmp/deploy/ipk/Packages to > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/rootfs//var/lib/opkg/lists/oe. > | ERROR: Function failed: do_rootfs (see > /home/evadeflow/projects/poky/build/tmp/work/tegra-poky-linux-gnueabi/core-image-minimal-1.0-r0/temp/log.do_rootfs.8152 > for further information) > ERROR: Task 6 > (/home/evadeflow/projects/poky/meta/recipes-core/images/core-image-minimal.bb, > do_rootfs) failed with exit code '1' > NOTE: Tasks Summary: Attempted 1183 tasks of which 1182 didn't need to be > rerun and 1 failed. > > Summary: 1 task failed: > /home/evadeflow/projects/poky/meta/recipes-core/images/core-image-minimal.bb, > do_rootfs > Summary: There was 1 WARNING message shown. > Summary: There was 1 ERROR message shown, returning a non-zero exit code. 26833 1 richard.purdie 2012-10-23 08:56:19 +0000 (In reply to comment #0) > If I delete the tmp/deploy folder from the build directory and re-run > bitbake, it fails with a ton of scary output (see below). > > Steps to reproduce: > > > % source oe-init-build-env > > % bitbake -c rootfs core-image-minimal > > % rm -rf tmp/deploy > > % bitbake -c rootfs core-image-minimal > > Setting aside the question of why I did this (ANSWER: I'm a n00b and don't > know any better), I'm wondering: > > 1. Can bitbake really not recover from this? If not: > 2. Can the error message below be improved? The build system is an extremely complex piece of code which has many different demands being placed on it which include some that cover performance. When you run a bitbake command, its not possible to perform an integrity check on the whole system and ensure files are consistent. The problem you're running into is that the stamps directory says one thing (tmp/delpoy/ipk is full of .ipk files) yet you went and changed that behind the system's back. There are 101 ways where you can delete files and cause the system to break in interesting ways. The answer is more about encouraging people *not to delete files from under bitbake* since I don't think it would be possible to validate every possibile combination, its simply not supported and if we did do it, we'd kill performance. FWIW we do support removing the whole of tmp/ and the system will reuse sstate components to rebuild it. Whilst I dislike it, we have have "rm_work" as a mechanism for clearing space during builds. 26853 2 evadeflow 2012-10-23 22:44:56 +0000 (In reply to comment #1) > The build system is an extremely complex piece of code which has many > different demands being placed on it... > > There are 101 ways where you can delete files and cause the system to break > in interesting ways. That's fair. I'm learning just how difficult it is to solve the problems that bitbake (+Yocto) is intended to solve. The fact that it solves them at *all* is increasingly impressive to me. Up to now, I've only worked with LTIB and LDAT; they're not any easier to use, and they won't build an entire cross toolchain for you(!) Yocto is pretty awesome, really. > FWIW we do support removing the whole of tmp/ and the system will reuse > sstate components to rebuild it. I knew this, and that's actually what made me try deleting only tmp/deploy. I was thinking it would behave like 'make', where removing an object that another depends on will cause that object to be regenerated. This doesn't seem like a crazy expectation, but I can see how trying to protect users from every single 'weird' thing they might try would be intractable. I'm only mentioning this particular item in case it's helpful in identifying trouble spots for new users. I won't be at all offended if you want to close this ticket. (That goes for any tickets I open while I'm still in my 'stumbling around' phase with Yocto. I'm hoping the feedback will be helfpul, but I understand it's not realistic to solve every little problem a new user might encounter.) 29888 3 richard.purdie 2013-02-17 09:46:10 +0000 I've put a patch out which makes the system error if it can't find any packages. I think that is as close as we can get to resolving this. 29899 4 richard.purdie 2013-02-18 05:20:19 +0000 Fixed in master http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=74938e387c19c7b72e1553d4148a58a5a3d1ceff