1250 2011-07-15 16:23:01 +0000 DEPENDS not honored when rebuilding from sstate with SSTATEPOSTINSTFUNCS 2011-07-25 08:11:45 +0000 1 1 7 Build System, Metadata & Runtime Meta-yocto meta-yocto 1.0 x86 Multiple RESOLVED FIXED High major 1.1 M3 1 scott.a.garman sgw poky.bs.watcher poky.watcher sgw --- --- --- oldest_to_newest 14748 0 scott.a.garman 2011-07-15 16:23:01 +0000 The following scenario was identified on the autobuilder during this build: http://autobuilder.pokylinux.org:8010/builders/crownbay/builds/25/steps/shell_17/logs/stdio At line 1711, docbook-sgml-dtd-3.1-native.bb failed because install-catalog did not exist. install-catalog is provided by sgml-common-native in the native sysroot, and the docbook-sgml-dtd-native.inc recipe has a DEPENDS = "sgml-common-native" in it. However, if you look at that log, you'll find that sgml-common-native's do_populate_sysroot is not run until line 1755. This seems to indicate that the DEPENDS is not being properly honored. Of note is that this build is being done from an empty TMPDIR but a fully populated sstate. 14754 1 sgw 2011-07-16 17:35:33 +0000 I was able to reproduce this even after adding a direct depends on populate_sysroot do_unpack[depends] += "sgml-common-native:do_populate_sysroot" Was added to openjade, docbook-sgml-dtd, and docbook-dsssl-sytlesheets. I then did a cleansstate on the following recipes: bitbake openjade-native sgml-common-native docbook-dsssl-stylesheets-native docbook-sgml-dtd-4.5-native docbook-utils-native -c cleanstate Followed by a bitbake of docbook-utils-native (has dependencies on the other docbook items). Then I did a bitbake -c clean of the above list and tried to build docbook-utils-native again, and I could reproduce the failure right away. bitbake docbook-utils-native Loading cache: 100% |####################################################################################################| ETA: 00:00:00 Loaded 1028 entries from dependency cache. OE Build Configuration: BB_VERSION = "1.13.2" TARGET_ARCH = "x86_64" TARGET_OS = "linux" MACHINE = "sugarbay" DISTRO = "poky" DISTRO_VERSION = "1.0+snapshot-20110717" TARGET_FPU = "" meta meta-yocto = "rc3:aca9d745202ebdc6d3638a62c22af7ab8fb9b1a1" meta-sugarbay = "master:92fc07a5f1b98779806cdcc2341487ff5ea5a238" NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Running setscene task 14 of 19 (/intel/poky/distro/meta/recipes-devtools/docbook-utils/docbook-utils-native_0.6.14.bb:do_populate_sysroot_setscene) NOTE: Running setscene task 15 of 19 (/intel/poky/distro/meta/recipes-devtools/docbook-utils/docbook-utils-native_0.6.14.bb:do_populate_lic_setscene) NOTE: Running setscene task 16 of 19 (/intel/poky/distro/meta/recipes-devtools/openjade/openjade-native_1.3.2.bb:do_populate_sysroot_setscene) NOTE: Running setscene task 17 of 19 (/intel/poky/distro/meta/recipes-devtools/docbook-dsssl-stylesheets/docbook-dsssl-stylesheets-native_1.79.bb:do_populate_sysroot_setscene) NOTE: package openjade-native-1.3.2-r3: task do_populate_sysroot_setscene: Started NOTE: package docbook-dsssl-stylesheets-native-1.79-r3: task do_populate_sysroot_setscene: Started NOTE: package docbook-utils-native-0.6.14-r1: task do_populate_sysroot_setscene: Started NOTE: package docbook-utils-native-0.6.14-r1: task do_populate_lic_setscene: Started NOTE: package docbook-utils-native-0.6.14-r1: task do_populate_sysroot_setscene: Succeeded NOTE: package docbook-utils-native-0.6.14-r1: task do_populate_lic_setscene: Succeeded ERROR: Task 65 (/intel/poky/distro/meta/recipes-devtools/docbook-dsssl-stylesheets/docbook-dsssl-stylesheets-native_1.79.bb, do_populate_sysroot_setscene) failed with exit code '1' ERROR: Task 51 (/intel/poky/distro/meta/recipes-devtools/openjade/openjade-native_1.3.2.bb, do_populate_sysroot_setscene) failed with exit code '1' NOTE: Running setscene task 19 of 19 (/intel/poky/distro/meta/recipes-devtools/sgml-common/sgml-common-native_0.6.3.bb:do_populate_sysroot_setscene) NOTE: package sgml-common-native-0.6.3-r0: task do_populate_sysroot_setscene: Started NOTE: package sgml-common-native-0.6.3-r0: task do_populate_sysroot_setscene: Succeeded NOTE: Executing RunQueue Tasks NOTE: Running noexec task 110 of 111 (ID: 8, /intel/poky/distro/meta/recipes-devtools/docbook-utils/docbook-utils-native_0.6.14.bb, do_package_write) NOTE: Running noexec task 111 of 111 (ID: 5, /intel/poky/distro/meta/recipes-devtools/docbook-utils/docbook-utils-native_0.6.14.bb, do_build) NOTE: Tasks Summary: Attempted 111 tasks of which 109 didn't need to be rerun and 0 failed. 14758 2 richard.purdie 2011-07-18 06:15:58 +0000 By definition providing a sstate package means you that DEPENDS need not be honored. This isn't therefore a bug in the dependency handling but in the way sstate is being used :/ 14759 3 richard.purdie 2011-07-18 06:26:35 +0000 Another worrying factor is that even if the sstate package fails, it should just get rebuilt and things should continue so this error should not be fatal... 14760 4 richard.purdie 2011-07-18 07:00:44 +0000 Ok, I propose that we: a) Have any of the recipes with these postinstalls take a copy of install-catalog out of the sysroot and place it into recipe's sysroot contents with a suitable suffix, something like: SYSROOT_PREPROCESS_FUNCS += "xxx_sysroot_preprocess" xxx_sysroot_preprocess () { install -d ${SYSROOT_DESTDIR}${bindir_crossscripts}/ install -m 755 ${STAGING_BINDIR_NATIVE}/install-catalog ${SYSROOT_DESTDIR}${bindir_crossscripts}/install-catalog-xxx } which will always ensure this is present when the sstate package is processed. This could perhaps now best be done as a bbclass. b) We need to figure out why when the sstate package failed to install the system didn't recover as it should have done... 14761 5 richard.purdie 2011-07-18 08:02:31 +0000 On the failed sstate package handling issue, there is a fundamental problem with some of the assumptions sstate operates under. For target packages we'd never run them so its safe to assume that any (build) DEPENDS are now unneeded and also runtime RDEPENDS and so forth are not required. Its these cases the design of sstate is catering well for. The problem comes with native/cross sstate packages where build time dependencies shouldn't be required but runtime dependencies can be. Worse still, looking at RDEPENDS isn't enough since we autodetect shared library dependencies so the RDEPENDS information in the metadata isn't complete. The easiest solution is likely to make sstate start looking at DEPENDS+RDEPENDS in the native/cross case. Its worth nothing that the checksum code gives a significant amount of protection assuming sstate data is mirrored completely as only in the case of a failed package installation would an issue arise. 14971 6 sgw 2011-07-25 08:11:45 +0000 I implemented Richards suggestions from above in the following commits: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=afa695fd317f5de6cee803091b3c8fb25948237e http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e9e77799a1b9dd84c1d83cbc1efcf5bdb604f7a7 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f9fe0ae3c88ab4be954890df0053fc9c11cfc8a5