| Summary: | bitbake-1.24 cache producing different sstate signatures | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] BitBake | Reporter: | Martin Jansa <Martin.Jansa> | ||||||
| Component: | bitbake | Assignee: | Richard Purdie <richard.purdie> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | normal | ||||||||
| Priority: | Medium | CC: | poky.bs.watcher, poky.watcher | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | 1.8 | ||||||||
| Hardware: | x86 | ||||||||
| OS: | Multiple | ||||||||
| Whiteboard: | |||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||||
| Attachments: |
|
||||||||
|
Description
Martin Jansa
2015-01-28 07:47:56 UTC
It's a bit more complicated then described. It looks like the cache issue is still there in current bitbake master + oe-core master. I wasn't able to reproduce it in master, because: http://git.openembedded.org/openembedded-core/commit/?id=20b0168da47d6e30fcbaf6adab3bde0d398d0d00 also fixes the icecc different signatures issue (I had set_icecc_env[vardepsexclude] += "TARGET_FPU ARMPKGSFX_FPU" in my pending fix) So I've tested this with 1.24 + dizzy, 1.24 + master, master + master, master + master-revert-icecc-fix. Different signatures are in 1.24 + dizzy, 1.24 + master, master + master-revert-icecc-fix, but in all 3 cases detected only after cleaning the cache. icecc/1.24_oe-core-dizzy/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.9119854ccb782a2e4b1e4dc1ac784d72 icecc/1.24_oe-core-dizzy/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.a262909ea37e69d3a408e87858ac4bbd icecc/1.24_oe-core-dizzy/sstate-diff-icecc-enabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.a262909ea37e69d3a408e87858ac4bbd icecc/1.24_oe-core-dizzy/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.9119854ccb782a2e4b1e4dc1ac784d72 icecc/1.24_oe-core-master/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.9119854ccb782a2e4b1e4dc1ac784d72 icecc/1.24_oe-core-master/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.a262909ea37e69d3a408e87858ac4bbd icecc/1.24_oe-core-master/sstate-diff-icecc-enabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.a262909ea37e69d3a408e87858ac4bbd icecc/1.24_oe-core-master/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.9119854ccb782a2e4b1e4dc1ac784d72 icecc/master_oe-core-master-revert-icecc-fix/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.97e0ab9346c2be49aa6750be02d9ba91 icecc/master_oe-core-master-revert-icecc-fix/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.d9fe2c6712d0ad461e9efcc951855d2a icecc/master_oe-core-master-revert-icecc-fix/sstate-diff-icecc-enabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.d9fe2c6712d0ad461e9efcc951855d2a icecc/master_oe-core-master-revert-icecc-fix/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.97e0ab9346c2be49aa6750be02d9ba91 icecc/master_oe-core-master/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master/sstate-diff-icecc-enabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 master+master once again to be sure: icecc/master_oe-core-master1/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master1/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master1/sstate-diff-icecc-enabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 icecc/master_oe-core-master1/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 Added ARMPKGSFX_FPU variable is shown by bitbake-diffsigs even for sigdata files with the same checksum, e.g.: $ bitbake-diffsigs \ icecc/master_oe-core-master1/sstate-diff-icecc-disabled-clean-cache/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 \ icecc/master_oe-core-master1/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 List of dependencies for variable set_icecc_env changed from 'set(['CPPFLAGS', 'docdir', 'TARGET_CPPFLAGS', 'ICECC_CXX', 'PATCH_GET', 'BUILD_CXX', 'sharedstatedir', 'icc_get_and_check_tool', 'TARGET_LDFLAGS', 'localstatedir', 'ICE_PATH', 'BUILD_LD', 'CXXFLAGS', 'PKG_CONFIG_PATH', 'prefix', 'FC', 'ICECC_CC', 'bbnote', 'LC_ALL', 'CPP', 'icc_path', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'mandir', 'LD', 'BUILD_RANLIB', 'NM', 'infodir', 'BUILD_CFLAGS', 'exec_prefix', 'BUILD_FC', 'BUILD_CCLD', 'libdir', 'bbwarn', 'datadir', 'LDFLAGS', 'STRIP', 'use_icc', 'sbindir', 'oldincludedir', 'BUILD_AR', 'base_prefix', 'ICECC_AS', 'PKG_CONFIG_LIBDIR', 'ARMPKGSFX_FPU', 'CC', 'BUILD_LDFLAGS', 'nonarch_base_libdir', 'RANLIB', 'CFLAGS', 'AS', 'AR', 'ICECC_VERSION', 'BUILD_CXXFLAGS', 'BUILD_NM', 'TARGET_CXXFLAGS', 'icc_version', 'MAKE', 'STRINGS', 'PSEUDO_DISABLED', 'base_bindir', 'BUILD_CC', 'PKG_CONFIG_SYSROOT_DIR', 'base_sbindir', 'systemd_unitdir', 'BUILD_CPPFLAGS', 'OBJDUMP', 'sysconfdir', 'base_libdir', 'TARGET_CFLAGS', 'CXX', 'includedir', 'servicedir', 'bindir', 'CCLD', 'PKG_CONFIG_DIR', 'BUILD_STRIP', 'libexecdir', 'OBJCOPY', 'ICECC_WHICH_AS', 'BUILD_CPP', 'BUILD_AS'])' to 'set(['CPPFLAGS', 'docdir', 'TARGET_CPPFLAGS', 'ICECC_CXX', 'PATCH_GET', 'BUILD_CXX', 'sharedstatedir', 'icc_get_and_check_tool', 'TARGET_LDFLAGS', 'localstatedir', 'ICE_PATH', 'BUILD_LD', 'CXXFLAGS', 'PKG_CONFIG_PATH', 'prefix', 'FC', 'ICECC_CC', 'bbnote', 'LC_ALL', 'CPP', 'icc_path', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'mandir', 'LD', 'BUILD_RANLIB', 'NM', 'infodir', 'BUILD_CFLAGS', 'exec_prefix', 'BUILD_FC', 'BUILD_CCLD', 'libdir', 'bbwarn', 'datadir', 'LDFLAGS', 'STRIP', 'use_icc', 'sbindir', 'oldincludedir', 'BUILD_AR', 'base_prefix', 'ICECC_AS', 'PKG_CONFIG_LIBDIR', 'CC', 'BUILD_LDFLAGS', 'nonarch_base_libdir', 'RANLIB', 'CFLAGS', 'AS', 'AR', 'ICECC_VERSION', 'BUILD_CXXFLAGS', 'BUILD_NM', 'TARGET_CXXFLAGS', 'icc_version', 'MAKE', 'STRINGS', 'PSEUDO_DISABLED', 'base_bindir', 'BUILD_CC', 'PKG_CONFIG_SYSROOT_DIR', 'base_sbindir', 'systemd_unitdir', 'BUILD_CPPFLAGS', 'OBJDUMP', 'sysconfdir', 'base_libdir', 'TARGET_CFLAGS', 'CXX', 'includedir', 'servicedir', 'bindir', 'CCLD', 'PKG_CONFIG_DIR', 'BUILD_STRIP', 'libexecdir', 'OBJCOPY', 'ICECC_WHICH_AS', 'BUILD_CPP', 'BUILD_AS'])' changed items: set(['ARMPKGSFX_FPU']) $ bitbake-diffsigs \ icecc/master_oe-core-master/sstate-diff-icecc-disabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 \ icecc/master_oe-core-master/sstate-diff-icecc-enabled/qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.ca2db5ec0d680c8fbbfd6f94add0d1a5 List of dependencies for variable set_icecc_env changed from 'set(['CPPFLAGS', 'docdir', 'TARGET_CPPFLAGS', 'ICECC_CXX', 'PATCH_GET', 'BUILD_CXX', 'sharedstatedir', 'icc_get_and_check_tool', 'TARGET_LDFLAGS', 'localstatedir', 'ICE_PATH', 'BUILD_LD', 'CXXFLAGS', 'PKG_CONFIG_PATH', 'prefix', 'FC', 'ICECC_CC', 'bbnote', 'LC_ALL', 'CPP', 'icc_path', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'mandir', 'LD', 'BUILD_RANLIB', 'NM', 'infodir', 'BUILD_CFLAGS', 'exec_prefix', 'BUILD_FC', 'BUILD_CCLD', 'libdir', 'bbwarn', 'datadir', 'LDFLAGS', 'STRIP', 'use_icc', 'sbindir', 'oldincludedir', 'BUILD_AR', 'base_prefix', 'ICECC_AS', 'PKG_CONFIG_LIBDIR', 'CC', 'BUILD_LDFLAGS', 'nonarch_base_libdir', 'RANLIB', 'CFLAGS', 'AS', 'AR', 'ICECC_VERSION', 'BUILD_CXXFLAGS', 'BUILD_NM', 'TARGET_CXXFLAGS', 'icc_version', 'MAKE', 'STRINGS', 'PSEUDO_DISABLED', 'base_bindir', 'BUILD_CC', 'PKG_CONFIG_SYSROOT_DIR', 'base_sbindir', 'systemd_unitdir', 'BUILD_CPPFLAGS', 'OBJDUMP', 'sysconfdir', 'base_libdir', 'TARGET_CFLAGS', 'CXX', 'includedir', 'servicedir', 'bindir', 'CCLD', 'PKG_CONFIG_DIR', 'BUILD_STRIP', 'libexecdir', 'OBJCOPY', 'ICECC_WHICH_AS', 'BUILD_CPP', 'BUILD_AS'])' to 'set(['CPPFLAGS', 'docdir', 'TARGET_CPPFLAGS', 'ICECC_CXX', 'PATCH_GET', 'BUILD_CXX', 'sharedstatedir', 'icc_get_and_check_tool', 'TARGET_LDFLAGS', 'localstatedir', 'ICE_PATH', 'BUILD_LD', 'CXXFLAGS', 'PKG_CONFIG_PATH', 'prefix', 'FC', 'ICECC_CC', 'bbnote', 'LC_ALL', 'CPP', 'icc_path', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'mandir', 'LD', 'BUILD_RANLIB', 'NM', 'infodir', 'BUILD_CFLAGS', 'exec_prefix', 'BUILD_FC', 'BUILD_CCLD', 'libdir', 'bbwarn', 'datadir', 'LDFLAGS', 'STRIP', 'use_icc', 'sbindir', 'oldincludedir', 'BUILD_AR', 'base_prefix', 'ICECC_AS', 'PKG_CONFIG_LIBDIR', 'ARMPKGSFX_FPU', 'CC', 'BUILD_LDFLAGS', 'nonarch_base_libdir', 'RANLIB', 'CFLAGS', 'AS', 'AR', 'ICECC_VERSION', 'BUILD_CXXFLAGS', 'BUILD_NM', 'TARGET_CXXFLAGS', 'icc_version', 'MAKE', 'STRINGS', 'PSEUDO_DISABLED', 'base_bindir', 'BUILD_CC', 'PKG_CONFIG_SYSROOT_DIR', 'base_sbindir', 'systemd_unitdir', 'BUILD_CPPFLAGS', 'OBJDUMP', 'sysconfdir', 'base_libdir', 'TARGET_CFLAGS', 'CXX', 'includedir', 'servicedir', 'bindir', 'CCLD', 'PKG_CONFIG_DIR', 'BUILD_STRIP', 'libexecdir', 'OBJCOPY', 'ICECC_WHICH_AS', 'BUILD_CPP', 'BUILD_AS'])' changed items: set(['ARMPKGSFX_FPU']) I'll attach archive with all this sigdata files. Created attachment 2374 [details]
icecc-signatures.tar.bz2
I think this is actually usage/configuration error. In icecc.bbclass, there is:
BB_HASHBASE_WHITELIST = "ICECC_DISABLED"
amongst other things. bitbake.conf then does:
BB_HASHCONFIG_WHITELIST = "${BB_HASHBASE_WHITELIST}"
The net effect of this is the configuration instructs bitbake that the value of ICECC_DISABLED doesn't matter and hence it shouldn't reparse when it changes. So due to this, it will happily reuse the cached data when it should reparse.
BB_HASHCONFIG_WHITELIST_remove = "ICECC_DISABLED"
would probably "fix" the issue. Exactly what the correct fix is here, I'm less sure.
(In reply to comment #3) > I think this is actually usage/configuration error. In icecc.bbclass, there > is: > > BB_HASHBASE_WHITELIST = "ICECC_DISABLED" > > amongst other things. bitbake.conf then does: > > BB_HASHCONFIG_WHITELIST = "${BB_HASHBASE_WHITELIST}" Ah right, I forgot that BB_HASHBASE_WHITELIST is included in both :/. > The net effect of this is the configuration instructs bitbake that the value > of ICECC_DISABLED doesn't matter and hence it shouldn't reparse when it > changes. So due to this, it will happily reuse the cached data when it > should reparse. > > BB_HASHCONFIG_WHITELIST_remove = "ICECC_DISABLED" > > would probably "fix" the issue. Exactly what the correct fix is here, I'm > less sure. OK, I'll redo my tests with BB_HASHCONFIG_WHITELIST_remove. Maybe we don't need to fix it now, when you explained the root cause why the re-parse isn't happening and icecc.bbclass was fixed to produce the same signatures. Created attachment 2407 [details]
0001-icecc-Remove-ICECC_-variables-from-BB_HASHCONFIG_WHI.patch
I've tried it with oe-core and bitbake from today and the issue is completely resolved. I don't see what fixed bitbake re-parse, but now I get identical sigdata files (not only the sstate signature hash) even without adding BB_HASHCONFIG_WHITELIST_remove. OE qemuarm@ ~/build/oe-core $ md5sum icecc/*/sstate-diff-icecc-*/*/armv5*/zlib/*do_configure* | grep 2015 | sed 's#qemuarm/armv5te-oe-linux-gnueabi/zlib/1.2.8-r0.do_configure.sigdata.##g' c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-disabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-disabled/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-disabled2/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-enabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-enabled/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12-default/sstate-diff-icecc-enabled2/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12-remove/sstate-diff-icecc-disabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12-remove/sstate-diff-icecc-disabled/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12-remove/sstate-diff-icecc-enabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12-remove/sstate-diff-icecc-enabled/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12/sstate-diff-icecc-disabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c55f86d08f9ff15ced804ffec41bb968 icecc/master_oe-core-master-2015-02-12/sstate-diff-icecc-disabled/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12/sstate-diff-icecc-enabled-clean-cache/bcd980f69b596779260af9c62b2e0ec2 c98d36314edfc6f68fba29d5b94c9abc icecc/master_oe-core-master-2015-02-12/sstate-diff-icecc-enabled/bcd980f69b596779260af9c62b2e0ec2 Only difference between enabled and disabled sets is set_icecc_env dependency on ARMPKGSFX_FPU. *-remove directory is with BB_HASHCONFIG_WHITELIST_remove added in icecc (0001-icecc-Remove-ICECC_-variables-from-BB_HASHCONFIG_WHI.patch) Works OK with latest oe-core+bitbake. |