| Summary: | siginfo files in sstate using the wrong hash (probably from hashequiv) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Frans Meulenbroeks <fransmeulenbroeks> | ||||
| Component: | oe-core other | Assignee: | Unassigned <unassigned> | ||||
| Status: | ACCEPTED --- | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | Medium | CC: | randy.macleod, ross.burton, yoann.congal | ||||
| Version: | unspecified | ||||||
| Target Milestone: | Future | ||||||
| Hardware: | x86 | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | |||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||
| Attachments: |
|
||||||
|
Description
Frans Meulenbroeks
2021-08-17 07:07:38 UTC
*** Bug 14514 has been marked as a duplicate of this bug. *** I wrote a little script to look for files in the wrong directory and ran it on my local sstate. It found 12 files (out of 128GB of sstate). For example: 7a/9b/sstate:gcc::11.1.0:r0::3:6051b86bc3b9fc3d6e045f6d5a007ff13eff6f8f87359b2488a9e31b4fb807d5_fetch.tgz.siginfo They were all .siginfo files. Borrowing Ross' script I see many in my local sstate, also all siginfo files for non-sstate tasks. This pinpoints the code responsible, likely when hash equiv is active. richard@jet:/media/build1/poky/build/tmp/work/core2-64-poky-linux/lttng-tools$ ~/checksstate /media/sstate/master/ /media/sstate/master/72/1d/sstate:libgcrypt-native::1.9.3:r0::3:9d238d74043afc35b62e463fd90f2c90ad23876a1b86928f4ab3bb1f04052d93_fetch.tgz.siginfo /media/sstate/master/1e/8e/sstate:gettext-native::0.21:r0::3:a35650e3df48dc376b32cfa8e456e2ef3fdc365a224eef0f119cdbc78342e088_fetch.tgz.siginfo /media/sstate/master/1e/d7/sstate:libffi-native::3.4.2:r0::3:6bdb8672668ab592077855f1a0c41b02695c822140e88e7cb1189e3bf950b1a9_patch.tgz.siginfo /media/sstate/master/68/13/sstate:libunistring-native::0.9.10:r0::3:57041c2202204dc2eaf0294401057f618ff780f1147391475e79390be7f0f532_unpack.tgz.siginfo /media/sstate/master/68/83/sstate:texinfo-dummy-native::1.0:r0::3:228a8c69eacbcb4ec92309405b53de69ebd7b5393dcdd765168cce86f12d46e2_patch.tgz.siginfo /media/sstate/master/68/f2/sstate:libffi-native::3.4.2:r0::3:9a8318c48c7b54057acdf80c706422d7f50356d63d98b44c187216d48c57ec24_unpack.tgz.siginfo /media/sstate/master/f3/cf/sstate:pkgconfig-native::0.29.2+gitAUTOINC+edf8e6f0ea:r0::3:7f71df1daef53249ddabb0decb3053e293c079dcb4f163476ac4d895362e0683_unpack.tgz.siginfo /media/sstate/master/f3/50/sstate:opkg-utils-native::0.4.5:r0::3:a417d2029c4155c405afed5ee341e8cfdbde479530f55b656d865653f6753dd1_patch.tgz.siginfo /media/sstate/master/b6/45/sstate:texinfo-dummy-native::1.0:r0::3:19b0a546f6541598179be821e9899ad1b0dba881ebc4de416ff393eec7e97452_unpack.tgz.siginfo /media/sstate/master/67/6a/sstate:texinfo-dummy-native::1.0:r0::3:610ed4b8e8bf78bbcd4a667b6645a0276f5c8bfce5de4822923850d44d032bbe_fetch.tgz.siginfo /media/sstate/master/e6/7e/sstate:gperf-native::3.1:r0::3:22c33ca53b7d6ed96f6d00c58fbe29f73e961d35d7ff6bdbe57b5e008c3567b7_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libarchive-native::3.5.1:r0::3:191bef23e54e2c3d07a23ff1e988f05f9daad498762932f3a3b81a68da089e63_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:file-native::5.40:r0::3:e0b9d4944fcae0a44e777eeb7f04f472d0eb8783ea6b0cae99f1fbea13c2ef98_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:sqlite3-native::3.36.0:r0::3:ddd91f354e98169f487620d979b6fcd636b7bbabb6af7686ed39b1e78f1e1b5e_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:xz-native::5.2.5:r0::3:89ce6e27eed4e28b66c51636c95fb45e1d60bc27ba9bfc204b22be9589b44b45_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libidn2-native::2.3.1:r0::3:d9f61bc972c67488505fdd06581d81a0bec2a52ec2ce081d74e152cf93b2c8bd_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libmpc-native::1.2.1:r0::3:51213908c4adcc6a7ccdb3f84331f0cad3b5966ad87fb93e802b55720dbc6af6_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libpcre2-native::10.37:r0::3:69487653cc795c853593b0e3915d74bbbfcd68f831fd102a5007c6e6372ad839_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libnsl2-native::1.3.0:r0::3:a6b80843999761ac1433fc6e171b2e3e23966e80bffde05ca18358902789a65c_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libmicrohttpd-native::0.9.73:r0::3:efecc50687766a20fbf72f1f28462d0293040859acb32b52c672fdadc42f7f31_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:popt-native::1.18:r0::3:204ad4a09f5188ca73df954091ac2d49475e902a8b02cf60b5f1420ed4cd0683_patch.tgz.siginfo /media/sstate/master/e6/7e/sstate:libtirpc-native::1.3.2:r0::3:96dd13d8c0593608cc3c8b29d6178a71c751f0479cd3ae69561e22ec22d52efe_patch.tgz.siginfo /media/sstate/master/e6/6d/sstate:acl-native::2.3.1:r0::3:05771a46cec44b1af2812c5b6fa085b403ff30c495497a24e51a22c4f195c9ac_fetch.tgz.siginfo /media/sstate/master/10/87/sstate:perlcross-native::1.3.6:r0::3:639d7845b5b2f653b5c3f6a6683b2dd1567f986382536350505af8e3054cf3fa_patch.tgz.siginfo /media/sstate/master/10/9a/sstate:flex-native::2.6.4:r0::3:471670d41e6c62c12905af367109f717c7668701de7e93598b661ebb933473a5_patch.tgz.siginfo /media/sstate/master/21/61/sstate:lzo-native::2.10:r0::3:c8297d7f651884a03ad4fb7684a8aab4f7995fde462fe20967c3cf1fbd3ed4c9_fetch.tgz.siginfo /media/sstate/master/5a/49/sstate:perl-native::5.34.0:r0::3:fdd0d27d59174bb7447f1f811ac33288303db2a101eaf66af809afa2045ceba8_fetch.tgz.siginfo /media/sstate/master/63/d7/sstate:patch-native::2.7.6:r0::3:83ce20d39285eb7620a327de1f1e6c79341c28babf66bdfe4fff81e76b94c173_patch.tgz.siginfo /media/sstate/master/aa/b2/sstate:gmp-native::6.2.1:r0::3:38b1a130c4b93e47a90f76ccd7b1a7772bb17156dc810c63796e22fe1b3d43ff_fetch.tgz.siginfo /media/sstate/master/bb/18/sstate:pkgconfig-native::0.29.2+gitAUTOINC+edf8e6f0ea:r0::3:ec301f059f1df92f897e8347fc133ffbaf6e3019c88ae36e0e1f4d9fbd48b489_patch.tgz.siginfo /media/sstate/master/32/62/sstate:libgpg-error-native::1.42:r0::3:1e7bb7e91e3ce4265d419c7a0866dc7c3d9e7ad7486d8764ac5b026c92f06f08_unpack.tgz.siginfo /media/sstate/master/32/5c/sstate:libtool-native::2.4.6:r0::3:750961b43eac850b9a1bb1089b13dc6c7c85ec244f5def47155ded0fc02808a5_fetch.tgz.siginfo I'd also note these files are not used by the build itself, only for debugging so this is why we've not seen other issues due to this. (for info I have 191 of these in my local sstate) Having stared at the code long and hard, the only way this can happen is this code in siggen.py:
bb.error("Taskhash mismatch %s versus %s for %s" % (computed_taskhash, self.taskhash[tid], tid))
sigfile = sigfile.replace(self.taskhash[tid], computed_taskhash)
since everywhere else, the function at the top of sstate.bbclass us used to generate both the directory prefix and the filename which are done with the same hash.
As such I'm not sure these files are worth worrying about too much, they're debug only and the build generating them would show errors.
May still exist. Comment if this is important to you. - YP bug review. Probably worth re-running the check. Quick & dirty one-liner: cd .../sstate-cache; fdfind -t f --exclude universal/ |grep -v '\(..\)/\(..\)/.*:\1\2' It catches my test file (a550 -> a55f): a5/50/sstate:gdbm:x86-64-v3-poky-linux:1.26:r0:x86-64-v3:14:a55f1bd467e1e1bfcd7503ea889eba148adadabf0c03658ab46c5e026807a4e0_configure_ptest_base.tar.zst.siginfo_TEST Created attachment 5143 [details]
sstate-check.py
I lost the script I mentioned in the comments so I wrote it again. It needs py3 because life is too short to not use pathlib.
For my local sstate of two years work of work, it has quite a few files which are in the wrong path.
|