| Summary: | sstate-cache-management.sh can't remove duplicated sstate files for qemux86-64 | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Yi Zhao <yi.zhao> |
| Component: | core | Assignee: | Martin Jansa <Martin.Jansa> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | corneliux.stoicescu, kai.kang, ke.zou, meta.mr.watcher, meta.watcher, sgw, yi.zhao |
| Version: | 1.6 | ||
| Target Milestone: | 1.6 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
| Bug Depends on: | |||
| Bug Blocks: | 5872 | ||
|
Description
Yi Zhao
2014-02-21 10:31:35 UTC
--remove-duplicated is removing duplicated sstate files (keeping only the newest one), when arch was changed then they are not duplicated and you need to use --stamps-dir param. Please reopen if I misunderstood what you're expecting from --remove-duplicated. (In reply to comment #1) > --remove-duplicated is removing duplicated sstate files (keeping only the > newest one), when arch was changed then they are not duplicated and you need > to use --stamps-dir param. The arch is not changed during the testing. This issue only happens on qemux86-64. Other archs (qemux86, qemuarm, qemumips, qemuppc) are working fine. Here are more detail logs: For qemux86 (which has expected result): #################### $ bitbake man $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/ba/sstate:man:i586-poky-linux:1.6g:r1:i586:3:bac0c835fe6d0a78497b0ea57696beba_package.tgz ./sstate-cache/ba/sstate:man:i586-poky-linux:1.6g:r1:i586:3:bac0c835fe6d0a78497b0ea57696beba_package.tgz.siginfo # Edit man_1.6g.bb and increase the PR $ bitbake man $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/f6/sstate:man:i586-poky-linux:1.6g:r2:i586:3:f675a29448559a7d45ca948fde15d087_package.tgz ./sstate-cache/f6/sstate:man:i586-poky-linux:1.6g:r2:i586:3:f675a29448559a7d45ca948fde15d087_package.tgz.siginfo ./sstate-cache/ba/sstate:man:i586-poky-linux:1.6g:r1:i586:3:bac0c835fe6d0a78497b0ea57696beba_package.tgz ./sstate-cache/ba/sstate:man:i586-poky-linux:1.6g:r1:i586:3:bac0c835fe6d0a78497b0ea57696beba_package.tgz.siginfo $ sstate-cache-management.sh --cache-dir=./sstate-cache --remove-duplicated -y Figuring out the archs in the layers ... Done Figuring out the suffixes in the sstate cache dir ... Done The following suffixes have been found in the cache dir: package package_write_rpm packagedata populate_lic populate_sysroot Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: i586 x86_64_i586 x86_64 Figuring out the sstate:xxx_package.tgz ... (2 from 75 files for package suffix will be removed) Figuring out the sstate:xxx_package_write_rpm.tgz ... (2 from 25 files for package_write_rpm suffix will be removed) Figuring out the sstate:xxx_packagedata.tgz ... (2 from 25 files for packagedata suffix will be removed) Figuring out the sstate:xxx_populate_lic.tgz ... (2 from 2 files for populate_lic suffix will be removed) Figuring out the sstate:xxx_populate_sysroot.tgz ... (2 from 66 files for populate_sysroot suffix will be removed) (0 files with old sstate-* filenames will be removed) 10 from 689 files will be removed! Removing sstate-old-filenames.tgz (0 files) ... Done Removing sstate:xxx_package.tgz (2 files) ... Done Removing sstate:xxx_package_write_rpm.tgz (2 files) ... Done Removing sstate:xxx_packagedata.tgz (2 files) ... Done Removing sstate:xxx_populate_lic.tgz (2 files) ... Done Removing sstate:xxx_populate_sysroot.tgz (2 files) ... Done 10 files have been removed! builder@pek-yocto-build1:/buildarea3/yzhao1/poky-build/build-test$ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/f6/sstate:man:i586-poky-linux:1.6g:r2:i586:3:f675a29448559a7d45ca948fde15d087_package.tgz ./sstate-cache/f6/sstate:man:i586-poky-linux:1.6g:r2:i586:3:f675a29448559a7d45ca948fde15d087_package.tgz.siginfo ##################### For qemux86-64 #################### $ bitbake man $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz.siginfo ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz # Edit man_1.6g.bb and increase the PR $ bitbake man $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz.siginfo ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz.siginfo ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz $ sstate-cache-management.sh --cache-dir=./sstate-cache --remove-duplicated -y Figuring out the archs in the layers ... Done Figuring out the suffixes in the sstate cache dir ... Done The following suffixes have been found in the cache dir: package package_write_rpm packagedata populate_lic populate_sysroot Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: x86_64 Figuring out the sstate:xxx_package.tgz ... (0 from 75 files for package suffix will be removed) Figuring out the sstate:xxx_package_write_rpm.tgz ... (0 from 25 files for package_write_rpm suffix will be removed) Figuring out the sstate:xxx_packagedata.tgz ... (0 from 25 files for packagedata suffix will be removed) Figuring out the sstate:xxx_populate_lic.tgz ... (2 from 2 files for populate_lic suffix will be removed) Figuring out the sstate:xxx_populate_sysroot.tgz ... (0 from 66 files for populate_sysroot suffix will be removed) (0 files with old sstate-* filenames will be removed) 2 from 689 files will be removed! Removing sstate-old-filenames.tgz (0 files) ... Done Removing sstate:xxx_populate_lic.tgz (2 files) ... Done 2 files have been removed! $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz.siginfo ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz.siginfo ./sstate-cache/ca/sstate:man:core2-64-poky-linux:1.6g:r1:core2-64:3:ca2401d40ed247c8e41d45f5b66e9e69_package.tgz ######################### You could see when run the sstate-cache-management.sh script, 10 files are removed for qemux86, but only 2 files are removed for qemux86-64 OK, sorry I was confused by your > The qemux86-64 arch changed from x86-64 to core2-64 by commit I thought it was during the testing. See http://git.openembedded.org/openembedded-core/commit/?id=a27cc54fb2d0e59f3a800893c1848cb26a7c5335 add --extra-archs=core2-64 parameter and let me know if it works correctly now. (In reply to comment #4) > OK, sorry I was confused by your > > > The qemux86-64 arch changed from x86-64 to core2-64 by commit > > I thought it was during the testing. > > See > http://git.openembedded.org/openembedded-core/commit/ > ?id=a27cc54fb2d0e59f3a800893c1848cb26a7c5335 > > add --extra-archs=core2-64 parameter and let me know if it works correctly > now. Yes, after add this parameter it works correctly. ############################## $ sstate-cache-management.sh --cache-dir=./sstate-cache --remove-duplicated --extra-archs=core2-64 -y Done Figuring out the suffixes in the sstate cache dir ... Done The following suffixes have been found in the cache dir: package package_write_rpm packagedata populate_lic populate_sysroot Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: x86_64 core2-64 x86_64_core2-64 Figuring out the sstate:xxx_package.tgz ... (2 from 75 files for package suffix will be removed) Figuring out the sstate:xxx_package_write_rpm.tgz ... (2 from 25 files for package_write_rpm suffix will be removed) Figuring out the sstate:xxx_packagedata.tgz ... (2 from 25 files for packagedata suffix will be removed) Figuring out the sstate:xxx_populate_lic.tgz ... (2 from 2 files for populate_lic suffix will be removed) Figuring out the sstate:xxx_populate_sysroot.tgz ... (2 from 66 files for populate_sysroot suffix will be removed) (0 files with old sstate-* filenames will be removed) 10 from 689 files will be removed! Removing sstate-old-filenames.tgz (0 files) ... Done Removing sstate:xxx_package.tgz (2 files) ... Done Removing sstate:xxx_package_write_rpm.tgz (2 files) ... Done Removing sstate:xxx_packagedata.tgz (2 files) ... Done Removing sstate:xxx_populate_lic.tgz (2 files) ... Done Removing sstate:xxx_populate_sysroot.tgz (2 files) ... Done 10 files have been removed! $ find ./sstate-cache/ -name "sstate:man:*package.tgz*" ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz ./sstate-cache/10/sstate:man:core2-64-poky-linux:1.6g:r2:core2-64:3:10b3fe1ffe814cb647d4160d9286eb2c_package.tgz.siginfo ####################### The qemux86-64 is one of the default supporting archs in yocto. Other default supporting archs (qemuarm, qemuppc, qemumips, qemux86) are no need to add this extra option. Is this a issue need to be fixed ? > Is this a issue need to be fixed ?
It's possible enhancement, but doing that from AVAILTUNES isn't correct and the list of possible TUNE_PKGARCHs is very long (which makes this script a lot slower), that's why I've decided to leave that to caller.
Hello Martin, the --extra-archs seems not to be working correctly, as I believe it's limiting the used archs instead of expanding the range. Please see the following output: " cornel@NODE-3:~/SSD/poky/build$ sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated --extra-archs=core2-32 Done Figuring out the suffixes in the sstate cache dir ... Done The following suffixes have been found in the cache dir: deploy package packagedata package_write_rpm populate_lic populate_sysroot Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: allarch x86_64 Figuring out the sstate:xxx_deploy.tgz ... (0 from 1 files for deploy suffix will be removed) Figuring out the sstate:xxx_package.tgz ... (0 from 276 files for package suffix will be removed) Figuring out the sstate:xxx_packagedata.tgz ... (0 from 92 files for packagedata suffix will be removed) Figuring out the sstate:xxx_package_write_rpm.tgz ... (0 from 92 files for package_write_rpm suffix will be removed) Figuring out the sstate:xxx_populate_lic.tgz ... (4 from 130 files for populate_lic suffix will be removed) Figuring out the sstate:xxx_populate_sysroot.tgz ... (0 from 177 files for populate_sysroot suffix will be removed) (0 files with old sstate-* filenames will be removed) 4 from 2581 files will be removed! Do you want to continue (y/n)? n Nothing to do cornel@NODE-3:~/SSD/poky/build$ sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated Figuring out the archs in the layers ... Done Figuring out the suffixes in the sstate cache dir ... Done The following suffixes have been found in the cache dir: deploy package packagedata package_write_rpm populate_lic populate_sysroot Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: allarch i586 x86_64_i586 qemux86 x86_64 Figuring out the sstate:xxx_deploy.tgz ... (0 from 1 files for deploy suffix will be removed) Figuring out the sstate:xxx_package.tgz ... (2 from 276 files for package suffix will be removed) Figuring out the sstate:xxx_packagedata.tgz ... (2 from 92 files for packagedata suffix will be removed) Figuring out the sstate:xxx_package_write_rpm.tgz ... (2 from 92 files for package_write_rpm suffix will be removed) Figuring out the sstate:xxx_populate_lic.tgz ... (4 from 130 files for populate_lic suffix will be removed) Figuring out the sstate:xxx_populate_sysroot.tgz ... (2 from 177 files for populate_sysroot suffix will be removed) (0 files with old sstate-* filenames will be removed) 12 from 2581 files will be removed! Do you want to continue (y/n)? n Nothing to do " Please note that when using --extra-archs=core2-32 we get less files for deletion. Also when using --extra-archs=core2-32 the script output does not mention detection of i586 packages: " The following archs have been found in the cache dir: allarch x86_64 " And there are no i586 packages removed. Leaving this as REOPENED. That behavior is consistent with what the option description says:
--extra-archs=<arch1>,<arch2>...<archn>
Specify list of architectures which should be tested, this list
will be extended with native arch, allarch and empty arch. The
script won't be trying to generate list of available archs from
AVAILTUNES in tune files.
This is working as intended. |