| Summary: | sstate-cache-management.sh doesn't work with new sstate-cache layout | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Martin Jansa <Martin.Jansa> | ||||
| Component: | devtools / tool chain | Assignee: | Robert Yang <liezhi.yang> | ||||
| Status: | VERIFIED FIXED | QA Contact: | |||||
| Severity: | major | ||||||
| Priority: | Medium+ | CC: | jessica.zhang, liang.li2, meta.mr.watcher, meta.watcher, richard.purdie, sgw, song.liu, zhenfeng.zhao | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 1.3 M4 | ||||||
| Hardware: | x86 | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | (Scheduled) | ||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | --- | |||||
| Attachments: |
|
||||||
|
Description
Martin Jansa
2012-08-06 06:56:34 UTC
Thanks for the reporting, I'm trying to figure out a better way to clean the old sstate cache recently. Merged in to master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f8d5e2b6c84f2a9d48daee7101792e3e154f01af From commit fcb61aa552a641faafdf2216fa57ba5ac6b9991f: Note: It doesn't handle the old sstate-cache/sstate-xxx.tgz (or siginfo), the user should remove it manually (rm -fr sstate-cache/sstate-* would be OK). This does not work at all on host which uses sstate-cache from SSTATE_MIRROR over http://. This fix does remove symlinks in new layout, but big files sstate-cache/sstate-* are still there so it does not save much space. And suggested removal of all sstate-cache/sstate-* also cannot be used on such host, because it would break all symlinks from prefixed dirs (including those which are still valid and needed). (In reply to comment #3) > From commit fcb61aa552a641faafdf2216fa57ba5ac6b9991f: > Note: It doesn't handle the old sstate-cache/sstate-xxx.tgz (or > siginfo), the user should remove it manually (rm -fr > sstate-cache/sstate-* would be OK). > > This does not work at all on host which uses sstate-cache from SSTATE_MIRROR > over http://. > > This fix does remove symlinks in new layout, but big files > sstate-cache/sstate-* are still there so it does not save much space. > Hi Martin, remove the files from the mirror seems not a good idea, I think that run the script on the mirror host would be better. > And suggested removal of all sstate-cache/sstate-* also cannot be used on > such host, because it would break all symlinks from prefixed dirs (including I will update the script a little to let it can clean the obsolete sstate files, but it should be run on the mirror host. Night. > those which are still valid and needed). (In reply to comment #4) > (In reply to comment #3) > > From commit fcb61aa552a641faafdf2216fa57ba5ac6b9991f: > > Note: It doesn't handle the old sstate-cache/sstate-xxx.tgz (or > > siginfo), the user should remove it manually (rm -fr > > sstate-cache/sstate-* would be OK). > > > > This does not work at all on host which uses sstate-cache from SSTATE_MIRROR > > over http://. > > > > This fix does remove symlinks in new layout, but big files > > sstate-cache/sstate-* are still there so it does not save much space. > > > > Hi Martin, remove the files from the mirror seems not a good idea, I think > that run the script on the mirror host would be better. Hi Robert, I'm not talking about http mirror, I'm talking about host which is using sstate-cache provided by his distro (so that he doesn't have to build everything). local.conf: SSTATE_MIRRORS ?= "file://.* http://best.distro.org/sstate-cache/PATH" > > And suggested removal of all sstate-cache/sstate-* also cannot be used on > > such host, because it would break all symlinks from prefixed dirs (including > > I will update the script a little to let it can clean the obsolete sstate > files, but it should be run on the mirror host. See this thread for other issues with sstate-cache dir which can have at least work around in this management script: http://lists.linuxtogo.org/pipermail/openembedded-core/2012-August/027125.html (In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > From commit fcb61aa552a641faafdf2216fa57ba5ac6b9991f: > > > Note: It doesn't handle the old sstate-cache/sstate-xxx.tgz (or > > > siginfo), the user should remove it manually (rm -fr > > > sstate-cache/sstate-* would be OK). > > > > > > This does not work at all on host which uses sstate-cache from SSTATE_MIRROR > > > over http://. > > > > > > This fix does remove symlinks in new layout, but big files > > > sstate-cache/sstate-* are still there so it does not save much space. > > > > > > > Hi Martin, remove the files from the mirror seems not a good idea, I think > > that run the script on the mirror host would be better. > > Hi Robert, I'm not talking about http mirror, I'm talking about host which > is using sstate-cache provided by his distro (so that he doesn't have to > build everything). > > local.conf: > SSTATE_MIRRORS ?= "file://.* http://best.distro.org/sstate-cache/PATH" > OK, I will add an option "-L follow links" (default no) to remove the files from the mirror. > > > And suggested removal of all sstate-cache/sstate-* also cannot be used on > > > such host, because it would break all symlinks from prefixed dirs (including > > > > I will update the script a little to let it can clean the obsolete sstate > > files, but it should be run on the mirror host. > > See this thread for other issues with sstate-cache dir which can have at > least work around in this management script: > > http://lists.linuxtogo.org/pipermail/openembedded-core/2012-August/027125. > html * Updated for the SSTATE_MIRRORS. * Add a "-L, --follow-symlink" option. Merged into master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=6163148c1798586f9b0543fc0f3b149fa15d1fc8 Created attachment 740 [details]
sstate-cache-management.log.gz
This new version seems to break -d option. When all files are downloaded from SSTATE_MIRROR then it just removes everything. Log attached. (In reply to comment #9) > This new version seems to break -d option. When all files are downloaded > from SSTATE_MIRROR then it just removes everything. Log attached. Hi Martin, As the log shows, the sstate-cache/sstate-* is older than the sstate-cache/??/sstate-*, so it will be removed, for example: Aug 28 14:44 sstate-cache/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz Aug 28 15:26 sstate-cache/18/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz "15:26" is newer than "14:44". I have changed the "access time" to "change time" because of the symlink as I had described in the commit message. (In reply to comment #10) > (In reply to comment #9) > > This new version seems to break -d option. When all files are downloaded > > from SSTATE_MIRROR then it just removes everything. Log attached. > > Hi Martin, > > As the log shows, the sstate-cache/sstate-* is older than the > sstate-cache/??/sstate-*, so it will be removed, for example: > > Aug 28 14:44 > sstate-cache/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2- > 18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > Aug 28 15:26 > sstate-cache/18/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64- > 2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > "15:26" is newer than "14:44". > > I have changed the "access time" to "change time" because of the symlink as > I had described in the commit message. Hi, so it will keep symlink in sstate-cache/18 but remove file where that symlink points? Looks like that: 352 .done files: OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep done$ | wc -l 352 OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep -v done$ | wc -l 0 and 352 broken symlinks OE qemux86-64@ ~/oe-core $ find sstate-cache -type l | wc -l 352 OE qemux86-64@ ~/oe-core $ find sstate-cache -type l -xtype f | wc -l 0 Not sure if this is expected behavior.. and downloaded file will be probably always older then symlink created after download. (In reply to comment #11) > (In reply to comment #10) > > (In reply to comment #9) > > > This new version seems to break -d option. When all files are downloaded > > > from SSTATE_MIRROR then it just removes everything. Log attached. > > > > Hi Martin, > > > > As the log shows, the sstate-cache/sstate-* is older than the > > sstate-cache/??/sstate-*, so it will be removed, for example: > > > > Aug 28 14:44 > > sstate-cache/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2- > > 18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > > > Aug 28 15:26 > > sstate-cache/18/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64- > > 2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > "15:26" is newer than "14:44". > > > > I have changed the "access time" to "change time" because of the symlink as > > I had described in the commit message. > > Hi, > > so it will keep symlink in sstate-cache/18 but remove file where that > symlink points? > > Looks like that: > > 352 .done files: > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep done$ | wc -l > 352 > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep -v done$ | wc -l > 0 > > and 352 broken symlinks > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l | wc -l > 352 > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l -xtype f | wc -l > 0 > > Not sure if this is expected behavior.. and downloaded file will be probably > always older then symlink created after download. I wonder how did you download the sstate file, please? Did the bitbake download it automatically (how to set the SSTATE_MIRROR ?) or you manually copied them to the SSTATE_DIR ? (In reply to comment #12) > (In reply to comment #11) > > (In reply to comment #10) > > > (In reply to comment #9) > > > > This new version seems to break -d option. When all files are downloaded > > > > from SSTATE_MIRROR then it just removes everything. Log attached. > > > > > > Hi Martin, > > > > > > As the log shows, the sstate-cache/sstate-* is older than the > > > sstate-cache/??/sstate-*, so it will be removed, for example: > > > > > > Aug 28 14:44 > > > sstate-cache/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2- > > > 18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > > > > > > Aug 28 15:26 > > > sstate-cache/18/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64- > > > 2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > > > "15:26" is newer than "14:44". > > > > > > I have changed the "access time" to "change time" because of the symlink as > > > I had described in the commit message. > > > > Hi, > > > > so it will keep symlink in sstate-cache/18 but remove file where that > > symlink points? > > > > Looks like that: > > > > 352 .done files: > > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep done$ | wc -l > > 352 > > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep -v done$ | wc -l > > 0 > > > > and 352 broken symlinks > > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l | wc -l > > 352 > > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l -xtype f | wc -l > > 0 > > > > Not sure if this is expected behavior.. and downloaded file will be probably > > always older then symlink created after download. > > I wonder how did you download the sstate file, please? Did the bitbake > download it automatically (how to set the SSTATE_MIRROR ?) or you manually > copied them to the SSTATE_DIR ? bitbake fetcher downloads them automatically, see http://lists.linuxtogo.org/pipermail/openembedded-core/2012-August/027125 (In reply to comment #13) > (In reply to comment #12) > > (In reply to comment #11) > > > (In reply to comment #10) > > > > (In reply to comment #9) > > > > > This new version seems to break -d option. When all files are downloaded > > > > > from SSTATE_MIRROR then it just removes everything. Log attached. > > > > > > > > Hi Martin, > > > > > > > > As the log shows, the sstate-cache/sstate-* is older than the > > > > sstate-cache/??/sstate-*, so it will be removed, for example: > > > > > > > > Aug 28 14:44 > > > > sstate-cache/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64-2- > > > > 18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > > > > > > > > > Aug 28 15:26 > > > > sstate-cache/18/sstate-base-files-qemux86_64-oe-linux-3.0.14-r71-qemux86_64- > > > > 2-18d337fbde4aa5003e746b31ee7c2542_deploy-ipk.tgz > > > > > > > > "15:26" is newer than "14:44". > > > > > > > > I have changed the "access time" to "change time" because of the symlink as > > > > I had described in the commit message. > > > > > > Hi, > > > > > > so it will keep symlink in sstate-cache/18 but remove file where that > > > symlink points? > > > > > > Looks like that: > > > > > > 352 .done files: > > > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep done$ | wc -l > > > 352 > > > OE qemux86-64@ ~/oe-core $ find sstate-cache -xtype f | grep -v done$ | wc -l > > > 0 > > > > > > and 352 broken symlinks > > > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l | wc -l > > > 352 > > > OE qemux86-64@ ~/oe-core $ find sstate-cache -type l -xtype f | wc -l > > > 0 > > > > > > Not sure if this is expected behavior.. and downloaded file will be probably > > > always older then symlink created after download. > > > > I wonder how did you download the sstate file, please? Did the bitbake > > download it automatically (how to set the SSTATE_MIRROR ?) or you manually > > copied them to the SSTATE_DIR ? > > bitbake fetcher downloads them automatically, see > http://lists.linuxtogo.org/pipermail/openembedded-core/2012-August/027125 Thanks, I will talk with Richard whether we can put the downloaded file to the SSTATE_DIR/<subdirs>/ rather than put it to SSTATE_DIR/ directly and make a symlink to it. I'm sorry about the inconvenience, the use case is more duplicated than I thought. Maybe there is some problem even when --state-dir param is used instead of -d Look how many files it wants to remove: jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ openembedded-core/scripts/sstate-cache-management.sh -v -L --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache Figuring out all the md5sums in stamps dir ... Done Figuring out the files which will be removed ... Done 15504 files will be removed! Do you want to continue (y/n)? y^C jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache -type f | wc -l 13162 jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache -xtype f | wc -l 16101 jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ ls tmp-eglibc/stamps/ all-oe-linux armv7a-vfp-neon-oe-linux-gnueabi nokia900-oe-linux-gnueabi om_gta02-oe-linux-gnueabi x86_64-linux armv4t-oe-linux-gnueabi nokia900-oe-linux om_gta02-oe-linux work-shared jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ du -hs sstate-cache/ 39G sstate-cache/ And this sstate-cache and stamps dir is now fully populated (last build just finished) so it should keep at least 26G in sstate-cache. I cannot test it right now, because that server does not have enough space to backup sstate-cache before running management script.. Also the numbers doesn't match anymore (IIRC last number was sum of previous before). openembedded-core/scripts/sstate-cache-management.sh -v -d -L --cache-dir=sstate-cache | tee -a log ... OE om-gta02@shr ~/shr-core $ grep '\(Figuring\|removed\)' log Figuring out the archs in the layers ... Done Figuring out the archs in the sstate cache dir ... Done Figuring out the sstate-xxx_deploy-rpm.tgz ... (0 files will be removed) Figuring out the sstate-xxx_deploy-ipk.tgz ... Analyzing sstate-abiword-xxx_deploy-ipk.tgz (1166 files will be removed) Figuring out the sstate-xxx_deploy-deb.tgz ... (0 files will be removed) Figuring out the sstate-xxx_deploy.tgz ... Analyzing sstate-dfu-util-native-xxx_deploy.tgz (0 files will be removed) Figuring out the sstate-xxx_package.tgz ... Analyzing sstate-abiword-xxx_package.tgz (1136 files will be removed) Figuring out the sstate-xxx_populate-lic.tgz ... Analyzing sstate-abiword-xxx_populate-lic.tgz (1134 files will be removed) Figuring out the sstate-xxx_populate-sysroot.tgz ... Analyzing sstate-abiword-xxx_populate-sysroot.tgz (1158 files will be removed) 4594 files will be removed! Do you want to continue (y/n)? Nothing to do > Also the numbers doesn't match anymore (IIRC last number was sum of previous before).
Please ignore last comment, they do match and I need better glasses :)
(In reply to comment #15) > Maybe there is some problem even when --state-dir param is used instead of -d > > Look how many files it wants to remove: > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ > openembedded-core/scripts/sstate-cache-management.sh -v -L > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > Figuring out all the md5sums in stamps dir ... Done > Figuring out the files which will be removed ... Done > 15504 files will be removed! Do you want to continue (y/n)? y^C > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache -type > f | wc -l > 13162 > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache > -xtype f | wc -l > 16101 > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ ls tmp-eglibc/stamps/ > all-oe-linux armv7a-vfp-neon-oe-linux-gnueabi > nokia900-oe-linux-gnueabi om_gta02-oe-linux-gnueabi x86_64-linux > armv4t-oe-linux-gnueabi nokia900-oe-linux om_gta02-oe-linux > work-shared > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ du -hs sstate-cache/ > 39G sstate-cache/ > > And this sstate-cache and stamps dir is now fully populated (last build just > finished) so it should keep at least 26G in sstate-cache. I cannot test it > right now, because that server does not have enough space to backup > sstate-cache before running management script.. Hi Martin, I've done a fresh building (bitbake core-image-sato), also the sstate-cache dir is newly created: $ ../poky/scripts/sstate-cache-management.sh --cache-dir=sstate-cache/ --stamps-dir=tmp/stamps/ Figuring out all the md5sums in stamps dir ... Done Figuring out the files which will be removed ... Done No files to remove $ find sstate-cache/ -type f | wc -l 2904 The 15504 contains the symlink and its destination, if we count twice of the symlink, then there should be: 13162 + (16101 - 13162) * 2 = 19040 after the remove: 19040 - 15504 = 3536 This number is not accurate since we assume that all the symlinks have been removed, but it is close. $ du -sh sstate-cache 2.6G sstate-cache/ (In reply to comment #18) > (In reply to comment #15) > > Maybe there is some problem even when --state-dir param is used instead of -d > > > > Look how many files it wants to remove: > > > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ > > openembedded-core/scripts/sstate-cache-management.sh -v -L > > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > Figuring out all the md5sums in stamps dir ... Done > > Figuring out the files which will be removed ... Done > > 15504 files will be removed! Do you want to continue (y/n)? y^C > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache -type > > f | wc -l > > 13162 > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find sstate-cache > > -xtype f | wc -l > > 16101 > > > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ ls tmp-eglibc/stamps/ > > all-oe-linux armv7a-vfp-neon-oe-linux-gnueabi > > nokia900-oe-linux-gnueabi om_gta02-oe-linux-gnueabi x86_64-linux > > armv4t-oe-linux-gnueabi nokia900-oe-linux om_gta02-oe-linux > > work-shared > > jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ du -hs sstate-cache/ > > 39G sstate-cache/ > > > > And this sstate-cache and stamps dir is now fully populated (last build just > > finished) so it should keep at least 26G in sstate-cache. I cannot test it > > right now, because that server does not have enough space to backup > > sstate-cache before running management script.. > > Hi Martin, > > I've done a fresh building (bitbake core-image-sato), also the sstate-cache > dir is newly created: > > $ ../poky/scripts/sstate-cache-management.sh --cache-dir=sstate-cache/ > --stamps-dir=tmp/stamps/ > Figuring out all the md5sums in stamps dir ... Done > Figuring out the files which will be removed ... Done > No files to remove > > $ find sstate-cache/ -type f | wc -l > 2904 > > The 15504 contains the symlink and its destination, if we count twice of the > symlink, then there should be: > > 13162 + (16101 - 13162) * 2 = 19040 > > after the remove: > > 19040 - 15504 = 3536 > > This number is not accurate since we assume that all the symlinks have been > removed, but it is close. > > $ du -sh sstate-cache > 2.6G sstate-cache/ Hi Martin, This is merged into master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=43a7c66113f557f762c10756e4cb2f29f0870e93 About the --stamps-dir option, I've tested it and checked the code again, it works for me and seems the code is right. I will mark it as fixed, I would appreciate it you let me know it doesn't work well. I just did rebuild from scratch with old sstate-cache (I don't think much was reused from that)
After that I had:
# du -hs sstate-cache
88G
$ openembedded-core/scripts/sstate-cache-management.sh -v -L --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache
$ du -hs sstate-cache
26G
# probably good
$ openembedded-core/scripts/sstate-cache-management.sh -v -L -d --cache-dir=sstate-cache
..
3136 files have been removed!
$ du -hs sstate-cache
15G
# not so sure if it's good
random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz:
$ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz
sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz
sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz
sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz
sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16-nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz
sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz
sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17-om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz
Do you have any idea why I still have stalled -r16- version and 4 different versions for nokia900 are kept?
Only sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink
$ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz -exec ls -lah {} \;
-rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz
lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz
-rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz
-rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16-nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz
-rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz
-rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17-om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz
(In reply to comment #20) > I just did rebuild from scratch with old sstate-cache (I don't think much > was reused from that) > > After that I had: > # du -hs sstate-cache > 88G > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > $ du -hs sstate-cache > 26G > # probably good > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L -d > --cache-dir=sstate-cache > .. > 3136 files have been removed! > $ du -hs sstate-cache > 15G > # not so sure if it's good > > random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz: > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > Do you have any idea why I still have stalled -r16- version and 4 different > versions for nokia900 are kept? > > Only > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > -exec ls -lah {} \; > -rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> > /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate- > xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2- > 3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > -rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > -rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz Hi Martin, I think that you need to use the --extra-layer option since the nokia900 or om_gta02 arch is not in the oe-core layer: --extra-layer=<layer1>,<layer2>...<layern> Specify the layer which will be used for searching the archs, it will search the meta and meta-* layers in the top dir by default, and will search meta, meta-*, <layer1>, <layer2>, ...<layern> when specified. Use "," as the separator. This is useless for --stamps-dir. (In reply to comment #21) > (In reply to comment #20) > > I just did rebuild from scratch with old sstate-cache (I don't think much > > was reused from that) > > > > After that I had: > > # du -hs sstate-cache > > 88G > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L > > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > > > $ du -hs sstate-cache > > 26G > > # probably good > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L -d > > --cache-dir=sstate-cache > > .. > > 3136 files have been removed! > > $ du -hs sstate-cache > > 15G > > # not so sure if it's good > > > > random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz: > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > Do you have any idea why I still have stalled -r16- version and 4 different > > versions for nokia900 are kept? > > > > Only > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > -exec ls -lah {} \; > > -rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> > > /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate- > > xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2- > > 3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > Hi Martin, > > I think that you need to use the --extra-layer option since the nokia900 or > om_gta02 arch is not in the oe-core layer: > > --extra-layer=<layer1>,<layer2>...<layern> > Specify the layer which will be used for searching the archs, > it will search the meta and meta-* layers in the top dir by > default, and will search meta, meta-*, <layer1>, <layer2>, > ...<layern> when specified. Use "," as the separator. > > This is useless for --stamps-dir. Hi Robert, nokia900 and om-gta02 machine configs are in meta-smartphone repo, so I would expect them to be included by default (as option description suggests). But you're right, they are not, even when I use --extra-layer=meta-smartphone/meta-nokia/,meta-smartphone/meta-openmoko. Am I doing something wrong? Figuring out the archs in the sstate cache dir ... Done The following archs have been found in the cache dir: all armv4t armv7a x86_64 (In reply to comment #22) > (In reply to comment #21) > > (In reply to comment #20) > > > I just did rebuild from scratch with old sstate-cache (I don't think much > > > was reused from that) > > > > > > After that I had: > > > # du -hs sstate-cache > > > 88G > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L > > > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > > > > > $ du -hs sstate-cache > > > 26G > > > # probably good > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L -d > > > --cache-dir=sstate-cache > > > .. > > > 3136 files have been removed! > > > $ du -hs sstate-cache > > > 15G > > > # not so sure if it's good > > > > > > random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz: > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > > > Do you have any idea why I still have stalled -r16- version and 4 different > > > versions for nokia900 are kept? > > > > > > Only > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink > > > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > -exec ls -lah {} \; > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> > > > /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate- > > > xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2- > > > 3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > Hi Martin, > > > > I think that you need to use the --extra-layer option since the nokia900 or > > om_gta02 arch is not in the oe-core layer: > > > > --extra-layer=<layer1>,<layer2>...<layern> > > Specify the layer which will be used for searching the archs, > > it will search the meta and meta-* layers in the top dir by > > default, and will search meta, meta-*, <layer1>, <layer2>, > > ...<layern> when specified. Use "," as the separator. > > > > This is useless for --stamps-dir. > > Hi Robert, > > nokia900 and om-gta02 machine configs are in meta-smartphone repo, so I > would expect them to be included by default (as option description > suggests). But you're right, they are not, even when I use > --extra-layer=meta-smartphone/meta-nokia/,meta-smartphone/meta-openmoko. Am > I doing something wrong? > > Figuring out the archs in the sstate cache dir ... Done > The following archs have been found in the cache dir: > all armv4t armv7a x86_64 I'm cloning the repo now, but seems a big repo, and my bandwidth is not very good, will see it tomorrow. // Robert (In reply to comment #23) > (In reply to comment #22) > > (In reply to comment #21) > > > (In reply to comment #20) > > > > I just did rebuild from scratch with old sstate-cache (I don't think much > > > > was reused from that) > > > > > > > > After that I had: > > > > # du -hs sstate-cache > > > > 88G > > > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L > > > > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > > > > > > > $ du -hs sstate-cache > > > > 26G > > > > # probably good > > > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L -d > > > > --cache-dir=sstate-cache > > > > .. > > > > 3136 files have been removed! > > > > $ du -hs sstate-cache > > > > 15G > > > > # not so sure if it's good > > > > > > > > random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz: > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > > > > > Do you have any idea why I still have stalled -r16- version and 4 different > > > > versions for nokia900 are kept? > > > > > > > > Only > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink > > > > > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > > -exec ls -lah {} \; > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 > > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> > > > > /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate- > > > > xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2- > > > > 3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 > > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 > > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 > > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 > > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > > > Hi Martin, > > > > > > I think that you need to use the --extra-layer option since the nokia900 or > > > om_gta02 arch is not in the oe-core layer: > > > > > > --extra-layer=<layer1>,<layer2>...<layern> > > > Specify the layer which will be used for searching the archs, > > > it will search the meta and meta-* layers in the top dir by > > > default, and will search meta, meta-*, <layer1>, <layer2>, > > > ...<layern> when specified. Use "," as the separator. > > > > > > This is useless for --stamps-dir. > > > > Hi Robert, > > > > nokia900 and om-gta02 machine configs are in meta-smartphone repo, so I > > would expect them to be included by default (as option description > > suggests). But you're right, they are not, even when I use > > --extra-layer=meta-smartphone/meta-nokia/,meta-smartphone/meta-openmoko. Am > > I doing something wrong? > > > > Figuring out the archs in the sstate cache dir ... Done > > The following archs have been found in the cache dir: > > all armv4t armv7a x86_64 > > I'm cloning the repo now, but seems a big repo, and my bandwidth is not very > good, will see it tomorrow. > > // Robert Interesting thing is that it wasn't removed in first pass with -stamps-dir used (where --extra-layer option is not usefull) and there is just one stamp used for om-gta02 and one for nokia900. jenkins@h1987041:~/jobs/shr-core/workspace/shr-core$ find tmp-eglibc/stamps/ -name xserver-xf86-config\* tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_compile.sigdata.3e330dbfa500b57e3a252a1cc1d5d440 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_build tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_fetch.sigdata.f3204e7055c81eedac7a1af386c3d397 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_setscene.nokia900 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write_ipk.sigdata.929eb681be4c76b3f11ebd5e41181490 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_unpack.sigdata.767a527459ad49249f41d890231a58a2 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_sysroot_setscene.nokia900 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_sysroot.sigdata.75b7ef016eec496bee580fd8909f5429 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_patch.sigdata.d9a9310e377bc28c09f17a5fecc80b39 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_configure.sigdata.c3c549964dab530bbb8b42bd6cb674e4 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_rm_work tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package.sigdata.47be09abd262d4a1b4d01c4179fd34a7 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_lic_setscene tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_rm_work.sigdata.e535dd99947197cbff0e429a6f8cf639 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_install.sigdata.9819379dd14f9db08664c25414de3251 tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write tmp-eglibc/stamps/nokia900-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write_ipk tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_build tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_install.sigdata.4eea961b09164de9ea43dc788b2854f1 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_sysroot_setscene.om-gta02 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_compile.sigdata.9847a8983becf272e23460fbc5032448 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_setscene.om-gta02 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_rm_work.sigdata.4e5447868f888610482f625ec1b7a3f9 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package.sigdata.1829a0fdf8ac1383e55f011390de19fb tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_fetch.sigdata.207a3e43e79a89f110b6d567bba563b2 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_patch.sigdata.759dcf8ec68b5c1471e079b760c2d50c tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_sysroot.sigdata.3438f85898a76324326d679f6ebd0e40 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_rm_work tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_populate_lic_setscene tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write_ipk.sigdata.bdf22100b6af7e6b4176519fcbcd1160 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_unpack.sigdata.ff2f908101553a6ff9cdaf69a969b7a1 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_configure.sigdata.bf47788e032311d48b4c36b5d3630751 tmp-eglibc/stamps/om_gta02-oe-linux-gnueabi/xserver-xf86-config-0.1-r17.do_package_write_ipk (In reply to comment #24) > (In reply to comment #23) > > (In reply to comment #22) > > > (In reply to comment #21) > > > > (In reply to comment #20) > > > > > I just did rebuild from scratch with old sstate-cache (I don't think much > > > > > was reused from that) > > > > > > > > > > After that I had: > > > > > # du -hs sstate-cache > > > > > 88G > > > > > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L > > > > > --stamps-dir=tmp-eglibc/stamps/ --cache-dir=sstate-cache > > > > > > > > > > $ du -hs sstate-cache > > > > > 26G > > > > > # probably good > > > > > > > > > > $ openembedded-core/scripts/sstate-cache-management.sh -v -L -d > > > > > --cache-dir=sstate-cache > > > > > .. > > > > > 3136 files have been removed! > > > > > $ du -hs sstate-cache > > > > > 15G > > > > > # not so sure if it's good > > > > > > > > > > random pick sstate-xserver-xf86-config\*populate-sysroot\*tgz: > > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > > > > > > > Do you have any idea why I still have stalled -r16- version and 4 different > > > > > versions for nokia900 are kept? > > > > > > > > > > Only > > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz is symlink > > > > > > > > > > $ find sstate-cache/ -name sstate-xserver-xf86-config\*populate-sysroot\*tgz > > > > > -exec ls -lah {} \; > > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 26 12:10 > > > > > sstate-cache/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > > lrwxrwxrwx 1 jenkins nogroup 188 Aug 26 16:38 > > > > > sstate-cache/3f/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz -> > > > > > /var/lib/jenkins/jobs/shr-core/workspace/shr-core/sstate-cache/sstate- > > > > > xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17-nokia900-2- > > > > > 3f2aab7723487d6a264606eb212453a3_populate-sysroot.tgz > > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:50 > > > > > sstate-cache/bf/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-bf3b560f60db0aeb6cbfe23b0e4bcfbb_populate-sysroot.tgz > > > > > -rw-r--r-- 1 jenkins nogroup 45 Aug 24 11:46 > > > > > sstate-cache/61/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r16- > > > > > nokia900-2-61d34bef3c0c3a887b039293079b3d82_populate-sysroot.tgz > > > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 31 10:14 > > > > > sstate-cache/75/sstate-xserver-xf86-config-nokia900-oe-linux-gnueabi-0.1-r17- > > > > > nokia900-2-75b7ef016eec496bee580fd8909f5429_populate-sysroot.tgz > > > > > -rw-rw-r-- 1 jenkins nogroup 45 Aug 30 21:03 > > > > > sstate-cache/34/sstate-xserver-xf86-config-om_gta02-oe-linux-gnueabi-0.1-r17- > > > > > om_gta02-2-3438f85898a76324326d679f6ebd0e40_populate-sysroot.tgz > > > > > > > > Hi Martin, > > > > > > > > I think that you need to use the --extra-layer option since the nokia900 or > > > > om_gta02 arch is not in the oe-core layer: > > > > > > > > --extra-layer=<layer1>,<layer2>...<layern> > > > > Specify the layer which will be used for searching the archs, > > > > it will search the meta and meta-* layers in the top dir by > > > > default, and will search meta, meta-*, <layer1>, <layer2>, > > > > ...<layern> when specified. Use "," as the separator. > > > > > > > > This is useless for --stamps-dir. > > > > > > Hi Robert, > > > > > > nokia900 and om-gta02 machine configs are in meta-smartphone repo, so I > > > would expect them to be included by default (as option description > > > suggests). But you're right, they are not, even when I use > > > --extra-layer=meta-smartphone/meta-nokia/,meta-smartphone/meta-openmoko. Am > > > I doing something wrong? > > > > > > Figuring out the archs in the sstate cache dir ... Done > > > The following archs have been found in the cache dir: > > > all armv4t armv7a x86_64 > > > > I'm cloning the repo now, but seems a big repo, and my bandwidth is not very > > good, will see it tomorrow. > > > > // Robert > > Interesting thing is that it wasn't removed in first pass with -stamps-dir > used (where --extra-layer option is not usefull) and there is just one stamp > used for om-gta02 and one for nokia900. And looking at code I don't think --extra-layer option can add MACHINE to all_archs, it's looking only for more AVAILTUNES in conf/machine/include, but not */meta*/conf/machine/foo.conf, so list of MACHINEs is hardcoded in all_archs. echo -n "Figuring out the archs in the layers ... " topdir=$(dirname $(dirname $(readlink -e $0))) tunedirs="`find $topdir/meta* $layers -path '*/meta*/conf/machine/include'`" [ -n "$tunedirs" ] || echo_error "Can't find the tune directory" all_archs=`grep -r -h "^AVAILTUNES " $tunedirs | sed -e 's/.*=//' -e 's/\"//g'` # Add the qemu and native archs # Use the "_" to substitute "-", e.g., x86-64 to x86_64 # Sort to remove the duplicated ones all_archs=$(echo $all_archs qemuarm qemux86 qemumips qemuppc qemux86_64 $(uname -m) \ | sed -e 's/-/_/g' -e 's/ /\n/g' | sort -u) echo "Done" That explains why I was using my own copy of sstate-cache-management.sh with all MACHINEs I'm building added. Fixes for some issues reported when debugging this, sent to oe-core ML. (In reply to comment #26) > Fixes for some issues reported when debugging this, sent to oe-core ML. Thank you very much! (In reply to comment #27) > (In reply to comment #26) > > Fixes for some issues reported when debugging this, sent to oe-core ML. > > Thank you very much! You're welcome but -d is still broken. I've tried to rebuild from sstate-cache twice, once after cleaning with sstate-cache-management.sh and once without. Without it's good, almost everything was reused: tmp-eglibc/cooker$ grep -R "do_compile: Started$" . ./20120903084347.log:NOTE: recipe shr-version-1.020120903-r0: task do_compile: Started ./20120903084347.log:NOTE: recipe base-files-3.0.14-r79: task do_compile: Started And those depends on date, so it's not so big surprise they were rebuilt. But after cleanup it's much worse: $ grep -R "do_compile: Started$" . | wc -l 605 Including big things like qt4-x11-free etc.. If I try to execute sstate-cache-management.sh again with this sstate-cache which should be complete and all packages still needed (except maybe shr-version or base-files) I get 92 files will be removed! (mostly allarch.*do_deploy and some -native recipes) when --stamps-dir=tmp-eglibc/stamps/ is used and 4698 files will be removed! when -d is used This time I did not even use SSTATE_MIRROR and still it's not working properly :/. I guess you have to build at least 2 machines to reproduce this (right now I'm building qemux86-64 and qemux86-64copy with oe-core only to show you easier setup to reproduce it). I'll send patch to add --debug option and fix for this.. The problem is that you're looking for duplicates across all available archs, so armv4t and armv7a populate_sysroot are considered duplicate and only last one is kept, but to rebuild from sstate-cache you need both! -d option fixed by http://git.openembedded.org/openembedded-core/commit/?id=46309e96300c35f6e8a5f33512a6a38720c629d2 and removing some native recipes even with --stamps-dir explained and fixed in http://patchwork.openembedded.org/patch/35875/ verified git commit: 718eb85806e080d0b165344b272e531ef19421eb, branch :1.3_M5 |