| Summary: | Investigate sstate package compression improvements | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Paul Eggleton <bluelightning> |
| Component: | core | Assignee: | Unassigned <unassigned> |
| Status: | RESOLVED OBSOLETE | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium | CC: | meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie |
| Version: | 1.7 | ||
| Target Milestone: | Future | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
Paul Eggleton
2014-02-20 12:01:36 UTC
An old, old patch (that was not merged) that relates to this: http://lists.openembedded.org/pipermail/openembedded-core/2011-December/052656.html We did try xz but the issue is the build slow down in generating these. There is perhaps the idea of a server side process which converted .tgz -> tar.xz. This would mean the code checking sstate servers for multiple different compression types. Or we could have the code just decompress automatically and use a standard extension. Regardless, more thought needed... We switched to zstd. Already changed in October 2021: sstate: Switch to ZStandard compressor support This patch switches the compressor from Gzip to ZStandard for ssate cache files. Zstandard compression provides a significant improvement in decompression speed as well as improvement in compression speed and disk usage over the 'tgz' format in use. Furthermore, its configurable compression level offers a trade-off between time spent compressing sstate cache files and disk space used by those files. The reduced disk usage also contributes to saving network traffic for those sharing their sstate cache with others. Zstandard should therefore be a good choice when: * disk space is at a premium * network speed / resources are limited * the CI server can sstate packages can be created at high compression * less CPU on the build server should be used for sstate decompression Signed-off-by: Henry Kleynhans <hkleynhans@fb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> https://git.openembedded.org/openembedded-core/commit/?id=0710e98f40ec7394567ede1f88f7c7bae306b925 |