Due to the way swupd works by processing chroot-like bundle directories on each os release we potentially end up carrying a lot of duplicate files in DEPLOY_DIR_SWUPD. We should look at using the hardlink program (or possibly fdupes) to replace duplicate files in that directory with hard links in order to save disk space.
Could this be a requirement for swupd? Anyway it seems like an optional step. If one makes lots of throwaway builds, it might be faster to jsut bear with the duplicated files. I think it would be ok to have it enabled by default only if the overhead of identifying duplicates is minimum (could the hash from the manifests of previous releases help?)
(In reply to comment #1) > Could this be a requirement for swupd? At least partially, yes. In fact swupd-server introduced a feature to use hardlinks when possible during swupd_make_pack https://github.com/clearlinux/swupd-server/releases/tag/v3.1.2 I haven't had any chance to investigate it yet, nor try to determine whether it needs to be enabled or whether it requires extra utilities on the host (i.e. a hardlink-native). > Anyway it seems like an optional step. > If one makes lots of throwaway builds, it might be faster to jsut bear with > the duplicated files. > > I think it would be ok to have it enabled by default only if the overhead of > identifying duplicates is minimum (could the hash from the manifests of > previous releases help?) This is useful feedback, thanks. ISTR the request for hardlink support came from a colleague on the Ostro team? If the demand isn't still there I shall push this out to a later milestone.
The right approach for avoiding duplicate files in the www directory is to create updates from the last build: swupd_create_update then automatically reuses existing, unmodified files. As an added bonus, that also reduces time spent in swupd_make_fullfiles because only new or modified files need to be handled. I'm currently working on some code which does that. Note that this is different from computing deltas.
Let's pull this in. Concern about disk space in the update repo was also mentioned by other parties interested in using meta-swupd.
Current meta-swupd master (today's commit 07ad71b78) supports incremental builds, so no files get duplicated anymore.