A swupd bundle provides enhancements to the core os image that add a new feature. The swupd integration in #9039 will handle generating the swupd bundle artefacts but we first need something to process. swupd-server expects that a bundle "source" will be a chroot-like directory containing any files needed to enable the feature that don't exist in the core os filesystem. Current understanding is that the most reliable way to provide this is to post-process images comparing a whole OS image to the core os image and core os + bundle images: 0. Sanity check various preconditions — i.e. that the OS VERSION has increased 1. Build a mega-image including the core image and all bundle contents → this ensures all appropriate package contents are in the sysroot (is this relevant? my initial thought was to ensure there's a fully populated /etc/passwd and /etc/groups in the base image. Those files are liable to change between builds unless you're using useradd- staticids anyway.) 2. Build a core image + bundle image for each defined bundle → a diff of a bundle image manifest to the core-image gives us the bundle's file contents 3. Build the core image → the main image output of the process and the base to compare the other images against 4. Process the various manifests to determine the contents of the individual bundles and copy the bundle contents to versioned, per- bundle directories for processing by swupd 5. Have swupd process the base image and each bundle to generate the packs, deltas, etc.
Some comments with minor clarifications: #1: files such as password and group shouldn't change as the sstate-cache that contains them is untouched. However, if that re-generation changes, that's okay since it will produce a full system version number and in that number if will be consistent -- since all bundles are sharing the same file. #2: in swupd terminology there is no "core image", just bundles. There is a convention to call "core image" as "os-core bundle". #2: what are the plans for us to define/declare bundles? A bundle can be a package or a package-group. But not all package or package-group should be a bundle. #2: the diff needs to result in actual `sysroot` directories per-bundle, per-version. That is we need MYIMAGE/MYVERSION0/os-core and MYIMAGE/MYVERSION1/os-core. Then we'll give these two directories to swupd-tool and it's going to compare each file to see what changed, registering that in Manifest.os-core. This is what you wrote as #4. NOTE: if you can generate each bundle by using `hardlink` to original file, it may save lots of disk space. Sasha (now in CC) also thinks that we could benefit better from YP architecture if, instead of generating one big-image and slicing it as mentioned (as in my original idea), we could simply create various sysroots and install only the required packages in each sysroot.
(In reply to comment #1) > Some comments with minor clarifications: > > #1: files such as password and group shouldn't change as the sstate-cache > that contains them is untouched. However, if that re-generation changes, > that's okay since it will produce a full system version number and in that > number if will be consistent -- since all bundles are sharing the same file. > > #2: in swupd terminology there is no "core image", just bundles. There is a > convention to call "core image" as "os-core bundle". > > #2: what are the plans for us to define/declare bundles? A bundle can be a > package or a package-group. But not all package or package-group should be a > bundle. Right, we don't want to create bundles for all package-groups. The intent is to have some variable(s) which define bundles but I don't have a concrete example of how I expect this to work yet. > #2: the diff needs to result in actual `sysroot` directories per-bundle, > per-version. That is we need MYIMAGE/MYVERSION0/os-core and > MYIMAGE/MYVERSION1/os-core. Then we'll give these two directories to > swupd-tool and it's going to compare each file to see what changed, > registering that in Manifest.os-core. This is what you wrote as #4. NOTE: if > you can generate each bundle by using `hardlink` to original file, it may > save lots of disk space. Agreed, only our goal is to have these directories processed by swupd automatically via the class being implemented for #9039. ACK on hardlink, I'm investigating its use and have a hardlink-native recipe in my local tree. > Sasha (now in CC) also thinks that we could benefit better from YP > architecture if, instead of generating one big-image and slicing it as > mentioned (as in my original idea), we could simply create various sysroots > and install only the required packages in each sysroot. This was the route we originally took, however after discussion with various folks I think the image slicing route is safer as an initial implementation. There are several examples of things which _might_ not work well in the "installing package into sysroot" scenario which the image slicing approach should avoid: * creating files outside of package creation, i.e. generating files from a postinst * postinsts in general i.e adding users * full image processing as is done in, for example, the meta-intel-iot-security layer's IMA signing
A pull request has been submitted for initial swupd support, as part the swupd-image class will generate chroot-like directories for swupd bundles. http://lists.openembedded.org/pipermail/openembedded-core/2016-February/117983.html
The swupd-image and swupdbundle classes in meta-swupd[1] provide an implementation of the required functionality for this issue and bundle chroot-like directories are created as part of the process of building a swupd enabled image and update artefacts. I will document how to use these classes, as well as how to generate just the bundle chroots (without using swupd to generate update artefacts from within the build-system) as part of #9191. 1. http://git.yoctoproject.org/cgit/cgit.cgi/meta-swupd
Have my comments about integration with do_rootfs and actual file content been addressed? See the OE-core mailing list threat.
(In reply to comment #5) > Have my comments about integration with do_rootfs and actual file content > been addressed? See the OE-core mailing list threat. Apologies for not responding to that thread sooner, they are in the process of being addressed and I filed a ticket for them: https://bugzilla.yoctoproject.org/show_bug.cgi?id=9193
Does it make sense to mark this as resolved when the solution isn't complete yet?
Reopening until fix for #9193 is merged
The latest meta-swupd change implements populating the bundle and rootfs contents from the corresponding files in the mega image.