9599 2016-05-05 21:54:14 +0000 Reduce bundle overheads by not depending on os-core when generating bundle sysroots 2017-12-06 16:54:05 +0000 1 1 7 Build System, Metadata & Runtime Other YP Layers meta-swupd 2.2 x86 Multiple RESOLVED FIXED Medium+ enhancement 2.2 M1 1 joshuagloe joshuagloe igor.stoppa jose.perez.carranza --- --- No (bug/feature does not impact docs) oldest_to_newest 62004 0 joshuagloe 2016-05-05 21:54:14 +0000 We currently generate the bundle contents by performing a diff between an os-core image and an image that consists of os-core + the bundle contents. This adds significant overhead when adding bundles, especially noticeable with small bundles (an extreme example being a bundle which only installs a data file). It could be possible to improve performance significantly by only installing the contents of the bundle into the swupd image directory for processing by swupd. 62097 1 igor.stoppa 2016-05-11 11:53:49 +0000 This would be probably the most important feature, to improve the overall developer's experience. 62119 2 joshuagloe 2016-05-11 18:53:33 +0000 (In reply to comment #1) > This would be probably the most important feature, to improve the overall > developer's experience. Thanks for the feedback, the Medium+ Importance indicates that this feature should land in 2.2 M1 (but won't 'block' that milestone being completed) and therefore indicates it's a priority for the current development milestone. I agree it will be a big improvement for developer workflow and will prioritise accordingly. 62295 3 joshuagloe 2016-05-18 15:34:21 +0000 I have a partial implementation for this, unfortunately verification fails as follows: Verifying files Hash mismatch for file: /bin Hash mismatch for file: /etc Hash mismatch for file: /lib Hash mismatch for file: /lib/systemd Hash mismatch for file: /lib/systemd/system Hash mismatch for file: /lib/systemd/system/multi-user.target.wants Hash mismatch for file: /lib/systemd/system/sockets.target.wants Hash mismatch for file: /sbin Hash mismatch for file: /usr Hash mismatch for file: /usr/bin Hash mismatch for file: /usr/sbin Hash mismatch for file: /usr/share Hash mismatch for file: /usr/share/dbus-1 Hash mismatch for file: /var Inspected 2082 files 14 files did not match I've pushed the current WIP patch to the joshuagl/wip-rm-bundle-images: http://git.yoctoproject.org/cgit/cgit.cgi/meta-swupd/commit/?h=joshuagl/wip-rm-bundle-images&id=7e59fdb48dc28c3a8df17282f632a460a05725df 62355 4 joshuagloe 2016-05-19 22:21:08 +0000 The proposed solution now produces images which succeed at verification on my relatively simple OS. I intend to do further testing of: * swupdimage images (images consisting of os-core + N bundles) * updates * bundle addition * bundle removal I don't expect updates or bundle addition to be affected by the changes, but would like to have some certainty by testing. 62361 5 joshuagloe 2016-05-20 12:52:16 +0000 I realised there's an edge case we have to handle differently, when the bundle defines BUNDLE_FEATURES (which adds extra IMAGE_FEATURES for inclusion in that bundle) I think we must handle things the old way with a virtual image. We presume that the user is aware that an IMAGE_FEATURE that doesn't map to a set of installed packages is not really supported (because a bundle's contents must only add files to the system, not change files in os-core, I realise now that I need to add this to docs/Guide.md) but we can't be sure that an IMAGE_FEATURE directly maps to an installable target (i.e. a packagegroup) — I believe that this is true for dbg, dev and ptest IMAGE_FEATURES, thus for bundles which utilise BUNDLE_FEATURES we must build a virtual image and copy out the files from the resulting image which aren't in os-core. The good news here is that the copyxattrfiles() function we already added in this series makes copying files from a virtual image more efficient than before. 62566 6 joshuagloe 2016-06-01 16:44:51 +0000 This is now implemented in meta-swupd master, the change spans multiple commits with the main ones being: http://git.yoctoproject.org/cgit/cgit.cgi/meta-swupd/commit/?id=714e77c917cd991e34fd8c7c53293cbf41e81563 and http://git.yoctoproject.org/cgit/cgit.cgi/meta-swupd/commit/?id=57e636e999cd2598e907fe822a3cf548aac69555 to address the BUNDLE_FEATUREs case