7830 2015-06-01 08:11:20 +0000 Lack of clean way to generate images. 2015-06-04 11:22:01 +0000 1 1 7 Build System, Metadata & Runtime OE-Core configuration 1.8.1 x86 Multiple RESOLVED FIXED Undecided normal --- 1 igor.stoppa richard.purdie alexander.kanevskiy eduard.bartosh ross.burton --- --- Don't know oldest_to_newest 51376 0 igor.stoppa 2015-06-01 08:11:20 +0000 Image creation is implemented in a way that is difficult to track, understand and modify. Worse, it encourages proliferation of too many recipes trying to do almost the same, each in its own way. In particular: there are a bunch of different recipes all related to image creation (image.bbclass, boot.bbclass, etc.) and they even rely on python modules from oe (image.py), which depends on various parameters set from the recipes. What I would have expected: 1) build and install desired/required packages 2) create list or lists of packages to deploy, specifying the location (ex: root partition of efi dir, one or more rootfs - yes, some systems might have more than one) - such list cold be an enhanced the manifest or a collection of manifests 3) deploy files according to the manifest(s) 4) execution of wic, with .wks file describing, one for each line, the partitions, the content (directory from previous point), optionally the size. It should work both with directories (wic creates the partition) and files (wic does a binary dump). The latter case would support special raw partitions (ex: devices requiring the kernel in own specific location.) As user, I should not have to reverse engineer several layers of recipes and python libraries, writing own callbacks and setting a multitude of parameters. 51390 1 igor.stoppa 2015-06-01 15:23:11 +0000 The closest thing to what I described in point 3) is the tar image type, however that is still very bad, because it wastes processor time and disk space in creating a tarball that will not be used as such. From the perspective of performing the task in the most frugal way, it is preferable to avoid unnecessary steps. 51395 2 igor.stoppa 2015-06-01 16:02:13 +0000 This bug seems to be converging toward #7672, although it's not a duplicate. They are rather complementary. 51500 3 richard.purdie 2015-06-04 11:14:57 +0000 (In reply to comment #0) > Image creation Just to be clear, most people with experience of the system think of "image creation" as the construction of the rootfs with the package mamager (or whatever). There is then a packaging up of the rootfs phase (as a tarball or whatever) which is the part I believe you've concerns with. The bug title can therefore mislead people as to where the concern is. is implemented in a way that is difficult to track, > understand and modify. > Worse, it encourages proliferation of too many recipes trying to do almost > the same, each in its own way. > > In particular: there are a bunch of different recipes all related to image > creation (image.bbclass, boot.bbclass, etc.) and they even rely on python > modules from oe (image.py), which depends on various parameters set from the > recipes. > > What I would have expected: > > 1) build and install desired/required packages > > 2) create list or lists of packages to deploy, specifying the location (ex: > root partition of efi dir, one or more rootfs - yes, some systems might have > more than one) - such list cold be an enhanced the manifest or a collection > of manifests > > 3) deploy files according to the manifest(s) The way the system works, do_rootfs is a fairly monolithic task which does all of the above in steps, generating the rootfs directory and manifests and then packaging/deploying the result into a variety of forms. We do care about performance and where we can do things in parallel, we do. The reasons for do_rootfs being monolithic are historical. We did refactor the code there in a recent previous release, moving lots of archaic shell over to python classes which are a lot more understandable. Our intention was to further split the code up into specific tasks but as yet, we've not done this. > 4) execution of wic, with .wks file describing, one for each line, the > partitions, the content (directory from previous point), optionally the size. > It should work both with directories (wic creates the partition) and files > (wic does a binary dump). The latter case would support special raw > partitions (ex: devices requiring the kernel in own specific location.) > > As user, I should not have to reverse engineer several layers of recipes and > python libraries, writing own callbacks and setting a multitude of > parameters. I think #7672 is our best bet for moving this forwards. It gives a way forward to intetgrate wic into the system and doesn't break existing functionality. Once we have that, we can see which of the existing classes we can replicate with wic and the remove the old classes as obsolete. What we likely need is a set of bugs which split the process into logical steps, even one per class we intend to replace with wic functionality. That does leave the question of the objective of this specific bug and which actions would mean we'd resolved it? 51501 4 igor.stoppa 2015-06-04 11:22:01 +0000 ok, let's proceed as you propose