Bug 7672 - Allow automating running wic from the build system
Summary: Allow automating running wic from the build system
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: deployment (show other bugs)
Version: 1.9
Hardware: All Multiple
: Medium enhancement
Target Milestone: 1.9
Assignee: Igor Stoppa
QA Contact: Daniel Istrate
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-04-28 10:39 UTC by Paul Eggleton
Modified: 2015-09-21 13:11 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Eggleton 2015-04-28 10:39:07 UTC
At the moment if you want to use wic to generate a more complicated image file (as opposed to using it to write directly to some external media e.g. an SD card), having to run wic manually at the end of every build isn't ideal. It would be great if we could have a way of running it automatically from the build, perhaps by adding an additional class that you can inherit that would wrap around wic, giving you the ability to control how it is run through additional variables.
Comment 1 Ed Bartosh 2015-04-29 16:13:40 UTC
It's done a bit different way, but the result is the same, i.e. you don't need to run bitbake and then wic.

Here is how it's done: http://cgit.openembedded.org/openembedded-core/commit/?id=75ae0b7cf3863eb2857a2b6a7073beea626f751d

Please, let me know if this solution is ok for you.
Comment 2 Igor Stoppa 2015-05-04 19:07:57 UTC
I also would prefer that wic creates the image as part of the bb execution.

Reasons for this:

1) running wic after bb would require a duplication of the logic used to iterate over the possible image types

2) calling bb from wic would be even worse, as it would effectively void the optimizations that bb can do when there are multiple image targets.

3) architecturally, it seems anyway the proper choice, as bb is already capable of building images, albeit in a very limited variety (1), so it would seem only natural to extend the current capability by replacing the current image creator with wic, which should behave like a blackbox for current users and enhance the experience of users in need of more complex partition setup.
Comment 3 Ed Bartosh 2015-05-05 11:22:59 UTC
This feature will be implemented this way:

if IMAGE_FSTYPES contains "wic" bitbake will call "wic create <wic image> -e bitbake target>"

where <bitbake target> is a name of bitbake image target
       <wic image> is a name from the space separated list WIC_WKSFILES specified in <bitbake target>.bb

.wks files should be put to the same location with <bitbake target>.bb, i.e. to meta*/recipes*/images/
Comment 4 Ed Bartosh 2015-05-05 14:40:29 UTC
moved to 1.9
Comment 5 Igor Stoppa 2015-06-01 15:33:49 UTC
(In reply to comment #3)
> This feature will be implemented this way:
> 
> if IMAGE_FSTYPES contains "wic" bitbake will call "wic create <wic image> -e
> bitbake target>"
> 
> where <bitbake target> is a name of bitbake image target
>        <wic image> is a name from the space separated list WIC_WKSFILES
> specified in <bitbake target>.bb
> 
> .wks files should be put to the same location with <bitbake target>.bb, i.e.
> to meta*/recipes*/images/

I do not like this solution, at least not as final one, because wic should be _the_ way to create real images.
So it should not be necessary to say that the image type is wic.

For example the so called hddimg that is currently produced fro mthe ready-made recipes, is nothing more than a vfat partition.

Similarly, there is a pollution of recipes doing basically the same stuff (ex: populate a FAT32 partition), each in its own way.
And for what I could see, none of them uses the pre-defined recipes.

The solution you are proposing doesn't seem to be any different.

What I would like to have, for a clean implementation, is a recipe which creates and populates one or more directories, each of them corresponding to a partition.

Then wic should be called with a .wks file which describes, for each directory, in which type and size of partition it should go.

And it should be possible to create empty partitions, w/ and w/o filesystem.
Comment 6 Ed Bartosh 2015-06-10 11:18:39 UTC
It's not final solution of course. However, we can't just replace non-partitioned images produced by bitbake now with partitioned images produced by wic. We should be able to support both types at least for some time.(In reply to comment #5)
> (In reply to comment #3)
> > This feature will be implemented this way:
> > 
> > if IMAGE_FSTYPES contains "wic" bitbake will call "wic create <wic image> -e
> > bitbake target>"
> > 
> > where <bitbake target> is a name of bitbake image target
> >        <wic image> is a name from the space separated list WIC_WKSFILES
> > specified in <bitbake target>.bb
> > 
> > .wks files should be put to the same location with <bitbake target>.bb, i.e.
> > to meta*/recipes*/images/
> 
> I do not like this solution, at least not as final one, because wic should
> be _the_ way to create real images.

It's not final solution of course. However, we can't just replace non-partitioned images produced by bitbake now with partitioned images produced by wic. We should be able to support both types at least for some time.

> So it should not be necessary to say that the image type is wic.
>
> For example the so called hddimg that is currently produced fro mthe
> ready-made recipes, is nothing more than a vfat partition.
> 
It is a bootable image and we should continue support this.


> Similarly, there is a pollution of recipes doing basically the same stuff
> (ex: populate a FAT32 partition), each in its own way.
> And for what I could see, none of them uses the pre-defined recipes.
> 
> The solution you are proposing doesn't seem to be any different.
> 
I don't understand this. Of course it's different as it will produce partitioned images.

> What I would like to have, for a clean implementation, is a recipe which
> creates and populates one or more directories, each of them corresponding to
> a partition.
> 
Perfect. But this doesn't mean that we should break existing recipes. At least not before we have working solution. That's why introducing 'wic' image types should make it possible to build both types of images.  

> Then wic should be called with a .wks file which describes, for each
> directory, in which type and size of partition it should go.
> 
Yes, but how that conflicts with proposed solution?

> And it should be possible to create empty partitions, w/ and w/o filesystem.
I think it's already possible. At least there is bug 7131 about documenting it.
Comment 9 Ed Bartosh 2015-08-31 09:41:10 UTC
Changes have been merged to master: http://cgit.openembedded.org/openembedded-core/commit/?id=173d440c14ee3140ae08c6a87decc9b2f4c9e391
Comment 10 Ed Bartosh 2015-08-31 09:42:10 UTC
Assigned to Igor for verification.
Comment 11 Daniel Istrate 2015-09-21 13:11:07 UTC
Verified on master: 7b86c771c80d0759c2ca0e57c46c4c966f89c49e

There is a recipe: ./meta-selftest/recipes-test/images/wic-image-minimal.bb which has a wks file associated ./meta-selftest/recipes-test/images/wic-image-minimal.wks

This allows you to build a wic image from within bitbake:

bitbake wic-image-minimal

which deploys the images to:
tmp/deploy/images/genericx86-64/wic-image-minimal-genericx86-64*