Bug 11552

Summary: wic: avoid unnecessary dependencies
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Patrick Ohly <patrick.ohly>
Component: Scripts and ToolsAssignee: Ed Bartosh <eduard.bartosh>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: meta.mr.watcher, meta.watcher, mikko.ylinen, ross.burton
Version: unspecified   
Target Milestone: 2.4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Patrick Ohly 2017-05-22 09:28:31 UTC
At the moment, when building images with IMAGE_FSTYPES=wic one ends up depending on wic-tools and thus syslinux and grub-efi even when not using those at all. Ideally, building an image with wic should only build the tools and components really needed.

The problem is that "wic-tools" is needed also for the manual invocations of wic, in which case everything that might be needed has to be built in advance.

Proposal: "wic-tools" should be reserved for this "use wic manually" case and automatic building via bitbake should use something else ("wic-tools-light") or no wic-tools at all.

Then "manual mode" uses wic as a generic tool that works for multiple images (at the expense of having to build things with might not be needed), while building through bitbake can be as efficient as possible for the desired images.

A "wic-tools-light" could be used to stage content that is absolutely needed for all images, which might be a bit more efficient than staging them once per image. Remains to be seen...

Has implications on documentation because users of the manual mode need to know that they have to build "wic-tools" - details to be provided by Ed.
Comment 1 Ed Bartosh 2017-05-22 09:35:57 UTC
I'll try to completely remove dependency to wic-tools from image_types_wic.bbclass. All dependencies should be brought by the image recipe I believe.

sic-tools will be left for usage only in manual mode, i.e. when wic is run by user as a command line tool.

Does this make sense for you?
Comment 2 Patrick Ohly 2017-05-22 09:46:49 UTC
(In reply to comment #1)
> Does this make sense for you?

Yes.
Comment 3 Patrick Ohly 2017-06-30 06:18:41 UTC
To capture a discussion I had with Ed on IRC:
- Dependencies depend (sic) on the actual .wks file that is used,
  not the image recipe. So together with setting a WKS_FILE, one also
  has to set the corresponding WKS_FILE_DEPENDS.
- Removing the default dependencies brought in via the current wic-tools
  would break image recipes that currently do not set WKS_FILE_DEPENDS.
  Therefore we need a WKS_FILE_DEPENDS default that mirrors the "common"
  tools that might be needed by a .wks file. Then nothing breaks
  and developers can set more restrictive dependencies when (and if)
  they care.
Comment 4 Mikko Ylinen 2017-06-30 06:29:52 UTC
(In reply to comment #3)
>   Therefore we need a WKS_FILE_DEPENDS default that mirrors the "common"
>   tools that might be needed by a .wks file. Then nothing breaks
>   and developers can set more restrictive dependencies when (and if)
>   they care.

Looks like we could try this already now in refkit. WKS_FILE_DEPENDS has been there since RSS was introduced (based on a quick git blame check). Why wic-tools was needed in the first place?
Comment 5 Ed Bartosh 2017-06-30 07:23:24 UTC
we decided to use wic-tools in a rush to fix RSS issues. At that time it seemed to be acceptable compromise that works in both modes - when wic is run by bitbake and when it's run manually.

Now it looks like this is not that acceptable anymore and we need to do something better than quick workaround.
Comment 6 Ed Bartosh 2017-06-30 14:15:37 UTC
Looks like we have a problem here.

Adding dependency to image recipe doesn't generally mean that it will be available from its sysroot. That breaks wic as it can't find bootloader artifacts.

For example, adding 
WKS_FILE_DEPENDS_append_x86 = " syslinux grub-efi systemd-boot"
to image_types_wic.bbclass doesn't make grub-efi to appear in its sysroot.
However, doing the same for wic-tools does the trick:

$ bitbake core-image-minimal wic-tools
$ find ./tmp/ -name x86_64-efi
./tmp/work/core2-64-poky-linux/grub-efi/2.02-r0/sysroot-destdir/usr/lib/grub/x86_64-efi
./tmp/work/x86_64-linux/wic-tools/1.0-r0/recipe-sysroot/usr/lib/grub/x86_64-efi
./tmp/sysroots-components/core2-64/grub-efi/usr/lib/grub/x86_64-efi


I think it's because wic-tools considered as a native package, but image recipies are not.
Comment 7 Ed Bartosh 2017-06-30 14:35:24 UTC
I forgot to add 'wic' to IMAGE_FSTYPES. That caused the above issue. Sorry for the noise.
Comment 8 Ed Bartosh 2017-07-06 08:19:42 UTC
The fix has been sent for review: http://lists.openembedded.org/pipermail/openembedded-core/2017-July/139158.html

It removes wic-tools from the list of image dependencies and gets rid of wic-tools usage in wic core and plugins. wic-tools will still be used when wic is run manually.
Comment 10 Ed Bartosh 2017-07-18 10:39:37 UTC
the patchset has been accepted to poky master