Bug 10073 - wic: generic bootloader-agnostic EFI plugin
Summary: wic: generic bootloader-agnostic EFI plugin
Status: RESOLVED FIXED
Alias: None
Product: BSPs
Classification: Build System, Metadata & Runtime
Component: bsps-oe-core (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 4.99
Assignee: California Sullivan
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks: 10084 10966 11291
  Show dependency tree
 
Reported: 2016-08-03 01:12 UTC by Jianxun Zhang
Modified: 2018-03-06 11:09 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
WIP: patch (7.74 KB, patch)
2016-08-03 01:13 UTC, Jianxun Zhang
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jianxun Zhang 2016-08-03 01:12:04 UTC
wic: generic bootloader-agnostic EFI plugin

    This change tries to achieve more by doing less.

    The current design around EFI plugin and wks files are bootloader-
    specific in logic and usage. Current approach may cause fragmentation
    in functionality between bootloader features provided from OE and
    features in wic plugin and wks files in maintainence. For example,
    a build for a machince could have default boot entry files built
    from OE, but still uses a wks file to construct a direct-boot image.
    The two usages are not always synchronized.

    The idea is to have a generic EFI plugin to generate bootable EFI
    image compilant with UEFI spec, but don't care which bootloader and
    its configuration data any more. Developers are responsible to
    provide content of a boot image, which could be customized via OE
    or not, to plugin.

    This should streamline the design and functions between wic and
    OE. Some options only supported in the existing wic EFI plugin
    could be added in OE as a consequence if OE doesn't have them.


I attached a WIP patch. What missed is a black list parameter and some patches in OE to support options we strip out from this generic EFI plugin.
Comment 1 Jianxun Zhang 2016-08-03 01:13:01 UTC
Created attachment 3335 [details]
WIP: patch
Comment 2 Jianxun Zhang 2016-10-31 19:20:38 UTC
This task has been pending for a while. I will have a look on the recent changes in wic area in OE before resuming this task.
Comment 3 Ed Bartosh 2016-11-01 15:45:21 UTC
Can we utilise/extend existing bootimg-partition or rootfs plugin for this purpose?

If we can have a directory populated with EFI artefacts by bitbake build then wic task is simple - to copy it to the boot partition. This is exactly what booting-partition plugin is for.
Comment 4 Jianxun Zhang 2016-11-01 16:59:33 UTC
Ed,
Thanks for bring this bootimg-partition in my sight. Actually, we need an interface to deploy a non-bootloader-related database file to ESP for RMC feature. So far we are using EFI_PROVIDERS as a workaround in OE build and the existing bootimg-efi only deploy files for a bootloader. This is where this task is derived.

I just checked an old build:

() RMC database file is present in ./tmp/deploy/images/intel-corei7-64/rmc.db. This location should be $DEPLOY_DIR_IMAGE.

() From YP doc. source files for $IMAGE_BOOT_FILES are in $DEPLOY_DIR_IMAGE.

() $IMAGE_BOOT_FILES, from my quick look in code, should be the source of bootimg-partition plugin.

I feel rmc.db can be deployed if we switched to bootimg-partition plugin. I do see EFI binary of bootloader in ./tmp/deploy/images/intel-corei7-64/, but don't see the configuration files of the bootloader populated in this directory.

I will do a build to verify it.
Comment 5 Ed Bartosh 2016-11-02 11:05:16 UTC
Let me emphasise my opinion on this: wic should operate with already prepared content of partitions with as little modifications as possible. Even if booting-partition can't do what is needed we should modify it or rootfs plugin, but we should avoid configuring bootloaders or do modifications on wic side.
Comment 6 Ed Bartosh 2017-03-08 14:43:04 UTC
Started to work on this.
Comment 7 Ed Bartosh 2017-05-10 08:33:34 UTC
Instead of introducing yet another wic plugin it uses existing APIs from
EFI_PROVIDER classes to populate ${WORKDIR}/bootfs directory with EFI artifacts
and bootloader configuration. This directory can be used by wic rootfs plugin
to put into boot partition of the image.

Example kickstart file and 2 test cases were added to wic test suite to better
illustrate the approach.

I personally like this approach much better than duplicating oe image creation
functionality in wic plugins. This way we can have the code in one place and
bootloaders can be configured the same way for oe and wic images.

The implementation has been sent to the mailing list for review:
http://lists.openembedded.org/pipermail/openembedded-core/2017-May/136463.html
Comment 10 brian avery 2017-06-13 17:51:14 UTC
got dropped?
Comment 11 Ed Bartosh 2017-06-14 14:15:08 UTC
hopefully not.

My patchset will probably be merged.
However, there is an alternative proposal: http://lists.openembedded.org/pipermail/openembedded-core/2017-June/137441.html
Comment 12 Ed Bartosh 2017-07-06 15:19:22 UTC
Assigning to Saul for experiments.

The idea is to take this further and make bootfs populated by the recipe. I tried to do that, but failed to find a way to make unique partition UUID generated in the recipe and synchronise it with wic. It was not easy even in my initial implementation (I did it in .bbclass by merging 2 tasks into one: http://lists.openembedded.org/pipermail/openembedded-core/2017-June/138204.html)

Please, reassign back to me when you find out how we should proceed further.
Comment 13 Saul Wold 2017-11-22 17:17:58 UTC
Cal is now looking into this an has WIP that uses a rootfs postprocess function to copy the bootfs 

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=clsulliv/image-creation-rebased

This allows for recipes to install rather then deploy bootfs and bootloader items, this allows for layers to add bootfs/bootloader items.

We are also looking into how to address the multi-kernel, but the challenge is who actually owns writing bootloader entries.
Comment 14 California Sullivan 2017-11-22 18:48:35 UTC
With the above branch anything installed to /boot/ ends up in the boot partition with no additional work. It also has multi kernel support thanks to Harris' multi kernel patch.

Some limitations/issues:
* Must use efi-bootdisk.wks.in as EFI_PROVIDER
* core-image-minimal doesn't install the preferred kernel to /boot/
* In other images the preferred kernel ends up in /boot/ twice, once as bzimage and once as its whole name due to my copy function
* Only systemd-boot and grub-efi bootloaders have packages that install to /boot/ so far
* live images (with rootfs.img) can't be created with this yet
Comment 15 Stephano Cetola 2018-02-22 23:07:55 UTC
Moving this to 2.6. The EFI enhancements make this project much easier, and those should hopefully make 2.5.
Comment 16 California Sullivan 2018-02-26 20:51:22 UTC
I sent a pared down version of my patchset to the list: http://lists.openembedded.org/pipermail/openembedded-core/2018-February/147837.html

It does not include the random UUID stuff, as I was unable to get it working sanely before M3. Instead we assume /dev/sda, but its overwritable.
Comment 17 California Sullivan 2018-03-06 11:09:38 UTC
v3 of the patchset was accepted. See here (and the previous 20ish commits):
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=ff9a4d598919e6f148c01017f2fc8ecc784b722d

I'll open a new bug for the UUID portion to be done in 2.6.