Bug 4082

Summary: Replace grub-efi with gummiboot
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Darren Hart <dvhart>
Component: deploymentAssignee: Darren Hart <dvhart>
Status: VERIFIED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: alexandrux.palalau, bluelightning, jason.wessel, koen, peterlei, sstncr
Version: 1.4   
Target Milestone: 1.6 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on:    
Bug Blocks: 4106, 4639    

Description Darren Hart 2013-03-21 22:42:27 UTC
For EFI machines, replace the rather hacked together grub-efi recipe with gummiboot.

http://freedesktop.org/wiki/Software/gummiboot
Comment 1 Jason Wessel 2013-03-25 13:45:30 UTC
It is not clear that gummiboot supports concurrent serial and VGA, but this is needed to support simplified installation scenarios. 

Please see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=3944

The above reference shows screen shots of the existing loader features where serial and vga can be enabled concurrently while also displaying a boot splash screen.
Comment 2 Darren Hart 2013-05-03 15:01:11 UTC
Recipe and patches available in the minnow layer. Works in master, fails in dylan. Need some help understanding why.

http://git.infradead.org/users/dvhart/meta-minnow
Comment 3 Darren Hart 2013-09-17 17:05:19 UTC
Gummiboot has been updated upstream and the recipe in master has been updated to use the upstream commits.

commit 8d3f10e5e760d0af3f068d442655b176953f2b5a
Author: Darren Hart <dvhart@linux.intel.com>
Date:   Thu Aug 22 11:30:46 2013 -0700

    gummiboot: Update to current HEAD (35+)

However, due to the dependency on grub-efi of the recent EFI and bootimage work by Jason Wessel, replacing grub-efi with gummiboot would be far to disruptive at this point in the release cycle for 1.5.

Moving to 1.6.
Comment 4 Jason Wessel 2013-09-17 17:20:37 UTC
I agree that this can wait.

Let us sort out the unified booting with grub first, now that it is working.  Then the bootimg.bbclass can be cleaned up a bit to support the bootloader of choice more easily.
Comment 5 Stefan Stanacar 2014-02-27 16:06:06 UTC
I now just saw this feature and coincidentally I'm working on something along the lines, but for other reasons than those stated here.

I have imported gummiboot from meta-intel and updated it and also have a gummiboot class that should emulate grub-efi and be a drop in replacement in bootimg.

Here's a preview (it's missing a create config section where I'm stuck because gummiboot seems to support multiple configurations...)

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=stefans/gummiboot

Any help is appreciated.
Comment 6 Darren Hart 2014-03-04 22:20:26 UTC
Koen is working on something similar. This looks to be a good start. Can you elaborate on the missing config section? Gummiboot does allow for multiple .conf files being combined together - are you running into a problem with how to copy that resulting set of .conf files?
Comment 7 Stefan Stanacar 2014-03-05 08:30:36 UTC
So I've successfully built and booted a genericx86-64 on NUC with gummiboot.
Just to make it clear: - the gummiboot recipe is just fine, it installs and if you have a properly configured kernel (with CONFIG_EFI_STUB and CONFIG_EFI_VARS) you can actually use it to boot the system once you've provided a /boot/loader/loader.conf and a entry.conf file.

My problem was with the gummiboot class and specifically with creating those configs similar to how grub-efi does. I think I solved that, I'm hoping to push something usable today.

Now, you've said something interesting: that gummiboot allows for one configuration file? How so? As far as I could tell from the little docs I've found it's config system works as follows:
 - $(esp)/loader/loader.conf - the general config where the default boot entry option is
  $(esp)/loader/entries/<name>.conf (similar to each menuentry in grub)
That's what I saw here at the example section:
http://freedesktop.org/wiki/Software/gummiboot/
Comment 8 Stefan Stanacar 2014-03-05 08:32:17 UTC
Ss for what I still need to do: update the install shell scripts as they assume grub (meta/recipes-core/initrdscripts/files/init-install-efi.sh and such). For now I've used hacked local version.
Comment 9 Darren Hart 2014-03-05 20:36:59 UTC
Apologies, you are correct. I remembered it *could* support multiple conf files, but had forgotten or not realized that the loader.conf and entries.conf were both required in the minimal case. Congrats, and thank you. I look forward to seeing the pull request. Please Cc me.
Comment 10 Stefan Stanacar 2014-03-06 18:18:42 UTC
Patches posted on the ML.
I think for this feature to be closed when/if those get merged we still need to: 
- remove the duplicate recipes from meta-intel
- change gummiboot to be the default
- remove grub-efi
I'm not sure all that can happen for 1.6 though.
Comment 11 Darren Hart 2014-03-06 18:21:17 UTC
No need to remove grub-efi. Others will use it. Changing the default is sufficient.
Comment 12 Stefan Stanacar 2014-03-24 16:52:42 UTC
Patches merged, some commits are:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=b588a243e79433e76c904e338865c2b822f9f114
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=9d04183cf8aed9eaa2aa03058488b1198f73cea0
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=5f05bdda0bf3f9fadf287a0632c2fa697380273f

We can now select gummiboot as the boot loader using EFI_PROVIDER = "gummiboot". The default it's still grub-efi.
The old gummiboot recipe in meta-intel has been removed.
Darren is there anything else here that should be done so this can be closed? :)
Comment 13 Stefan Stanacar 2014-03-24 16:55:09 UTC
And I've missed the commit actually adding the gummiboot recipe
: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=de18567e623ffa1033dc1ba236cad2fa34143e56
Comment 14 Darren Hart 2014-03-24 17:20:09 UTC
Nope, I think that is adequate. We don't actually replace grub here yet... but I think we've accomplished the goal. It is up to the BSP layer to choose which bootloader they want to use, so it is beyond the scope of this bug to make that happen. This can be closed in my opinion provided sufficient has testing and validation has been performed (I have not personally been able to build and test with this).... I shall do so right now....
Comment 15 Darren Hart 2014-03-24 18:39:45 UTC
I was able to successfully boot an x86_64 board using gummiboot. The EFI_PROVIDER had to be set in my local.conf, it didn't take in my image layer for some reason, and took the grub-efi default.
Comment 16 Stefan Stanacar 2014-03-25 07:31:42 UTC
Hmm, why wouldn't that work I wonder? I suppose is something about the order of overrides. Was this an image recipe? Because I do think that distro/machine/local conf should work, I don't think recipes can overwrite that.
Comment 17 Darren Hart 2014-03-25 15:32:23 UTC
I specified this in a layer.conf.
Comment 18 Stefan Stanacar 2014-03-25 15:48:27 UTC
Ah, I don't think that can work.
I just did a build with EFI_PROVIDER = "gummiboot" in meta-yocto-bsp/conf/machine/genericx86-64.conf and it worked as expected.
So it has to be in local.conf or a distro conf or a machine conf.
Comment 19 Darren Hart 2014-03-25 16:00:34 UTC
Or perhaps in a layer.conf that is processed before poky/oe-core...
Comment 20 Darren Hart 2014-03-25 16:01:08 UTC
In any case, this is closed, thanks for the efforts.
Comment 21 Darren Hart 2014-03-25 19:33:43 UTC
Getting some reports on IRC as broken:

[12:23]  <volker-> dvhart: ExpansionError: Failure expanding variable EFI_CLASS, expression was ${@base_contains("MACHINE_FEATURES", "efi", "grub
[12:23]  <volker-> -efi", "", d)} which triggered exception NameError: global name 'oe' is not defined
[12:23]  <volker-> ERROR: Unable to parse bootimg: Failure expanding variable EFI_CLASS, expression was ${@base_contains("MACHINE_FEATURE
[12:23]  <volker-> S", "efi", "grub-efi", "", d)} which triggered exception NameError: global name 'oe' is not defined
[12:23]  <volker-> I think actually the error happens on the "inherit ${EFI_CLASS}"
[12:23]  <volker-> dora git, meta/classes/bootimg.bbclass

Was this tested with no efi in MACHINE_FEATURES
Comment 22 Stefan Stanacar 2014-03-26 07:30:14 UTC
Wait what? Of course it was tested with efi in MACHINE_FEATURES.
Can we get a commit ? or better yet a full bug report?
It looks strange to me that the guy said dora git? He backported the changes?
And this: "NameError: global name 'oe' is not defined" looks like the real problem. base_contains is defined in utils.bbclass which actually calls oe.utils.contains (meta/lib/oe/utils.py) so something is wrong the import/namespace in his case.
More information will definitely help in debugging this...
Comment 23 Stefan Stanacar 2014-03-26 07:56:33 UTC
And I've re-read :). Yup, I've tested with no efi too... I've also tested on dora too. Without more info about his configuration, I can't really do anything.
Comment 24 Stefan Stanacar 2014-03-26 08:03:50 UTC
Okay, I did a bit of digging around my IRC logs, and I luckily found these lines:
<volker-> hi
<volker-> I try to use "bootimg" with core-image-minimal. So I added it to USER_CLASSES in conf/local.conf.   Now I get the error that global name 'oe' is not defined when expanding EFI_CLASS

I'm pretty sure this isn't related to gummiboot or it's changes and also I don't think that's the right way to use bootimg.
I've added bootimg in local.conf to USER_CLASSES and I get the same thing as him:

ERROR: Unable to parse bootimg: Failure expanding variable EFI_CLASS, expression was ${@base_contains("MACHINE_FEATURES", "efi", "grub-efi", "", d)} which triggered exception NameError: global name 'oe' is not defined

This was on dora and master. So this is configuration error imho. I'm not saying we shouldn't error more nicely than showing an entire stack trace, but it has nothing to do with gummiboot or bootimg :)
Comment 25 Darren Hart 2014-03-26 18:01:09 UTC
Yes, I got volker sorted out. It was mass confusion about how to get an ISO in the end :-) Sorry for the noise. /me leaves this bug alone now.
Comment 26 Alexandru Palalau 2014-04-09 16:02:24 UTC
Verified on master branch.
Commit ID: bb66113bde5361b869dce2bdaece5b938f077ea8