Bug 5654

I'm going to summarize the results of the first day's experimentation and discussion on this bug.

Using QEMU 1.5.0 and the latest upstream OVMF firmware on my workstation, I was able to get QEMU to boot EFI images. I successfully booted a Debian installer USB drive that I had lying around with qemu -hda /dev/sdi. I also found that a core-image-base .hddimg file built by the intel-corei7-64 BSP booted correctly, and I was able to verify that it did indeed boot via an EFI code path by looking though dmesg.

So one task is going to be teaching Yocto's built-in QEMU (and the runqemu script) how to do this automatically, which is better than documenting how to do it manually. That also means including a BitBake recipe for the OVMF firmware itself. Matt Fleming of the Linux UEFI Validation Distribution [1] has already made a lot of progress along these lines, so closing this bug will probably involve upstreaming a bunch of his work.

The other task is actually adding UEFI support to the QEMU BSPs. This is not as easy as just adding stuff to MACHINE_FEATURES. Right now, the QEMU BSPs only generate EXT3 rootfs images, whereas the EFI firmware only understands FAT32 (well, actually, a special FAT32 variant.)

As I mentioned, the .hddimg files built by intel-corei7-64 work correctly with the OVMF firmware. However, according to Darren Hart, this is not actually technically a compliant configuration according to the EFI specification, so the fact that the OVMF firmware can cope with it is merely luck. The solution is to teach Yocto how to generate disk images that actually have partition tables on them.

However, having the QEMU BSPs generate .hddimg files in addition to .ext3 files would be a good first step, because we've already got code for it. It should be a matter of adding "hddimg" to IMAGE_FSTYPES, and in fact that is what I'm about to try.

Darren also pointed out that there is a possibility that some additional kernel configuration fragments may be needed for the QEMU BSP or else the kernel will refuse to boot via an EFI codepath. No way of finding out if that's needed until I have a .hddimg file so I can at least try EFI plus the QEMU kernel. 

Finally, I know this is a long way out, but wouldn't it be great if we could eventually retire qemux86 and qemux86-64 and just do, say, "runqemu intel-corei7-64" instead? qemuARM might need to stick around for a while, since my understanding is there's no standardized equivalent to EFI in the ARM camp.

[1] https://github.com/01org/luv-yocto

Created attachment 2010 [details]
qemux86 and qemux86-64 booting via EFI on QEMU

I now have both qemux86 and qemux86-64 building .hddimg files that can be booted via EFI on QEMU. I also confirmed that I can dd the qemux86-64 .hddimg to a USB stick and then boot it on a Minnowboard MAX. As mentioned earlier, these .hddimg files are not technically EFI compliant, but they seem to work nonetheless.

This was accomplished by adding the following lines to meta/conf/machine/qemux86.conf and meta/conf/machine/qemux86-64.conf:
    # UEFI stuff
    MACHINE_FEATURES += "efi pcbios"
    IMAGE_FSTYPES += "hddimg"
    KERNEL_FEATURES += "cfg/efi-ext.scc"
However, this is probably not the "correct" way to do it, especially the KERNEL_FEATURES bit.

Interestingly, if I just used cfg/efi.scc, the kernel would not boot. efi-ext.scc is required.

Adding hddimg to IMAGE_FSTYPES adds almost a thousand extra tasks to the core-image-base recipe, causing it to take much longer-- this may not be something we want to do, since qemux86 is supposed to be the quickie introductory default BSP. Perhaps separate QEMU EFI configurations would be better, to keep qemux86 building as quickly as possible.

I am continuing to work with Matt Fleming on OVMF support within Yocto. For now, this work is continuing within the luv-yocto fork. I've submitted a patch series to luv-yocto with some fixes. [1]

luv-yocto is (I believe) forked from Yocto 1.5.x, making it awkward to incorporate changes from upstream Yocto back into luv-yocto. Therefore, it seems premature to attempt to upstream the OVMF recipe quite yet. However, the changes and fixes to the runqemu scripts could probably be upstreamed with much less fuss.

I have also been transplanting the OVMF recipe over to a current Yocto tree and testing it there. OVMF uses the BaseTools build system, which presently has no support for GCC 4.9.0 as used by default in current Yocto master. A workaround is to fool BaseTools into thinking the compiler version is GCC 4.8 even when it's 4.9, which is actually very trivial to do. However, OVMF itself doesn't compile correctly with GCC 4.9.0 if the target architecture is 64-bit (it *does* work if the target architecture is 32-bit.) 64-bit GCC 4.9 builds of OVMF fail at the do_compile stage. 

A quick Google search indicates that the EDK2 developers are aware of the issue, and it will probably be resolved in either the next version of OVMF or the next version of GCC. [2] 

Forcing GCC to 4.8 in local.conf allows OVMF to compile correctly for 32-bit and 64-bit targets in Poky git master, but if it doesn't work with the default compiler, that's another reason not to upstream the OVMF recipe quite yet. 

[1] https://github.com/01org/luv-yocto/pull/14
[2] http://www.mail-archive.com/edk2-devel@lists.sourceforge.net/msg07459.html

(In reply to comment #2)
> Created attachment 2010 [details]
> qemux86 and qemux86-64 booting via EFI on QEMU
> 
> I now have both qemux86 and qemux86-64 building .hddimg files that can be
> booted via EFI on QEMU. I also confirmed that I can dd the qemux86-64
> .hddimg to a USB stick and then boot it on a Minnowboard MAX. As mentioned
> earlier, these .hddimg files are not technically EFI compliant, but they
> seem to work nonetheless.
> 
> This was accomplished by adding the following lines to
> meta/conf/machine/qemux86.conf and meta/conf/machine/qemux86-64.conf:
>     # UEFI stuff
>     MACHINE_FEATURES += "efi pcbios"
>     IMAGE_FSTYPES += "hddimg"
>     KERNEL_FEATURES += "cfg/efi-ext.scc"
> However, this is probably not the "correct" way to do it, especially the
> KERNEL_FEATURES bit.

This isn't "wrong" but yes, this feature should be added to the linux-yocto meta data for common-pc which qemu uses. Although even that isn't ideal as it should be using a qemux86 specific kernel that is even more tailored to the available "hardware".

> 
> Interestingly, if I just used cfg/efi.scc, the kernel would not boot.
> efi-ext.scc is required.

Hrm.... possibly due to the lack of VFAT FS support which efi-ext adds, but which most real hardware gets from the live feature.

> 
> Adding hddimg to IMAGE_FSTYPES adds almost a thousand extra tasks to the
> core-image-base recipe, causing it to take much longer-- this may not be
> something we want to do, since qemux86 is supposed to be the quickie
> introductory default BSP. Perhaps separate QEMU EFI configurations would be
> better, to keep qemux86 building as quickly as possible.

Yup, which hasn't been necessary with qemu machines until now. Some effort to see if there is a way to test this for VMs without a live image (like with two FS images instead) is warranted.

The upstream EDK2 developers have fixed the build issues with GCC 4.9 in their SVN repository. They also informed me that since they do not tag stable versions of OVMF, there's no reason not to just use the lateste SVN in the OVMF recipe. I have submitted the change to luv-yocto. 

I have also found that the qemux86 and qemux86-64 images boot just fine with OVMF, even without .hddimg files. It's just that this configuration doesn't test the bootloader or certain functionalities of UEFI itself. However, after speaking with Darren, I believe this is not necessary in order to close this bug.

Given that conclusion, as soon as my pull request is accepted, luv-yocto will have a *complete* solution to this bug. I'll leave upstreaming it at Matt Fleming's disgression, but when it does get upstreamed, this bug can probably be closed.

One caveat though. Running qemu -vga vmware will not work when using the OVMF firmware instead of SeaBIOS. I spoke to the EDK2 developers and they confirmed that this was by design: OVMF only supports the Cirrus and QXL VGA adapters when running under QEMU. In my pull request to luv-yocto, I changed runqemu to use the default Cirrus adapter if OVMF is used (vmware will continue to be selected otherwise.) If someone has a good reason to use QXL instead of Cirrus, or if there's a good reason the vmware VGA adapter was used in the original runqemu, this would be a good time to explain why.

As of linux-yocto-3.14 meta branch commit dacb1c1ab0119476333912f27375d0b16a7f9d99, qemux86 and qemux86-64 can now boot on via EFI. [1] However, until the OVMF recipe is upstreamed from luv-yocto, you'll have to supply your own OVMF build in order to do this.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto-3.14/commit/?h=meta&id=dacb1c1ab0119476333912f27375d0b16a7f9d99

Assigning this to Matt Fleming, given my impending departure. Just need to upstream OVMF and make those runqemu tweaks.

*** Bug 4417 has been marked as a duplicate of this bug. ***

I am adding a note for possible documentation impact.  Searching through the YP user docs, I think the only area that might be impacted would be the QEMU chapter in the dev-manual.  Perhaps a general statement indicating that EFI is supported for a boot option and a mention that you can download and use qemu with an EFI firmware.  

I am not sure of this but wanted to capture it.

Scott

(In reply to comment #9)
> I am adding a note for possible documentation impact.  Searching through the
> YP user docs, I think the only area that might be impacted would be the QEMU
> chapter in the dev-manual.  Perhaps a general statement indicating that EFI
> is supported for a boot option and a mention that you can download and use
> qemu with an EFI firmware.  

Scott, I already have a documentation patch regarding this. Once the bug/feature patch is send and ACK (to be send today or tomorrow, just being reviewed internally), I will sent the primer to the yocto@ list (and cc you). 

> 
> I am not sure of this but wanted to capture it.
> 
> Scott

(In reply to comment #10)
> (In reply to comment #9)
> > I am adding a note for possible documentation impact.  Searching through the
> > YP user docs, I think the only area that might be impacted would be the QEMU
> > chapter in the dev-manual.  Perhaps a general statement indicating that EFI
> > is supported for a boot option and a mention that you can download and use
> > qemu with an EFI firmware.  
> 
> Scott, I already have a documentation patch regarding this. Once the
> bug/feature patch is send and ACK (to be send today or tomorrow, just being
> reviewed internally), I will sent the primer to the yocto@ list (and cc
> you). 
> 
> > 

Leonardo, okay.  I did apply one small patch but I don't think it is the one you are referring to here.  I will look for it on the mailing list.

> > I am not sure of this but wanted to capture it.
> > 
> > Scott

The patches sent to the mailing list containing iasl and ovmf failed on the AB:

https://autobuilder.yoctoproject.org/main/builders/nightly-world/builds/495
https://autobuilder.yoctoproject.org/main/builders/nightly-world-lsb/builds/213

The reason is that iasl recipe needed a COMPATIBLE_MACHINE statement and ovmf recipe/patches were merged without considering the ending CR character, so patching phase failed. There is one more problem: ovmf has not been tested with GCC5.X. 

Local builds show an error on the compilation phase, apparently the ovmf build scripts only expect 4.X GCC versions. The luv-yocto main developers are aware and will provide either patches or recipes upgraded to support 5.X version.

Created attachment 2732 [details]
OVMF build error with GCC 5.2

Moving it to 2.0. luv-yocto team will test with gcc 5.2 and let us know the results.

I read through the comments here and am assuming that my Comment 9 suggestion is still the way to go once this issue passes test?  I am not sure.  I made that comment a while back so am trying to catch up on these types of fixes for the docs.  Let me know if anything changes for the documentation beyond what I think should be done.

Thanks,
Scott

This is a comment coming from Jordan Justen, a developer from edk2.

----------------
The EDK II GCC52 toolchain *does not exists*. There has not yet been a
reason to add an EDK II GCC5 toolchain, becuase the build rules in the
EDK II GCC49 toolchain work fine with GCC 5.*.

What I am saying is that you should not use, or create an EDK II GCC52
toolchain. You should use the EDK II GCC49 toolchain with GCC 5.*.
----------------

So I am pretty unclear on the doc aspect of this bug.  It has a long history.  That last Comment 16 concerns EDK, which does not appear in any of the documentation.  

Comment 12 has a couple links to patches that don't resolve for me.  So I am also not sure if I ever got a subsequent doc patch mentioned in Comment 10 given the long history of the bug and the fact that I can't resolve those patch links in Comment 12. 

Do I still need to be concerned with a doc aspect for this bug?

Thanks, 
Scott

(In reply to comment #17)
> So I am pretty unclear on the doc aspect of this bug.  It has a long
> history.  That last Comment 16 concerns EDK, which does not appear in any of
> the documentation.  
> 
> Comment 12 has a couple links to patches that don't resolve for me.  So I am
> also not sure if I ever got a subsequent doc patch mentioned in Comment 10
> given the long history of the bug and the fact that I can't resolve those
> patch links in Comment 12. 
> 
> Do I still need to be concerned with a doc aspect for this bug?
> 


Scott, sorry for the delay.

The problem is that the new feature does not support the current compiler version (GCC5.2), so recipe can not be on master. So without the feature, no need to do any documentation.

I will set the bug in its appropriate status.

> Thanks, 
> Scott

The EDK II project [1] that implements the UEFI specification, does not support the cross-compiler gcc with versions 5.X, so the recipe cannot be integrated at Jethro/Master. I am not aware of future plans of EDKII but I believe at some point the EDKII will be tested with GCC5.X. Moving the status to WaitForUpstream.


[1] http://www.tianocore.org/edk2/

*** Bug 9399 has been marked as a duplicate of this bug. ***

I set the documentation flag for this bug to "No".

Scott

(In reply to comment #19)
> The EDK II project [1] that implements the UEFI specification, does not
> support the cross-compiler gcc with versions 5.X, so the recipe cannot be
> integrated at Jethro/Master.

Leonardo, I think you misunderstood the comment from Jordan Justen (comment #16): what he's saying is that OVMF compiles fine with gcc 5.x when using the toolchain options for gcc 4.9 and therefore there's no need to introduce a GCC52 toolchain.

I arrived at the same conclusion after changing ovmf_git.bb like this:

GCC_VER="$(${CC} -v 2>&1 | tail -n1 | awk '{print $3}' | awk -F. '{print $1$2}' | sed -e 's/5./49/')"

OVMF builds and works fine with that, using current OE-core master (or more precisely, Ostro OS master, which is a bit behind).

IMHO nothing prevents finishing the work, we just need to agree on how to do it.

My proposal:
- move ovmf_git.bb from meta-luv to OE-core
- move iasl_20120215.bb from meta-luv to OE-core
- add "efi" to MACHINE_FEATURES for qemux86
- build ovmf when building qemu for qemux86 or qemux86-64
- optional: special support for ovmf in runqemu

To illustrate what I mean with that last point, let me show how booting works at the moment (using Ostro OS as example, because it has an image which depends on UEFI).

The recommended way for persistent UEFI variables is to use "-drive
if=pflash,format=raw,file=my-bios.bin" (http://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764918#47). In this mode, variables get stored in my-bios.bin and thus we should use a temporary copy.

So to start with qemu and OVMF, one has to:
$ cp tmp-glibc/sysroots/qemux86/usr/share/ovmf/bios.bin ./my-bios.bin
$ runqemu 'qemuparams=-drive if=pflash,format=raw,file=my_bios.bin -vga std' qemux86 ostro-image-noswupd dsk

"-vga std" is necessary because OVMF does not support the "-vga vmware" from qemux86.qemuboot.conf. Overriding that only works when patching runqemu so that parameters specified by the user come after the automatically determined parameters (a change which IMHO makes sense also by its own).

My proposal for making this simpler is to add a "ovmf" keyword which tells runqemu to look for bios.bin in STAGING_DIR_HOST/usr/share/ovmf and make a copy, then start qemu with these additional parameters. biosdir and bios special parameters could be used in combination with ovmf to override that.

I'm open for suggestions as to *where* that temporary copy should be located. In the current directory it'll survive removal of tmp*, which may or may not be intentional. In tmp*/deploy/image/<arch> its "closer" to the root partition, which also might get modified as part of booting. I suspect that this has to be configurable, so I propose introducing an optional "nvram" parameter which specifies where to store persistent data for the virtual machine. When combined with ovmf, that'll be the copy of the bios.bin.

The alternative is to implement option (3b) from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764918#47 and have a read-only flash with the OVMF_CODE.fg and a read/write flash with a copy of OVMF_VARS.fd. I like that even better...

Any comments?

Saul, if you want, I can take this bug and try to finish it.

Besides using the GCC49 toolchain definition with the current revision of the OVMF code, we can (and should) also update to the latest master, which adds a special GCC5 toolchain definition.

As an added bonus, latest master uses openssl 1.0.2j when enabling secure boot, instead of the out-dated 0.9.8za in the current code.

ovmf + acpica are now in OE-core master.

Remaining: removal of acpica from meta-oe

(In reply to comment #24)
> ovmf + acpica are now in OE-core master.
> 
> Remaining: removal of acpica from meta-oe

Both done.