Bug 7374

Summary: Add support for VirtualBox VDI images
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Juro Bystricky <juro.bystricky>
Component: coreAssignee: Juro Bystricky <juro.bystricky>
Status: VERIFIED FIXED QA Contact: Daniel Istrate <daniel.alexandrux.istrate>
Severity: enhancement    
Priority: Medium CC: bogdanx.a.voiculescu, geoffroy.vancutsem, meta.mr.watcher, meta.watcher, ross.burton, srifenbark, ulf.hofemeier
Version: unspecified   
Target Milestone: 1.9   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)
Attachments:
Description Flags
patch for VDI image support
none
VDI support patch none

Description Juro Bystricky 2015-02-26 23:52:15 UTC
Created attachment 2425 [details]
patch for VDI image support

Yocto does not support VirtualBox sparse image VDI file format.
This could be achieved by the attached patch. The support is almost identical to the VMDK support for VMware. The only subtle difference is that qemu-native has be be built with uuid enabled, as VDI images need a valid UUID in order to be recognized by VirtualBox.
I am aware that VirtualBox can use VMDK images as well, but VDI images do have some advantages in certain situations.
I tested the patch by creating a core-image-minimal in VDI format and booting the image successfully in VirtualBox SATA drive. 

core-image-minimal-vdi.bb:

IMAGE_FSTYPES = "vdi"
require core-image-minimal.bb
Comment 1 Saul Wold 2015-04-28 20:31:13 UTC
*** Bug 7661 has been marked as a duplicate of this bug. ***
Comment 2 Ross Burton 2015-05-08 15:37:59 UTC
Juro, can you split that up into logical patches and send it to the list please?
Comment 3 Juro Bystricky 2015-05-08 22:41:39 UTC
(In reply to comment #2)
> Juro, can you split that up into logical patches and send it to the list
> please?

Rebased, retested and sent to the list....
Comment 4 Scott Rifenbark 2015-06-02 15:04:19 UTC
Note about the documentation flag: 

I don't think this affect the YP user documentation manual set.  

Scott
Comment 5 Ross Burton 2015-06-02 15:12:50 UTC
Scott: this would impact the documentation as there'll be a new image type to document for IMAGE_FSTYPES.
Comment 6 Scott Rifenbark 2015-06-02 15:26:19 UTC
Ross, 

Thank you.  The variable affected would be IMAGE_TYPES, which lists all the supported image types as follows.  Here is a link to the change: http://www.yoctoproject.org/docs/1.9/ref-manual/ref-manual.html#var-IMAGE_TYPES.

I am setting the documentation flag to "done" for this bug. If you feel there is more, let me know.

Scott
Comment 7 Ross Burton 2015-06-02 15:50:43 UTC
The patch hasn't been merged yet, so the docs are now wrong :)
Comment 8 Juro Bystricky 2015-06-02 22:46:23 UTC
Created attachment 2535 [details]
VDI support patch
Comment 9 Juro Bystricky 2015-06-02 22:46:56 UTC
I attached a new patch that I plan on submitting (I'll break it into several smaller patches). The new patch merges the two image classes vmdk,vdi into one:
image-vm.bbclass (Saul's suggestion).
The actual type of the resulting image is determined
by setting

IMAGE_FSTYPES = "vmdk"

or

IMAGE_FSTYPES = "vdi"

I tested this with core-image-minimal-vmdk.bb and core-image-minimal.vdi.bb
The only recipe affected seems to be build-appliance-image.
I'll test it a bit more, meanwhile any comments are welcome.
Comment 10 Geoffroy Van Cutsem 2015-06-03 07:26:33 UTC
I have a question on this patch... I'm no expert on Yocto and these classes so please bear with me if I don't make a lot of sense... 

The patch includes this:
(line 12 - 13 of meta/classes/image-vm.bbclass)
...
# creating VM images relies on having a hddimg so ensure we inherit it here.
inherit boot-directdisk
...

This is a little misleading as inheriting boot-directdisk actually creates an ISO image (and not an hddimg one)> I believe this is a thing of the past where the original class for vmdk was originally using hddimg as the baseline but then later moved to using an ISO image as the starting point (I don't know the reason for that).

Anyway, in our project we actually only generate hddimg images and if we want to generate a vdi or vmdk one, it actually now also automatically creates an iso because of that inheritance (and the resulting vmdk image does not boot for reasons that I have not investigated while doing a qemu-img convert of the hddimg to vmdk actually works so this extra step creating an iso file messes something up).

Would it be possible to not depend on one type of image and be able to use whatever is being created already?
Comment 11 Scott Rifenbark 2015-06-03 12:38:10 UTC
Juro, 

Does your change affect the list in http://www.yoctoproject.org/docs/1.9/ref-manual/ref-manual.html#var-IMAGE_TYPES ?

Scott
Comment 12 Juro Bystricky 2015-06-03 15:10:25 UTC
(In reply to comment #11)
> Juro, 
> 
> Does your change affect the list in
> http://www.yoctoproject.org/docs/1.9/ref-manual/ref-manual.html#var-
> IMAGE_TYPES ?
> 
> Scott

The document is correct. 
Generally, what applies to vmdk also applies to vdi.
Comment 13 Juro Bystricky 2015-06-03 16:12:59 UTC
(In reply to comment #10)
> I have a question on this patch... I'm no expert on Yocto and these classes
> so please bear with me if I don't make a lot of sense... 
> 
> The patch includes this:
> (line 12 - 13 of meta/classes/image-vm.bbclass)
> ...
> # creating VM images relies on having a hddimg so ensure we inherit it here.
> inherit boot-directdisk
> ...
> 
> This is a little misleading as inheriting boot-directdisk actually creates
> an ISO image (and not an hddimg one)> I believe this is a thing of the past
> where the original class for vmdk was originally using hddimg as the
> baseline but then later moved to using an ISO image as the starting point (I
> don't know the reason for that).
> 
> Anyway, in our project we actually only generate hddimg images and if we
> want to generate a vdi or vmdk one, it actually now also automatically
> creates an iso because of that inheritance (and the resulting vmdk image
> does not boot for reasons that I have not investigated while doing a
> qemu-img convert of the hddimg to vmdk actually works so this extra step
> creating an iso file messes something up).
> 
> Would it be possible to not depend on one type of image and be able to use
> whatever is being created already?

The patch contains two simple recipes I used for testing:
core-image-minimal-vmdk.bb
core-image-minimal-vdi.bb

They both create a corresponding hddimg but there is no ISO image being created.
The resulting .vmdk and .vdi images need to use SATA controller, with IDE controller they will not boot.
Comment 14 Juro Bystricky 2015-06-03 18:11:25 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > Juro, can you split that up into logical patches and send it to the list
> > please?
> 
> Rebased, retested and sent to the list....

OK, I just sent an series of patches to the list (7 commits).
Comment 15 Geoffroy Van Cutsem 2015-06-04 09:00:34 UTC
(In reply to comment #13)
> (In reply to comment #10)
> The patch contains two simple recipes I used for testing:
> core-image-minimal-vmdk.bb
> core-image-minimal-vdi.bb
> 
> They both create a corresponding hddimg but there is no ISO image being
> created.
> The resulting .vmdk and .vdi images need to use SATA controller, with IDE
> controller they will not boot.

Thanks for the clarification. The restriction regarding the IDE controller is not a big issue but can you tell me where it comes from?
Comment 17 Daniel Istrate 2015-09-28 12:22:10 UTC
Verified on master: 7b86c771c80d0759c2ca0e57c46c4c966f89c49e

Steps to verify:
1. Create a simple recipe 'core-image-minimal-vdi.bb' under 'meta/recipes-core/images'.
2. In that recipe add:
IMAGE_FSTYPES = "vdi"
require core-image-minimal.bb
3. Build the recipe 'bitbake core-image-minimal-vdi' and the vdi file should be created: tmp/deploy/images/genericx86-64/core-image-minimal-vdi-genericx86-64.vdi
4. In VirtualBox create a new VirtualMachine and when choosing the hard disk chose 'Use an existing vurtual had disk file' and point to the vdi file created at step 3
5. Start the VirtualMachine earlier created and make sure it boots.