Bug 4072

Bumping this to M3, since further design is required.

Hey Bruce, can you provide an update? Cutoff for M4 is Aug 25th.

I'll check with a few people and update the progress.

We are doing a RFC implementation. Jason Wessel can update when things are ready to post.

A solution posted for review to the oe-core mailling list.  Patch is posted with patch header:

    kernel.bbclass,image.bbclass: Implement kernel INITRAMFS dependency and bundling
    
    This patch aims to fix the following two cases for the INITRAMFS generation.
      1) Allow an image recipe to specify a paired INITRAMFS recipe such
         as core-image-minimal-initramfs.  This allows building a base
         image which always generates the needed initramfs image in one step
      2) Allow building a single binary which contains a kernel and
         the initramfs.
    
    A key requirement of the initramfs is to be able to add kernel
    modules.  The current implementation of the INITRAMFS_IMAGE variable
    has a circular dependency when using kernel modules in the initramfs
    image.bb file that is caused by kernel.bbclass trying to build the
    initramfs before the kernel's do_install rule.
    
    The solution for this problem is to have the kernel's
    do_bundle_initramfs_image task depend on the do_rootfs from the
    INITRAMFS_IMAGE and not some intermediate point.  The image.bbclass
    will also sets up dependencies to make the initramfs creation task run
    last.
    
    The code to bundle the kernel and initramfs together has been added.
    At a high level, all it is doing is invoking a second compilation of
    the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
    to the generated initramfs from the image recipe.

Patch at:

http://lists.openembedded.org/pipermail/openembedded-core/2013-August/082861.html

Bruce, where are we on this?

Jason is running with it, I'll bring him into the loop. The first set of patches were rejected,
so we are re-evaluating.

We are at a standstill on this issue. We don't have any clear direction
to solve it after the initial submission of the changes. 

I suggest moving this to 1.6.

merged to oe-core master

Is this in poky dora? Please paste the commit link.

It is in dora, http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0fa12e44663c9e27de7d2c33be9132127679d0d3

Hello,

Being a high feature, can you please help us with a compatible test scenario?

I cannot find INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE being used in poky or meta-intel. What I do see is INITRD_IMAGE ?= "core-image-minimal-initramfs" being used in image-live.bbclass .

I also tried testing the resulting artifacts with runqemu but encountered issues:

1) runqemu does not recognise the initramfs format:
Error: unknown file arg [tmp/deploy/images/qemux86-64/core-image-minimal-initramfs-qemux86-64.cpio.gz]

2) using INITRAMFS_IMAGE_BUNDLE = "1", the bot process hangs after "mounting media"

Tested with qemux86-64 machine on dora/master:faf8f8660fae26a14578738b642fdeb08bcb9d54

(In reply to comment #13)
> Hello,
> 
> Being a high feature, can you please help us with a compatible test scenario?
> 
> I cannot find INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE being used in poky
> or meta-intel. What I do see is INITRD_IMAGE ?=
> "core-image-minimal-initramfs" being used in image-live.bbclass .
> 
> I also tried testing the resulting artifacts with runqemu but encountered
> issues:
> 
> 1) runqemu does not recognise the initramfs format:
> Error: unknown file arg
> [tmp/deploy/images/qemux86-64/core-image-minimal-initramfs-qemux86-64.cpio.
> gz]
> 
> 2) using INITRAMFS_IMAGE_BUNDLE = "1", the bot process hangs after "mounting
> media"
> 
> Tested with qemux86-64 machine on
> dora/master:faf8f8660fae26a14578738b642fdeb08bcb9d54

Jason W posted the test steps with the series on the list. Have we tried those ?

Hello. I could not find those steps. Can you please direct us to the specific mails that contain them (or their titles).

Thank you!

The real use case behind this feature is a bit more complex in that it is aimed at customized images where the initramfs comes from one recipe and it will pivot root to the real file system.

There is a simple way to test this feature though:

Add the following to local.conf:

INITRAMFS_IMAGE_BUNDLE = "1"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"

Now bitbake core-image-minimal

You can boot the resulting image as follows:

kvm -nographic -kernel tmp-eglibc/deploy/images/qemux86-64/bzImage-initramfs-qemux86-64.bin -append "console=ttyS0,115200 rdinit=/bin/sh"

Next you can add a kernel dependency to the initramfs to test the circular dependency logic.  For this you can add the kernel-module-reiserfs to the core-image-minimal-initramfs.bb.

========== Example ====================

--- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
+++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
@@ -3,7 +3,7 @@ DESCRIPTION = "Small image capable of booting a device. The kernel includes \
 the Minimal RAM-based Initial Root Filesystem (initramfs), which finds the \
 first 'init' program more efficiently."
 
-IMAGE_INSTALL = "initramfs-live-boot initramfs-live-install initramfs-live-install-efi busybox udev base-passwd"
+IMAGE_INSTALL = "initramfs-live-boot initramfs-live-install initramfs-live-install-efi busybox udev base-passwd kernel-module-reiserfs"
 
 # Do not pollute the initrd image with rootfs features
 IMAGE_FEATURES = ""

========== End Example ====================

Now bitbake the core-image-minimal again, and boot it with kvm or qemu.

kvm -nographic -kernel tmp-eglibc/deploy/images/qemux86-64/bzImage-initramfs-qemux86-64.bin -append "console=ttyS0,115200 rdinit=/bin/sh"

# Finally inspect the kernel module is there

/ # find /lib/modules/ |grep reiserfs.ko
/lib/modules/3.10.19-yocto-standard/kernel/fs/reiserfs/reiserfs.ko

---

With this functionality it is possible to generate images which have a kernel and initramfs which containes kernel modules and use it to attach to other file systems or pivot root etc...

I was able to test this with the instructions Jason gave. I used qemu to test it instead of kvm.

One more thing needs to be taken care of: documentation.

Scott, is there documentation for this in the manual? All the documentation I was able to find is in the poky sources, inside meta-yocto/conf/local.conf.sample.extended . Is this enough or do we need to create a manual entry?

For clarity I pulled in the clip from the oe-core's:
meta/conf/local.conf.sample.extended

---------- start ------------
#
# Kernel image features
#
# The INITRAMFS_IMAGE image variable will cause an additional recipe to
# be built as a dependency to the what ever rootfs recipe you might be
# using such as core-image-sato.  The initramfs might be needed for
# the initial boot of of the target system such as to load kernel
# modules prior to mounting the root file system.
#
# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
# specified by the INITRAMFS_IMAGE will be run through an extra pass
# through the kernel compilation in order to build a single binary
# which contains both the kernel image and the initramfs.  The
# combined binary will be deposited into the tmp/deploy directory.
# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
#       INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
#
#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
#INITRAMFS_IMAGE_BUNDLE = "1"
---------- end ------------

If something more is needed or you have questions, please let me know.

Right now we document only the INITRAMFS_FSTYPES variable in our reference manual.  The fix described in this bug seems to also involve the INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE variables.  So, I need some descriptions of these variables in the context of how a user would use them (not just for this solution, but for the variables in general).  

Another variable we probably should create an entry for is CONFIG_INITRAMFS_SOURCE.  So, I need some information for that.  

Adding variable entries for INITRAMFS_IMAGE_BUNDLE, CONFIG_INITRAMFS_SOURCE and INITRAMFS_IMAGE would be part of the documentation change we need to fully resolve this bug.

I put in an explanation of the circular dependency problem and how to get around it through use of the variable(s) we add.  If that is sufficient, we don't need to create any separate usage sections.  However, if we feel we need a whole separate section on creating initram type stuff... then we need more.

Comments?

Thanks,
Scott

Scott,

Your plan is fine with me. Just the basic documentation updates you are
describing are good enough.

A full section about build, deployment and boot would be appropriate 
(if they don't already exist), but with upcoming changes to deployment
in Yocto 1.6, it would be better to wait until then for a bigger doc
update.

Bruce

(In reply to comment #19)
> Right now we document only the INITRAMFS_FSTYPES variable in our reference
> manual.  The fix described in this bug seems to also involve the
> INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE variables.  So, I need some
> descriptions of these variables in the context of how a user would use them
> (not just for this solution, but for the variables in general).  
> 
> Another variable we probably should create an entry for is
> CONFIG_INITRAMFS_SOURCE.  So, I need some information for that.  
> 
> Adding variable entries for INITRAMFS_IMAGE_BUNDLE, CONFIG_INITRAMFS_SOURCE
> and INITRAMFS_IMAGE would be part of the documentation change we need to
> fully resolve this bug.
> 
> I put in an explanation of the circular dependency problem and how to get
> around it through use of the variable(s) we add.  If that is sufficient, we
> don't need to create any separate usage sections.  However, if we feel we
> need a whole separate section on creating initram type stuff... then we need
> more.
> 
> Comments?
> 
> Thanks,
> Scott

Hi 

I have added two new variable descriptions to the ref-manual and they need to be looked at.  I took some guesses on how to describe these based on 'git grep' results I go inside poky.  Here are the links:

http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-INITRAMFS_IMAGE

http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-INITRAMFS_IMAGE_BUNDLE

Regarding the CONFIG_INITRAMFS_SOURCE variable.  I see that it is used in the kernel.bbclass file but I don't understand it.  Comment #5 below from Jason says that it simply points to the generated initramfs from the additional recipe.  Is that it?  Is this all automated so that if a user includes the kernel.bbclass and then sets INITRAMFS_IMAGE and specifies INITRAMFS_IMAGE_BUNDLE the CONFIG_INITRAMFS_SOURCE variable is automatically handled without any user intervention?

Note that I have not mentioned the circular dependencies issue at all yet in the INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE variable descriptions.  I need to add that in still.  I would like to get the basic descriptions of these variables nailed down first though. 

thanks,
Scott

Can we have an update on the documentation status? Is the current documentation enough? Also, is it correct?

Before we can close the doc portion of this, I need the following:

* A description of the CONFIG_INITRAMFS_SOURCE variable.

* An explanation describing what steps the user needs to take to avoid the circular dependency.  Presumably, the user is going to set some magic combination of the CONFIG_INITRAMFS_SOURCE, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE variables?

I can't close this doc part until I have this information. 

Thanks, 
Scott

(In reply to comment #23)
> Before we can close the doc portion of this, I need the following:
> 
> * A description of the CONFIG_INITRAMFS_SOURCE variable.

This is the kernel Kconfig variable, passed via the build system as an
environment variable to the kernel build. The details here are a cut and
paste from the kernel Kconfig file:

config INITRAMFS_SOURCE
        string "Initramfs source file(s)"
        default ""
        help
          This can be either a single cpio archive with a .cpio suffix or a
          space-separated list of directories and files for building the
          initramfs image.  A cpio archive should contain a filesystem archive
          to be used as an initramfs image.  Directories should contain a
          filesystem layout to be included in the initramfs image.  Files
          should contain entries according to the format described by the
          "usr/gen_init_cpio" program in the kernel tree.

          When multiple directories and files are specified then the
          initramfs image will be the aggregate of all of them.

> 
> * An explanation describing what steps the user needs to take to avoid the
> circular dependency.  Presumably, the user is going to set some magic
> combination of the CONFIG_INITRAMFS_SOURCE, INITRAMFS_IMAGE and
> INITRAMFS_IMAGE_BUNDLE variables?

They don't need to set the initramfs source, from the comments in the
code is the rest of the answer:

+#
+# Kernel image features
+#
+# The INITRAMFS_IMAGE image variable will cause an additional recipe to
+# be built as a dependency to the what ever rootfs recipe you might be
+# using such as core-image-sato.  The initramfs might be needed for
+# the initial boot of of the target system such as to load kernel
+# modules prior to mounting the root file system.
+#
+# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
+# specified by the INITRAMFS_IMAGE will be run through an extra pass
+# through the kernel compilation in order to build a single binary
+# which contains both the kernel image and the initramfs.  The
+# combined binary will be deposited into the tmp/deploy directory.
+# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
+#       INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
+#
+#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
+#INITRAMFS_IMAGE_BUNDLE = "1"


> 
> I can't close this doc part until I have this information. 
> 
> Thanks, 
> Scott

Hi, 

I have made some additional documentation changes to hopefully close this bug out.  I added the variable entry in the glossary for CONFIG_INITRAMFS_SOURCE.  Please look this over at http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-CONFIG_INITRAMFS_SOURCE.  It basically says that this identifies the initramfs source.

The other change was to the INITRAMFS_IMAGE_BUNDLE variable.  I added some to the first paragraph of this variable explaining how it can be used to avoid the circular dependency issue.  Please look it over at http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-INITRAMFS_IMAGE_BUNDLE.

It seems to me that this should do it for the docs.  I could be wrong though so comment as needed.

Thanks, 
Scott

The doc updates look fine to me.

Bruce

(In reply to comment #25)
> Hi, 
> 
> I have made some additional documentation changes to hopefully close this
> bug out.  I added the variable entry in the glossary for
> CONFIG_INITRAMFS_SOURCE.  Please look this over at
> http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-
> CONFIG_INITRAMFS_SOURCE.  It basically says that this identifies the
> initramfs source.
> 
> The other change was to the INITRAMFS_IMAGE_BUNDLE variable.  I added some
> to the first paragraph of this variable explaining how it can be used to
> avoid the circular dependency issue.  Please look it over at
> http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-
> INITRAMFS_IMAGE_BUNDLE.
> 
> It seems to me that this should do it for the docs.  I could be wrong though
> so comment as needed.
> 
> Thanks, 
> Scott

Thanks Bruce, 

I am resetting the doc flag on this bug to "Done".

Scott

Thank you all for the work on the feature and its documentation! Setting to VERIFIED.