Bug 12149 - Racing lz4 u-boot CONVERSION_CMDs in do_image_cpio
Summary: Racing lz4 u-boot CONVERSION_CMDs in do_image_cpio
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.3.2
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.3.4
Assignee: Maxin B. John
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-10-02 09:12 UTC by Niko Mauno
Modified: 2018-03-20 04:06 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Niko Mauno 2017-10-02 09:12:58 UTC 
- In yocto-2.3.2 to create initramfs image with IMAGE_FSTYPES = "cpio.lz4.u-boot" resulted in 
    
    ERROR: When reparsing .../images/my_image_recipe.bb.do_image_cpio, the
    basehash value changed from f58c1df3d5d9c4828fa93897760378a2 to
    4b41270a04a07df4476ef4f6767958ac. The metadata is not deterministic
    and this needs to be fixed.

 - By resorting to post-yocto-2.3.2 commit: <https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=pyro&id=717303e6fbcbbe181ad9645d762eb5a85d934523>, above error was encountered no longer -- however, it still results in non-functional image artifact. More specifically, during target device boot-up process, when it's time for kernel to mount initramfs rootfs, the following line is printed in console:

    RAMDISK: Couldn't find valid RAM disk image starting at 0.

 - Looking into the issue, I noticed that in yocto-2.3.2, the "non-legacy" command pair: "CONVERSION_CMD_lz4 ; CONVERSION_CMD_u-boot" was executed between bitbake operations occasionally before the "legacy" command: "CONVERSION_CMD_lz4.u-boot" and occasionally after in do_image_cpio, apparently resulting in non-deterministic metadata error. ("non-legacy" and "legacy" here refer to same set of commands with the exception that in former, lz4's -l switch is omitted, and in latter, -l is included.)

   After commit 717303e6fbcbbe181ad9645d762eb5a85d934523, the "legacy" command is executed consistently before "non-legacy" command, resulting in image unusable by Linux kernel. Apparently for lz4 compressed initramfs generation, only legacy lz4 format should be used (ie. CONVERSION_CMD_lz4.u-boot, which internally resorts to CONVERSION_CMD_lz4_legacy). 

   The generated initramfs artifact becomes bootable/usable by removing superfluous "non-legacy" initramfs generation from do_image_cpio function by following override:

    CONVERSION_CMD_lz4 = " "
    CONVERSION_CMD_u-boot = " "
Comment 1 Armin Kuster 2017-10-05 15:15:40 UTC 
image_type_uboot.bbclass removed in
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/classes?id=e18cec750b51267d9c130b395c7a53585f4ae7ac

so master may not be affected?
Comment 2 brian avery 2017-10-06 00:00:59 UTC 
Question:
I have a built cpio.lz4.u-boot for beaglebone black to check and see if this issue also occurs on master.

Are there instructions anywhere for how to set up the beaglebone using the compressed rootfs (aka cpio.lz4.u-boot) ?  

Other ways to verify welcome as well.
Comment 3 Niko Mauno 2017-10-31 08:44:26 UTC 
In yocto-2.4 it seems that (at least for me) issues still remain with IMAGE_FSTYPES = "cpio.lz4.u-boot", ie. generated image results in "RAMDISK: Couldn't find valid RAM disk image starting at 0.". Apparently non-legacy CONVERSION_CMD_lz4 is used by default. A working image can be generated by issuing following override:

CONVERSION_CMD_lz4 = "${CONVERSION_CMD_lz4_legacy}"
Comment 4 Ross Burton 2018-01-04 16:30:54 UTC 
A reminder that bug 12461 is related.
Comment 5 Armin Kuster 2018-01-06 06:31:17 UTC 
updates SCR_URI
Comment 6 Armin Kuster 2018-01-06 06:37:13 UTC 
(In reply to comment #5)
> updates SCR_URI

sorry. wrong bug. this auto next feature is annoying