Bug 6330

Summary:	Add support for cpio - 'xz' and 'gz' and 'bz2' compressed u-boot image-types
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Mark Hatle <mark.hatle>
Component:	deployment	Assignee:	Hongxu Jia <hongxu.jia>
Status:	VERIFIED FIXED	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	hongxu.jia, liezhi.yang, sgw
Version:	1.7
Target Milestone:	1.7 M2
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Yes (doc changes required)

Description Mark Hatle 2014-05-14 20:06:45 UTC

Extend the image_types_uboot class to produce CPIO archives, as well as compressing them with xz (lzma), gz and bz2.

It may make sense to add 'xz' generically as well, since that should be replacing the lzma extension.

Comment 1 Hongxu Jia 2014-07-11 05:07:23 UTC

The oe-core has already implemented to support for ext2.lzma.u-boot/
ext3.gz.u-boot/ cpio.lzma.u-boot/ cpio.gz.u-boot and cpio.bz2.u-boot
type compressed uboot rootfs images.

It didn't support the cpio.xz.u-boot type for now, but use the cpio.lamz.u-boot
to instead.

Comment 2 Hongxu Jia 2014-07-11 05:08:16 UTC

1. How to config in Yocto
...
MACHINE ?= "beaglebone"
IMAGE_CLASSES ?= "image_types_uboot"
IMAGE_FSTYPES += " cpio.lzma.u-boot cpio.gz.u-boot cpio.bz2.u-boot"
...

2. bitbake core-image-minimal

3. Runtime test on beaglebone borad, load ramdisk cpio.gz.u-boot and boot it

U-Boot# setenv serverip 128.224.162.194;
U-Boot# setenv ipaddr 128.224.163.245;
U-Boot# tftp ${loadaddr} uImage-beaglebone.bin;
U-Boot# tftp ${rdaddr}  core-image-minimal-beaglebone-20140710121908.rootfs.cpio.gz.u-boot;
U-Boot# tftp ${fdtaddr} uImage-am335x-bone.dtb
U-Boot# setenv bootargs 'console=ttyO0,115200n8 root=/dev/ram0 rw ramdisk_size=65536 initrd=${rdaddr},64M';
U-Boot# bootm ${loadaddr} ${rdaddr} ${fdtaddr};

...
## Booting kernel from Legacy Image at 82000000 ...
   Image Name:   Linux-3.14.5-yocto-standard
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5045408 Bytes = 4.8 MiB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    2509178 Bytes = 2.4 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Loading Kernel Image ... OK
   Loading Ramdisk to 8fd9b000, end 8ffff97a ... OK
   Loading Device Tree to 8fd91000, end 8fd9afb3 ... OK
...
The linux system boot up successfully.

Comment 3 Hongxu Jia 2014-07-11 07:34:37 UTC

BTW: We need to config kernel to support gzip/bzip2/LZMA ramdisk decompress: 
...
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
...

How to config in yocto (A tricky way)
1. bitbake linux-yocto -ccleansstate && bitbake linux-yocto -cmenuconfig

2. On menuconfig
 General setup-->Initial RAM filesystem and RAM disk (initramfs/initrd) support
 --> Support initial ramdisks compressed using gzip/bzip2/LZMA

3. bitbake linux-yocto

Comment 4 Hongxu Jia 2014-07-14 07:50:13 UTC

Currently oe-core has supported for building cpio.lzma.u-boot/
cpio.gz.u-boot/cpio.bz2.u-boot, but it is lacking support for
building *cpio.xz.u-boot* type images.

The reason is u-boot didn't support xz compress/decompress,
but I found two patches from upstream to support that.
http://lists.denx.de/pipermail/u-boot/2010-December/083155.html
http://lists.denx.de/pipermail/u-boot/2010-December/083154.html

But the upstream has rejected it, the reason is LZMA and XZ
overlap on the internal LZMA code (each one has the own
implementation), there is just toolittle time for the
cleanup, so they decide to deprecate LZMA and just support XZ
*in the future*.
http://lists.denx.de/pipermail/u-boot/2010-December/083231.html

Comment 5 Mark Hatle 2014-07-14 19:49:12 UTC

I haven't looked at the latest version of the u-boot settings, but we should follow whatever u-boot supports.. They they do not support 'XZ' (those emails were from 2010, so I'm curious if things have changed) then we should stick with the supported configurations.

Comment 6 Hongxu Jia 2014-07-15 05:53:09 UTC

Got it, and I will do the test to load the u-boot ramdisk image to a beaglebone (arm).

Comment 7 Hongxu Jia 2014-07-15 05:54:41 UTC

Verifie on OE-Core rev: fa325e44f5b429b4038022b31285af9c94672943

Steps:

1. How to config in Yocto
...
MACHINE ?= "beaglebone"
IMAGE_CLASSES ?= "image_types_uboot"
IMAGE_FSTYPES += "ext2.u-boot ext2.gz.u-boot ext2.bz2.u-boot ext2.lzma.u-boot"
IMAGE_FSTYPES += "ext3.u-boot ext3.gz.u-boot ext3.bz2.u-boot ext3.lzma.u-boot"
IMAGE_FSTYPES += "ext4.u-boot ext4.gz.u-boot ext4.bz2.u-boot ext4.lzma.u-boot"
IMAGE_FSTYPES += "cpio.u-boot cpio.gz.u-boot cpio.bz2.u-boot cpio.lzma.u-boot"
...

2. we need to config kernel to support gzip/bzip2/LZMA ramdisk decompress: 
...
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
...

How to config in yocto (A tricky way)
  1) bitbake linux-yocto -ccleansstate && bitbake linux-yocto -cmenuconfig

  2) while menuconfig
    General setup-->Initial RAM filesystem and RAM disk (initramfs/initrd) support
     --> Support initial ramdisks compressed using gzip/bzip2/LZMA

  3) bitbake linux-yocto

3. bitbake core-image-minimal

4. start a tftp server and config the image download directory

5. Runtime test on beaglebone borad, load ramdisk
    {cpio,ext2,ext3,ext4}.{gz.u-boot,bz2.u-boot,lzma.u-boot,u-boot}
    and boot it

U-Boot# setenv serverip 128.224.162.194;
U-Boot# setenv ipaddr 128.224.163.245;
U-Boot# tftp ${loadaddr} uImage-beaglebone.bin;
U-Boot# tftp ${rdaddr}  core-image-minimal-beaglebone-20140710121908.rootfs.cpio.gz.u-boot;
U-Boot# tftp ${fdtaddr} uImage-am335x-bone.dtb
U-Boot# setenv bootargs 'console=ttyO0,115200n8 root=/dev/ram0 rw ramdisk_size=65536 initrd=${rdaddr},64M';
U-Boot# bootm ${loadaddr} ${rdaddr} ${fdtaddr};


Testing, Expected Results:
--------------------------
cpio.bz2.u-boot:
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (bzip2 compressed)
   Data Size:    2397852 Bytes = 2.3 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

cpio.gz.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    2509434 Bytes = 2.4 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

cpio.lzma.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (lzma compressed)
   Data Size:    1689519 Bytes = 1.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK

...

cpio.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    5109760 Bytes = 4.9 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext2.bz2.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (bzip2 compressed)
   Data Size:    2390181 Bytes = 2.3 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext2.gz.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    2595868 Bytes = 2.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext2.lzma.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (lzma compressed)
   Data Size:    1707162 Bytes = 1.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext2.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    67108864 Bytes = 64 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext3.bz2.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (bzip2 compressed)
   Data Size:    2400152 Bytes = 2.3 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext3.gz.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    2604122 Bytes = 2.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext3.lzma.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (lzma compressed)
   Data Size:    1709109 Bytes = 1.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...



ext3.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    67108864 Bytes = 64 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext4.bz2.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (bzip2 compressed)
   Data Size:    2378577 Bytes = 2.3 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

gz.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    2583845 Bytes = 2.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...


ext4.lzma.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (lzma compressed)
   Data Size:    1702801 Bytes = 1.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

ext4.u-boot
...
## Loading init Ramdisk from Legacy Image at 88080000 ...
   Image Name:   core-image-minimal-beaglebone-20
   Image Type:   ARM Linux RAMDisk Image (uncompressed)
   Data Size:    67108864 Bytes = 64 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
...

The linux system boot up successfully.