Bug 3848

1.3 M5 has passed....

Hi Darren,

Here are the initial design, please help me to take a look at it and feel free to give your comments.

* Upgrade the e2fsprogs to the update to date version, I'd like to use the git repo if you are fine with it, it is e2fsprogs-1.42.6.tar.gz in yocto currently, which doesn't have the "symlink" commmand in debugfs shell.

* Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass:
  - I will go on working on your mkdebugfs.sh, I'd like to add it to the e2fsprogs package in yocto, and install it to sysroot/usr/bin/mkdebugfs.sh

  - When the mkdebugfs.sh is ready, the new steps to generate the root filesystem would be:
    -> use "dd" command to make rootfs.extX file
    -> run "mke2fs -t X rootfs.extX" to create the filesystem
    -> run the mkdebugfs.sh to generate the root filesystem

This looks right to me. Once complete and shown to be working, the next step (separate bug/task) will be to refactor debugfs and move some code into libext2fs and enable mke2fs to prepopulate with an initial directory. We would then need to update yocto again for that. I see these as necessary incremental steps.

(In reply to comment #0)
> Created attachment 1060 [details]
> example debugfs script
> 
> Now that symlink has been merged into upstream into e2fsprogs, we can
> prototype proper ext4 filesystem generation using a debugfs script and
> pseudo. This will take the place of genext2fs and serve as the proof of
> concept before we take the next step and incorporate the initial directory
> support into mke2fs directly.
> 
> The attached debugfs script is a start at what will be required to complete
> the task. It will need to be modified to make the stat calls on each file
> and update the destination inodes accordingly. Pseudo will be needed to

Hi Darren,

I've tried a core-image-minimal rootfs generation, it works well, but would you please give me more information about this:

"""
It will need to be modified to make the stat calls on each file
and update the destination inodes accordingly.
"""

What information should we update the inode, please? Would you please give me an example ?

The patch will be done sooner.

// Robert

> provide the user/permission mapping to the files on disk.

You need to use stat (which pseudo intercepts) to get the user, group, and permissions of each file. Pseudo will look them up in the database and return the appropriate values in the statbuf. You then pass these values to debugfs with the sif command.

Hi Darren,

I've finished the patches and pushed them to:

  git://git.pokylinux.org/poky-contrib robert/e2fsprogs
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/e2fsprogs

Robert Yang (7):
  e2fsprogs: upgrade to the git version (rename only)
  e2fsprogs: upgrade to the git version
  e2fsprogs: add the original mkdebugfs.sh
  mkdebugfs.sh: convert the tab to 4 spaces
  mkdebugfs.sh: several fixes
  e2fsprogs: ship mkfsdebug.sh
  image_types.bbclass: replace genext2fs with mkdebugfs.sh

It would be great if you can help to take a look at them, or I will send them to the oe-core mailing list later after enough testing.

NOTE:


1) These patches would cause the rootfs generation a little slower:

* For a core-image-minimal generation (IMAGE_FSTYPES="tar.bz2 ext3"):

Before:
$ time bitbake core-image-minimal

real    1m10.823s
user    0m37.108s
sys     0m15.894s

After:
$ time bitbake core-image-minimal

real    1m17.501s
user    0m29.304s
sys     0m20.731s

7 seconds lost.

* For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

Before:
$ time bitbake core-image-sato

real    11m10.645s
user    2m43.503s
sys     1m1.589s

After:
$ time bitbake core-image-sato

real    11m53.131s
user    3m18.988s
sys     2m8.350s

43 seconds lost.

2) The image size are the same by "ls -h", but different by "du -sh": (core-image-sato)

$ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
357M BEFORE.rootfs.ext3
357M AFTER.rootfs.ext3

$ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
238M    BEFORE.rootfs.ext3
357M    AFTER.rootfs.ext3

It seems that the genext2fs has done several optimization during the rootfs generation.

The patch has been sent to oe-core mailing list:

http://lists.linuxtogo.org/pipermail/openembedded-core/2013-February/036290.html

Move it to 1.5

Merged:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f1e6091a1b20f66ede7098e54a2974232943c1c7
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a758b9302145d0a5b8f6b0e7d276b521ccd49061
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2cece576b351228cb24bf24a53845646a51f7f6b
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1b214059be350a3abfc57140cbad00f5bab437bf
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=39d80f6d4ff56e13a621d5f3b9e2b2348efdf3d6
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=6a78e22aaede80cd4e0bcbc06b878574766d7b7c

set to verified.