3848 2013-02-05 23:08:18 +0000 Create ext* filesystems using libext2fs and debugfs 2013-09-09 08:10:16 +0000 1 1 7 Build System, Metadata & Runtime OE-Core deployment unspecified x86 Multiple VERIFIED FIXED Medium+ enhancement 1.5 M4 4640 1 dvhart liezhi.yang bruce.ashfield mark.hatle mihai.lindner richard.purdie sgw tom.zanussi zhenfeng.zhao --- --- No (bug/feature does not impact docs) oldest_to_newest 29549 0 1060 dvhart 2013-02-05 23:08:18 +0000 Created attachment 1060 example debugfs script Now that symlink has been merged into upstream into e2fsprogs, we can prototype proper ext4 filesystem generation using a debugfs script and pseudo. This will take the place of genext2fs and serve as the proof of concept before we take the next step and incorporate the initial directory support into mke2fs directly. The attached debugfs script is a start at what will be required to complete the task. It will need to be modified to make the stat calls on each file and update the destination inodes accordingly. Pseudo will be needed to provide the user/permission mapping to the files on disk. 29591 1 dvhart 2013-02-06 17:12:30 +0000 1.3 M5 has passed.... 29980 2 liezhi.yang 2013-02-20 09:08:57 +0000 Hi Darren, Here are the initial design, please help me to take a look at it and feel free to give your comments. * Upgrade the e2fsprogs to the update to date version, I'd like to use the git repo if you are fine with it, it is e2fsprogs-1.42.6.tar.gz in yocto currently, which doesn't have the "symlink" commmand in debugfs shell. * Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass: - I will go on working on your mkdebugfs.sh, I'd like to add it to the e2fsprogs package in yocto, and install it to sysroot/usr/bin/mkdebugfs.sh - When the mkdebugfs.sh is ready, the new steps to generate the root filesystem would be: -> use "dd" command to make rootfs.extX file -> run "mke2fs -t X rootfs.extX" to create the filesystem -> run the mkdebugfs.sh to generate the root filesystem 29994 3 dvhart 2013-02-20 16:00:18 +0000 This looks right to me. Once complete and shown to be working, the next step (separate bug/task) will be to refactor debugfs and move some code into libext2fs and enable mke2fs to prepopulate with an initial directory. We would then need to update yocto again for that. I see these as necessary incremental steps. 30025 4 liezhi.yang 2013-02-22 09:37:37 +0000 (In reply to comment #0) > Created attachment 1060 [details] > example debugfs script > > Now that symlink has been merged into upstream into e2fsprogs, we can > prototype proper ext4 filesystem generation using a debugfs script and > pseudo. This will take the place of genext2fs and serve as the proof of > concept before we take the next step and incorporate the initial directory > support into mke2fs directly. > > The attached debugfs script is a start at what will be required to complete > the task. It will need to be modified to make the stat calls on each file > and update the destination inodes accordingly. Pseudo will be needed to Hi Darren, I've tried a core-image-minimal rootfs generation, it works well, but would you please give me more information about this: """ It will need to be modified to make the stat calls on each file and update the destination inodes accordingly. """ What information should we update the inode, please? Would you please give me an example ? The patch will be done sooner. // Robert > provide the user/permission mapping to the files on disk. 30045 5 dvhart 2013-02-22 16:52:24 +0000 You need to use stat (which pseudo intercepts) to get the user, group, and permissions of each file. Pseudo will look them up in the database and return the appropriate values in the statbuf. You then pass these values to debugfs with the sif command. 30080 6 liezhi.yang 2013-02-25 16:11:04 +0000 Hi Darren, I've finished the patches and pushed them to: git://git.pokylinux.org/poky-contrib robert/e2fsprogs http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/e2fsprogs Robert Yang (7): e2fsprogs: upgrade to the git version (rename only) e2fsprogs: upgrade to the git version e2fsprogs: add the original mkdebugfs.sh mkdebugfs.sh: convert the tab to 4 spaces mkdebugfs.sh: several fixes e2fsprogs: ship mkfsdebug.sh image_types.bbclass: replace genext2fs with mkdebugfs.sh It would be great if you can help to take a look at them, or I will send them to the oe-core mailing list later after enough testing. NOTE: 1) These patches would cause the rootfs generation a little slower: * For a core-image-minimal generation (IMAGE_FSTYPES="tar.bz2 ext3"): Before: $ time bitbake core-image-minimal real 1m10.823s user 0m37.108s sys 0m15.894s After: $ time bitbake core-image-minimal real 1m17.501s user 0m29.304s sys 0m20.731s 7 seconds lost. * For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"): Before: $ time bitbake core-image-sato real 11m10.645s user 2m43.503s sys 1m1.589s After: $ time bitbake core-image-sato real 11m53.131s user 3m18.988s sys 2m8.350s 43 seconds lost. 2) The image size are the same by "ls -h", but different by "du -sh": (core-image-sato) $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}' 357M BEFORE.rootfs.ext3 357M AFTER.rootfs.ext3 $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 238M BEFORE.rootfs.ext3 357M AFTER.rootfs.ext3 It seems that the genext2fs has done several optimization during the rootfs generation. 30107 7 liezhi.yang 2013-02-26 10:11:33 +0000 The patch has been sent to oe-core mailing list: http://lists.linuxtogo.org/pipermail/openembedded-core/2013-February/036290.html 32196 8 liezhi.yang 2013-04-16 06:06:24 +0000 Move it to 1.5 35940 9 liezhi.yang 2013-08-27 04:36:03 +0000 Merged: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f1e6091a1b20f66ede7098e54a2974232943c1c7 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a758b9302145d0a5b8f6b0e7d276b521ccd49061 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2cece576b351228cb24bf24a53845646a51f7f6b http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1b214059be350a3abfc57140cbad00f5bab437bf http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=39d80f6d4ff56e13a621d5f3b9e2b2348efdf3d6 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=6a78e22aaede80cd4e0bcbc06b878574766d7b7c 36411 10 mihai.lindner 2013-09-09 08:10:16 +0000 set to verified. 1060 2013-02-05 23:08:18 +0000 2013-02-05 23:08:18 +0000 example debugfs script mkdebugfs.sh application/x-sh 879 dvhart IyEvYmluL3NoClNSQ0RJUj0kMQpERVZJQ0U9JDIKREVCVUdGUz0iZGVidWdmcy9kZWJ1Z2ZzIgoK eWVzIHwgbWtmcy5leHQ0IHRlc3QuaW1nCgp7CglDV0Q9Ii8iCglmaW5kICRTUkNESVIgfCB3aGls ZSByZWFkIEZJTEU7IGRvCgkJI1RHVD0ke0ZJTEUjJFNSQ0RJUn0KCQlUR1Q9JChiYXNlbmFtZSAi JHtGSUxFIyRTUkNESVJ9IikKCQlESVI9JChkaXJuYW1lICIke0ZJTEUjJFNSQ0RJUn0iKQoKCQkj IFNraXAgdGhlIHJvb3QgZGlyCgkJaWYgWyAteiAiJFRHVCIgXTsgdGhlbgoJCQljb250aW51ZQoJ CWZpCgoJCWlmIFsgIiRESVIiICE9ICIkQ1dEIiBdOyB0aGVuCgkJCWVjaG8gImNkICRESVIiCgkJ CUNXRD0iJERJUiIKCQlmaQoKCQljYXNlICQoc3RhdCAtYyAiJUYiICRGSUxFKSBpbgoJCSJkaXJl Y3RvcnkiKQoJCQllY2hvICJta2RpciAkVEdUIgoJCQk7OwoJCSJyZWd1bGFyIGZpbGUiKQoJCQll Y2hvICJ3cml0ZSAkRklMRSAkVEdUIgoJCQk7OwoJCSJzeW1ib2xpYyBsaW5rIikKCQkJTElOS19U R1Q9JChscyAtbCAkRklMRSB8IHNlZCAtZSAncy8uKi0+IC8vJykKCQkJZWNobyAic3ltbGluayAk VEdUICRMSU5LX1RHVCIKCQkJOzsKCQkiYmxvY2sgc3BlY2lhbCBmaWxlIikKCQkJREVWTk89JChz dGF0IC1jICIldCAlVCIgJEZJTEUpCgkJCWVjaG8gIm1rbm9kICRUR1QgYiAkREVWTk8iCgkJCTs7 CgkJImNoYXJhY3RlciBzcGVjaWFsIGZpbGUiKQoJCQlERVZOTz0kKHN0YXQgLWMgIiV0ICVUIiAk RklMRSkKCQkJZWNobyAibWtub2QgJFRHVCBjICRERVZOTyIKCQkJOzsKCQkqKQoJCQllY2hvICJV bmtub3duIGZpbGUgJEZJTEUiIDE+JjIKCQkJOzsKCQllc2FjCglkb25lCn0gfCAkREVCVUdGUyAt dyAtZiAvZGV2L3N0ZGluICRERVZJQ0UK