Bug 7341

Summary: Performance issue in do_rootfs (mkfs.ext*)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Richard Purdie <richard.purdie>
Component: coreAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: meta.mr.watcher, meta.watcher, ross.burton
Version: 1.8   
Target Milestone: 1.8 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Richard Purdie 2015-02-20 13:08:53 UTC
See bug 7339 for debug details, we're spending over eight minutes in the mkfs.ext3 code for a rootfs which only took us less than four minutes to create in the first place. This can clearly be demonstrated by:

diff --git a/meta/lib/oe/image.py b/meta/lib/oe/image.py
index 0ce303d..ba5e8ff 100644
--- a/meta/lib/oe/image.py
+++ b/meta/lib/oe/image.py
@@ -10,12 +10,19 @@ def generate_image(arg):
     bb.note("Running image creation script for %s: %s ..." %
             (type, create_img_cmd))
 
+
+    bb.warn("Running image creation script for %s: %s ..." %
+            (type, create_img_cmd))
+
     try:
         subprocess.check_output(create_img_cmd, stderr=subprocess.STDOUT)
     except subprocess.CalledProcessError as e:
         return("Error: The image creation script '%s' returned %d:\n%s" %
                (e.cmd, e.returncode, e.output))
 
+    bb.warn("Completed image creation script for %s: %s ..." %
+            (type, create_img_cmd))
+
     return None
 
which then shows something like:

Thu Feb 19 22:37:00 UTC 2015: WARNING: Finished Executing prelink_image ...
Thu Feb 19 22:37:08 UTC 2015: WARNING: Running image creation script for ext3: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato-sdk/1.0-r0/temp/create_image.ext3 ...
Thu Feb 19 22:37:08 UTC 2015: WARNING: Running image creation script for tar: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato-sdk/1.0-r0/temp/create_image.tar ...
Thu Feb 19 22:40:30 UTC 2015: WARNING: Completed image creation script for tar: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato-sdk/1.0-r0/temp/create_image.tar ...
Thu Feb 19 22:46:20 UTC 2015: WARNING: Completed image creation script for ext3: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato-sdk/1.0-r0/temp/create_image.ext3 ...
Thu Feb 19 22:46:20 UTC 2015: WARNING: Executing buildhistory_get_imageinfo ...
Thu Feb 19 22:46:34 UTC 2015: WARNING: Finished Executing buildhistory_get_imageinfo ...
Thu Feb 19 22:46:38 UTC 2015: NOTE: recipe core-image-sato-sdk-1.0-r0: task do_rootfs: Succeeded
Thu Feb 19 22:46:38 UTC 2015: NOTE: Tasks Summary: Attempted 4651 tasks of which 4650 didn't need to be rerun and all succeeded.
Thu Feb 19 22:46:38 UTC 2015: NOTE: Writing buildhistory

I'm hoping we can find some performance tweaks to the filesystem population within mkfs.
Comment 1 Ross Burton 2015-02-20 13:14:42 UTC
*** Bug 6977 has been marked as a duplicate of this bug. ***
Comment 2 Richard Purdie 2015-02-20 13:32:07 UTC
I did strace the mkfs command and it seems to be seeking all over the output file which seems to be one of the main issues. Removing pseudo from the equation did not help at all performance wise.
Comment 3 Robert Yang 2015-02-21 01:42:30 UTC
I think that RP has made a patch for it, so re-assign it to RP.