Bug 2075

Summary: bitbake: replace older functions with the subprocess module
Product: [Build System, Metadata & Runtime] BitBake Reporter: Robert Yang <liezhi.yang>
Component: bitbakeAssignee: Robert Yang <liezhi.yang>
Status: VERIFIED FIXED QA Contact:
Severity: enhancement    
Priority: Low CC: laurentiu.serban, liang.li2, poky.bs.watcher, poky.watcher, sgw, song.liu, yi.zhao, zhenfeng.zhao
Version: 1.2   
Target Milestone: 1.3 M1   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=2454
Whiteboard: (P2)(No QA needed)(Patch Review) 20%
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Robert Yang 2012-03-13 01:26:37 UTC 
The os.popen() fail silently if the executed program cannot be found, so the build would go on when some files can not be found. The os.popen() has been deprecated since python version 2.6, use the subprocess module to replace it, the following functions should be
replaced:

os.system
os.spawn*
os.popen*
popen2.*
commands.*

More information:

http://docs.python.org/library/subprocess.html#subprocess-replacements 

And Richard said:

I'm broadly in favour of this but I have noticed some performance
degradation in the one case I tried converting things. I'd therefore
like to see performance tests of the changes to ensure no regressions
occur. This change is also more 1.3 material at this point. Other than
that, I agree its a good idea though
Comment 1 Robert Yang 2012-05-09 08:20:52 UTC 
Design Doc:
1) Replace the os.* for bitbake and metadata under the instructions from:
   http://docs.python.org/library/subprocess.html#subprocess-replacements

2) Do the performance testing to see whether it has performance issue:
   $ time bitbake core-image-sato 
   # Compare the time between with/without the replacing
   # All the sources should be on the local disk

3) If the step 2) is OK, file another feature bug for the metadata part and send
   the pull requests separately (oe-core and bitbake-devel)
Comment 2 Robert Yang 2012-05-14 08:12:25 UTC 
I've done the testing to make sure that it doesn't cause the performance problems:

# The sources are on local disk

1) Before applied the pacthes to bitbake and oe-core:
$ time bitbake core-image-sato
real    177m50.723s
user    436m1.551s
sys     71m29.588s

2) After applied the pathes:
$ time bitbake core-image-sato
real    176m26.194s
user    436m7.931s
sys     71m1.994s

We can see that after applied these patches, the time has reduced 84 seconds,
this is very slight, I think that we can assume this is just a deviation, it
doesn't cause performance problems.

I've sent the patches for bitbake to bitbake-devel mailing list, and will file another
feature to oe-core and send patches for it after this has been merged to bitbake.
Comment 4 Laurentiu Serban 2012-06-29 13:32:12 UTC 
Verified