Bug 12907

Summary: Python3 builds have slowed down significantly for non-x86 targets
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Khem Raj <raj.khem>
Component: devtools / tool chainAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: High CC: meta.mr.watcher, meta.watcher, richard.purdie
Version: 2.6   
Target Milestone: 2.6 M3   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Khem Raj 2018-08-30 18:14:22 UTC 
Since commit

http://git.openembedded.org/openembedded-core/commit/?id=05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b

We are enabling PGO for generating optimized python3 in a cross build environment. This is running a command under qemu usermode

pybench.py -n 10

which will execute 10 iterations of benchmarks to generate profile instrumentation data

For raspberrypi3 builds on a Xeon E5 it now takes 1503 seconds (approx 25 minutes) in do_compile, its clearly not coming without a huge build time penalty

when running a raspberrypi3 build its running the test with qemuarm usermode
rpi3 being (armv7) and qemuarm being (armv5) I wonder how accurate the information is going to be.

This works much faster for x86 targets but thats probably due to host and target arches being same.

At the least this should be turned into a PACKAGECONFIG so users can disable it

I would also like to propose to disable it for non-x86 architectures by default or more appropriately for cases when buildhost arch != targethost arch

What kind of performance tests can be done, to assess the benefit of this change
Comment 1 Richard Purdie 2018-09-05 20:44:44 UTC 
The time penalty is large even on x86 as far as I can tell. We have already dropped to -n 1 which is helping and I believe Ross has some other improvements which should help.
Comment 2 Ross Burton 2018-09-13 14:48:40 UTC 
Fixed in oe-core 98b2b6c3eae15d9b84c97c6a1495040f6b7e389a.