Bug 12907 - Python3 builds have slowed down significantly for non-x86 targets
Summary: Python3 builds have slowed down significantly for non-x86 targets
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 2.6
Hardware: All Multiple
: High major
Target Milestone: 2.6 M3
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2018-08-30 18:14 UTC by Khem Raj
Modified: 2018-09-13 14:48 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Khem Raj 2018-08-30 18:14:22 UTC
Since commit

http://git.openembedded.org/openembedded-core/commit/?id=05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b

We are enabling PGO for generating optimized python3 in a cross build environment. This is running a command under qemu usermode

pybench.py -n 10

which will execute 10 iterations of benchmarks to generate profile instrumentation data

For raspberrypi3 builds on a Xeon E5 it now takes 1503 seconds (approx 25 minutes) in do_compile, its clearly not coming without a huge build time penalty

when running a raspberrypi3 build its running the test with qemuarm usermode
rpi3 being (armv7) and qemuarm being (armv5) I wonder how accurate the information is going to be.

This works much faster for x86 targets but thats probably due to host and target arches being same.

At the least this should be turned into a PACKAGECONFIG so users can disable it

I would also like to propose to disable it for non-x86 architectures by default or more appropriately for cases when buildhost arch != targethost arch

What kind of performance tests can be done, to assess the benefit of this change
Comment 1 Richard Purdie 2018-09-05 20:44:44 UTC
The time penalty is large even on x86 as far as I can tell. We have already dropped to -n 1 which is helping and I believe Ross has some other improvements which should help.
Comment 2 Ross Burton 2018-09-13 14:48:40 UTC
Fixed in oe-core 98b2b6c3eae15d9b84c97c6a1495040f6b7e389a.