Bug 5346 - Investigate build performance variance between different host OS-es on same HW
Summary: Investigate build performance variance between different host OS-es on same HW
Status: RESOLVED WORKSFORME
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 1.6
Hardware: x86 Multiple
: Medium normal
Target Milestone: Future
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-10-14 10:44 UTC by Alexandru Georgescu
Modified: 2025-06-19 16:40 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandru Georgescu 2013-10-14 10:44:57 UTC
Currently QA runs the performance scripts on same HW but different OS-es. 
Details here: https://wiki.yoctoproject.org/wiki/Performance_Test

All system/distro settings are distro's defaults, that includes:

- FS mount options: ext4 rw,relatime,data=ordered

- IO scheduler: cfq and CPU governer: ondemand

HW config is: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD 

We have to investigate why there are big differences between them.
Comment 1 Richard Purdie 2019-06-06 15:19:43 UTC
There still are performance differences between our Ubuntu and Fedora tests. A part of this was found to be python being compiled with profile guided optimisation which makes a big difference. Our buildtools tarball now has that.
Comment 2 Randy MacLeod 2021-06-10 15:03:47 UTC
We're still seeing a non-trivial difference between Fedora and Ubuntu.
Comment 3 Randy MacLeod 2025-06-19 16:11:11 UTC
The build performance difference between Debian-12 and Alma-8 is 
not huge and is now acceptable.

Measurement        Debian        Alma           Percent Diff     Note

sato build         1:14:59       1:15:13         + 0.31 %
sato sstate           1:09.4        1:07.5       - 2.74 %
kernel build          4:20.3        4:06.3       -12.1 %         
bitbake -p (1)          12.2          13.8       +13 %           python version?

https://valkyrie.yocto.io/pub/non-release/20250619-53/testresults/buildperf-debian11/perf-debian12-vk_master_20250619090025_7733ddf733.html

https://valkyrie.yocto.io/pub/non-release/20250619-55/testresults/buildperf-alma8/perf-alma8-vk_master_20250619090031_7733ddf733.html

I may look into the kernel build time but I suspect there is more variance than the ~1 second stddev reported on the build time graphs.
Comment 4 Randy MacLeod 2025-06-19 16:40:48 UTC
Since we didn't change anything, I've changed the resolution status to WORKSFORME