Bug 5346

Summary:	Investigate build performance variance between different host OS-es on same HW
Product:	[QA/Testing] Build Testing	Reporter:	Alexandru Georgescu <alexandru.c.georgescu>
Component:	general	Assignee:	Unassigned <unassigned>
Status:	RESOLVED WORKSFORME	QA Contact:
Severity:	normal
Priority:	Medium	CC:	randy.macleod, richard.purdie, sgw
Version:	1.6
Target Milestone:	Future
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Alexandru Georgescu 2013-10-14 10:44:57 UTC

Currently QA runs the performance scripts on same HW but different OS-es. 
Details here: https://wiki.yoctoproject.org/wiki/Performance_Test

All system/distro settings are distro's defaults, that includes:

- FS mount options: ext4 rw,relatime,data=ordered

- IO scheduler: cfq and CPU governer: ondemand

HW config is: SandyBridge i7-2600 CPU @ 3.40GHz HT-enabled (4 cores/8 threads), 8GB RAM, 1 x 500 GB HDD 

We have to investigate why there are big differences between them.

Comment 1 Richard Purdie 2019-06-06 15:19:43 UTC

There still are performance differences between our Ubuntu and Fedora tests. A part of this was found to be python being compiled with profile guided optimisation which makes a big difference. Our buildtools tarball now has that.

Comment 2 Randy MacLeod 2021-06-10 15:03:47 UTC

We're still seeing a non-trivial difference between Fedora and Ubuntu.

Comment 3 Randy MacLeod 2025-06-19 16:11:11 UTC

The build performance difference between Debian-12 and Alma-8 is 
not huge and is now acceptable.

Measurement        Debian        Alma           Percent Diff     Note

sato build         1:14:59       1:15:13         + 0.31 %
sato sstate           1:09.4        1:07.5       - 2.74 %
kernel build          4:20.3        4:06.3       -12.1 %         
bitbake -p (1)          12.2          13.8       +13 %           python version?

https://valkyrie.yocto.io/pub/non-release/20250619-53/testresults/buildperf-debian11/perf-debian12-vk_master_20250619090025_7733ddf733.html

https://valkyrie.yocto.io/pub/non-release/20250619-55/testresults/buildperf-alma8/perf-alma8-vk_master_20250619090031_7733ddf733.html

I may look into the kernel build time but I suspect there is more variance than the ~1 second stddev reported on the build time graphs.

Comment 4 Randy MacLeod 2025-06-19 16:40:48 UTC

Since we didn't change anything, I've changed the resolution status to WORKSFORME