Bug 11751

Summary:	Performance regressions detected on core-image-sato and virtual/kernel at 2.4M1 rc1
Product:	[QA/Testing] Build Testing	Reporter:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Component:	general	Assignee:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status:	RESOLVED WONTFIX	QA Contact:
Severity:	normal
Priority:	Undecided	CC:	alex.kanavin, richard.purdie
Version:	2.4
Target Milestone:	---
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Leonardo Sandoval Gonzalez 2017-07-04 14:14:02 UTC

Performances curves [1,2] indicate that a commit(s) introduced during the M1 window degrade the build time performance of bitbake core-image sato (with the rm-work) and bitbake virtual/kernel

[1] https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rmwork.html

[2] https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-virtual-kernel.html

git-bisection is required to figure out those commits introducing the performance overhead.

Comment 1 Leonardo Sandoval Gonzalez 2017-07-04 14:14:37 UTC

After

Comment 2 Leonardo Sandoval Gonzalez 2017-07-04 14:18:56 UTC

The following perf summary contains the two commits where curves start raising.


General
Current commit	Comparing with
Hostname	ypperf-fedora25	ypperf-fedora25
Branch	master	master
Commit number	45983	45928
Commit	ba0098e0d16	b1d5267f0e3
Number of test runs	1	12


Test result summary
test1: Build core-image-sato	build time	1:33:53	+1:32.7	+1.7 %
                                tmpdir size	29.67 GiB	-327.76 MiB	-1.1 %

test12: Build virtual/kernel	build time	7:52.8	+20.2 s	+4.5 %
test13: Build core-image-sato with rm_work enabled	build time	1:34:08	+2:39.8	+2.9 %
tmpdir size	6.20 GiB	-331.22 MiB	-5.0 %

test2: Run core-image-sato do_rootfs with sstate	do_rootfs time	3:07.3	+0.57 s	+0.3 %

test3: Bitbake parsing (bitbake -p)	parse_1 time	33.9 s	+0.35 s	+1.1 %
parse_2 time	21.3 s	+0.17 s	+0.8 %
parse_3 time	2.0 s	-0.07 s	-3.4 %

test4: eSDK metrics	installer_bin size	1000.93 MiB	-272.10 MiB	-21.4 %
deploy time	3:27.7	-16.7 s	-7.4 %
deploy_dir size	3.20 GiB	-606.33 MiB	-15.6 %

Comment 3 Leonardo Sandoval Gonzalez 2017-07-04 14:23:39 UTC

Manual bisecting show the the following numbers at the fedora 25 machine

ypperf01,(nobranch):ba0098e0d16f38e53b8aeeaca894a46dd4577fab,ba0098e0d16f38e53b8aeeaca894a46dd4577fab,1:02:25.25,0:05:14.23,1:03:50.53,0:01:55.60,0:00:25.06,0:00:15.52,0:00:01.37,0:02:14.46,31286900,6600632,1024212,3370388
ypperf01,(nobranch):f1a5c7c53be36bbee3b7476b336bab5a04c2d46d,f1a5c7c53be36bbee3b7476b336bab5a04c2d46d,1:02:18.65,0:05:21.29,1:03:40.96,0:01:56.57,0:00:24.48,0:00:15.12,0:00:01.40,0:02:13.64,31282112,6602564,1024355,3370482
ypperf01,(nobranch):66683198cf22d2dc0b18ec4d5f2218a51b8a1685,66683198cf22d2dc0b18ec4d5f2218a51b8a1685,1:02:23.47,0:05:20.12,1:03:44.34,0:01:54.68,0:00:24.33,0:00:15.16,0:00:01.36,0:02:14.70,31284428,6602396,1024435,3370296
ypperf01,(nobranch):b7a173ef4132ea2b4ecb4ba0dc9d7b3a4163383e,b7a173ef4132ea2b4ecb4ba0dc9d7b3a4163383e,1:00:40.77,0:05:08.07,1:02:00.91,0:01:54.06,0:00:24.63,0:00:15.12,0:00:01.35,0:02:21.81,31624084,6945176,1303670,3992402
ypperf01,(nobranch):453835679ba36b2e07f7aa8aa653a8cfeda95675,453835679ba36b2e07f7aa8aa653a8cfeda95675,1:00:27.25,0:05:03.89,1:01:59.06,0:01:53.45,0:00:24.85,0:00:15.42,0:00:01.43,0:02:16.64,31618588,6939884,1302993,3991709


So commit 66683198cf22d2dc0b18ec4d5f2218a51b8a1685 is the one introducing the perf overhead, corresponding to:


commit 66683198cf22d2dc0b18ec4d5f2218a51b8a1685
Author: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Date:   Fri Jun 9 14:13:58 2017 +0300

    package_rpm.bbclass: use multithreaded xz compression
    
    RPM's default is single-threaded gz; the change greatly helps with
    both buildtimes (when there is a small number of large-sized packages)
    and disk space taken by resulting rpms.
    
    (From OE-Core rev: f108c4d09926bd28e7a57b665fc8cb5373827780)
    
    Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
    Signed-off-by: Ross Burton <ross.burton@intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index a844c4d..dc24197 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -644,6 +644,8 @@ python do_package_rpm () {
     cmd = cmd + " --define '_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'"
     cmd = cmd + " --define '_use_internal_dependency_generator 0'"
     cmd = cmd + " --define '_binaries_in_noarch_packages_terminate_build 0'"
+    cmd = cmd + " --define '_binary_payload w6T.xzdio'"
+    cmd = cmd + " --define '_source_payload w6T.xzdio'"
     if perfiledeps:
         cmd = cmd + " --define '__find_requires " + outdepends + "'"

Comment 4 Richard Purdie 2017-07-06 14:57:27 UTC

Thanks Leo, this is really good analysis. Looks like build times went up due to the change however we did see much better package and eSDK sizes as a result. We can therefore call this a reasonable compromise and decide to accept the small performance hit.