Performances curves [1,2] indicate that a commit(s) introduced during the M1 window degrade the build time performance of bitbake core-image sato (with the rm-work) and bitbake virtual/kernel [1] https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rmwork.html [2] https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-virtual-kernel.html git-bisection is required to figure out those commits introducing the performance overhead.
After
The following perf summary contains the two commits where curves start raising. General Current commit Comparing with Hostname ypperf-fedora25 ypperf-fedora25 Branch master master Commit number 45983 45928 Commit ba0098e0d16 b1d5267f0e3 Number of test runs 1 12 Test result summary test1: Build core-image-sato build time 1:33:53 +1:32.7 +1.7 % tmpdir size 29.67 GiB -327.76 MiB -1.1 % test12: Build virtual/kernel build time 7:52.8 +20.2 s +4.5 % test13: Build core-image-sato with rm_work enabled build time 1:34:08 +2:39.8 +2.9 % tmpdir size 6.20 GiB -331.22 MiB -5.0 % test2: Run core-image-sato do_rootfs with sstate do_rootfs time 3:07.3 +0.57 s +0.3 % test3: Bitbake parsing (bitbake -p) parse_1 time 33.9 s +0.35 s +1.1 % parse_2 time 21.3 s +0.17 s +0.8 % parse_3 time 2.0 s -0.07 s -3.4 % test4: eSDK metrics installer_bin size 1000.93 MiB -272.10 MiB -21.4 % deploy time 3:27.7 -16.7 s -7.4 % deploy_dir size 3.20 GiB -606.33 MiB -15.6 %
Manual bisecting show the the following numbers at the fedora 25 machine ypperf01,(nobranch):ba0098e0d16f38e53b8aeeaca894a46dd4577fab,ba0098e0d16f38e53b8aeeaca894a46dd4577fab,1:02:25.25,0:05:14.23,1:03:50.53,0:01:55.60,0:00:25.06,0:00:15.52,0:00:01.37,0:02:14.46,31286900,6600632,1024212,3370388 ypperf01,(nobranch):f1a5c7c53be36bbee3b7476b336bab5a04c2d46d,f1a5c7c53be36bbee3b7476b336bab5a04c2d46d,1:02:18.65,0:05:21.29,1:03:40.96,0:01:56.57,0:00:24.48,0:00:15.12,0:00:01.40,0:02:13.64,31282112,6602564,1024355,3370482 ypperf01,(nobranch):66683198cf22d2dc0b18ec4d5f2218a51b8a1685,66683198cf22d2dc0b18ec4d5f2218a51b8a1685,1:02:23.47,0:05:20.12,1:03:44.34,0:01:54.68,0:00:24.33,0:00:15.16,0:00:01.36,0:02:14.70,31284428,6602396,1024435,3370296 ypperf01,(nobranch):b7a173ef4132ea2b4ecb4ba0dc9d7b3a4163383e,b7a173ef4132ea2b4ecb4ba0dc9d7b3a4163383e,1:00:40.77,0:05:08.07,1:02:00.91,0:01:54.06,0:00:24.63,0:00:15.12,0:00:01.35,0:02:21.81,31624084,6945176,1303670,3992402 ypperf01,(nobranch):453835679ba36b2e07f7aa8aa653a8cfeda95675,453835679ba36b2e07f7aa8aa653a8cfeda95675,1:00:27.25,0:05:03.89,1:01:59.06,0:01:53.45,0:00:24.85,0:00:15.42,0:00:01.43,0:02:16.64,31618588,6939884,1302993,3991709 So commit 66683198cf22d2dc0b18ec4d5f2218a51b8a1685 is the one introducing the perf overhead, corresponding to: commit 66683198cf22d2dc0b18ec4d5f2218a51b8a1685 Author: Alexander Kanavin <alexander.kanavin@linux.intel.com> Date: Fri Jun 9 14:13:58 2017 +0300 package_rpm.bbclass: use multithreaded xz compression RPM's default is single-threaded gz; the change greatly helps with both buildtimes (when there is a small number of large-sized packages) and disk space taken by resulting rpms. (From OE-Core rev: f108c4d09926bd28e7a57b665fc8cb5373827780) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass index a844c4d..dc24197 100644 --- a/meta/classes/package_rpm.bbclass +++ b/meta/classes/package_rpm.bbclass @@ -644,6 +644,8 @@ python do_package_rpm () { cmd = cmd + " --define '_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'" cmd = cmd + " --define '_use_internal_dependency_generator 0'" cmd = cmd + " --define '_binaries_in_noarch_packages_terminate_build 0'" + cmd = cmd + " --define '_binary_payload w6T.xzdio'" + cmd = cmd + " --define '_source_payload w6T.xzdio'" if perfiledeps: cmd = cmd + " --define '__find_requires " + outdepends + "'"
Thanks Leo, this is really good analysis. Looks like build times went up due to the change however we did see much better package and eSDK sizes as a result. We can therefore call this a reasonable compromise and decide to accept the small performance hit.