Bug 12729 - testsdkext takes too long time to finish
Summary: testsdkext takes too long time to finish
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: High normal
Target Milestone: 2.6 M2
Assignee: Chen Qi
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2018-05-02 23:17 UTC by Chen Qi
Modified: 2018-07-07 12:50 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chen Qi 2018-05-02 23:17:07 UTC
Currently if we do a testsdkext task for the image, it would take about half an hour to finish. The time is mostly spent fetching kernel source via network.

We have in testsdk.bbclass:
	# Use our own SSTATE_DIR and DL_DIR so that updates to the eSDK come from our sstate cache
        # and we don't spend hours downloading kernels for the kernel module test
        # Abuse auto.conf since local.conf would be overwritten by the SDK
        with open(os.path.join(sdk_dir, 'conf', 'auto.conf'), 'a+') as f:
            f.write('SSTATE_MIRRORS += " \\n file://.* file://%s/PATH"\n' % test_data.get('SSTATE_DIR'))
            f.write('SOURCE_MIRROR_URL = "file://%s"\n' % test_data.get('DL_DIR'))
            f.write('INHERIT += "own-mirrors"\n')

However, this does not work. The log.do_fetch shows info as below.
DEBUG: Fetcher failure: Unable to find file file:///mnt/buildarea3/build-systemd/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz anywhere.

The .tar.gz file seems to be generated only if BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of BB_GENERATE_MIRROR_TARBALLS is "0", and according to the ref manual, users would choose to set it to "1" only if they are trying to make a source mirror. So generally, this var's value is "0".

To solve this problem, we can write to auto.conf some PREMIRRORS setting like below.

diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 2e43343..92671cc 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -157,6 +157,8 @@ def testsdkext_main(d):
             f.write('SSTATE_MIRRORS += " \\n file://.* file://%s/PATH"\n' % test_data.get('SSTATE_DIR'))
             f.write('SOURCE_MIRROR_URL = "file://%s"\n' % test_data.get('DL_DIR'))
             f.write('INHERIT += "own-mirrors"\n')
+            f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/linux-yocto.git git://%s/git2/git.yoctoproject.org.linux-yocto.git \\n"\n' % test_data.get('DL_DIR'))
+            f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/yocto-kernel-cache git://%s/git2/git.yoctoproject.org.yocto-kernel-cache \\n"\n' % test_data.get('DL_DIR'))
 
         # We need to do this in case we have a minimal SDK
         subprocess.check_output(". %s > /dev/null; devtool sdk-install meta-extsdk-toolchain" % \



Not sure if this is the correct solution. Maybe there are better ones.
Comment 1 Chen Qi 2018-07-06 02:07:31 UTC
The patch is in review.
I've pinged the patch today.