Example log from autobuilder: --- Running Extensible SDK tests ... Testing /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/testsdkext//tc/environment-setup-core2-64-poky-linux test_devtool_add_reset (oeqa.sdkext.devtool.DevtoolTest) ... ok test_devtool_build_cmake (oeqa.sdkext.devtool.DevtoolTest) ... ok test_devtool_build_esdk_package (oeqa.sdkext.devtool.DevtoolTest) ... ok test_devtool_build_make (oeqa.sdkext.devtool.DevtoolTest) ... ok test_devtool_kernelmodule (oeqa.sdkext.devtool.DevtoolTest) ... ok test_devtool_location (oeqa.sdkext.devtool.DevtoolTest) ... ok test_extend_autotools_recipe_creation (oeqa.sdkext.devtool.DevtoolTest) ... ok test_recipes_for_nodejs (oeqa.sdkext.devtool.DevtoolTest) ... ok test_sdk_update_http (oeqa.sdkext.sdk_update.SdkUpdateTest) ... ok ---------------------------------------------------------------------- Ran 9 tests in 770.203s --- 770 seconds sounds excessive. I did some testing and test_recipes_for_nodejs seems like a good candidate to look closer: I believe it runs something like recipetool --color=always create -o /tmp/devtoolhvyleul_ "npm://registry.npmjs.org;name=forever;version=0.15.1" -x /home/jku/src/poky/build/workspace/sources/devtoolsrc215gwwmm -N myapp The download + unpack alone took 10 minutes when I ran that in terminal. I'll test with the actual ESDK as well to make sure I didn't make mistakes (filing this now already so I don't forget).
I have sent a fix for the node.js test part at least, which should reduce the time for the test to under a minute (by using a different npm package to test with that has a much smaller dependency tree). However what puzzles me a bit is that the times reported in a recent autobuilder log only add up to about 33 minutes - even if you assume a few minutes for bitbake startup that's still some way short of the 52 minute time reported for the autobuilder job.
The node.js fix was merged into oe-core master with 638ee71da967f093071ba25e0ea5c467dab65339.
Including cProfile as seen on [1], these are the times profiled for each esdk using a local AB & nightly-x86 buildset 0.011 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:38(test_devtool_location) 0.012 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:38(test_devtool_location) 17.419 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:91(test_recipes_for_nodejs) 17.570 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:43(test_devtool_add_reset) 19.475 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:48(test_devtool_build_make) 21.027 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:48(test_devtool_build_make) 23.054 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:91(test_recipes_for_nodejs) 23.394 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:53(test_devtool_build_esdk_package) 24.624 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:58(test_devtool_build_cmake) 24.800 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:53(test_devtool_build_esdk_package) 30.295 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:43(test_devtool_add_reset) 62.687 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/sdk_update.py:30(test_sdk_update_http) 90.857 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/sdk_update.py:30(test_sdk_update_http) 121.314 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:63(test_extend_autotools_recipe_creation) 136.540 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:63(test_extend_autotools_recipe_creation) 375.240 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:58(test_devtool_build_cmake) 1977.591 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:77(test_devtool_kernelmodule) 2263.954 /home/ab/autobuilder/yocto-worker/nightly-x86/build/meta/lib/oeqa/sdkext/devtool.py:77(test_devtool_kernelmodule) From this data, we can say the following: * the most expensive tests is test_devtool_kernelmethod. This test builds a kernel driver and as I can see, it means rebuilding the whole kernel so this takes lots of time * tests are running twice, because the eSDK sanity tests are running on to targets, so this is expected at the AB nightly-x86 build set In my opinion, we should resolve this bug as wontfix. In case we need to speed up things, another bug should be filed in other to tackle optimization . [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lsandov1/oesdktest-profiling
Created attachment 3473 [details] AB log including profiling data This is the (ESDK) log produced by the AB launching a nightly x86 build. The table shown on https://bugzilla.yoctoproject.org/show_bug.cgi?id=10254#c3 was produced with the following cmd line: $ cat esdk.profile | grep sdkext | grep \(test_ | grep -v ok | awk '{print $4, $6}' | sort -n
After talking with Paul E, there are two approaches to reduce time: use share states on eSDK tests and reduce the number of targets on the relevant AB buildsets.
(In reply to comment #5) > After talking with Paul E, there are two approaches to reduce time: use > share states on eSDK tests and reduce the number of targets on the relevant > AB buildsets. I tried to implement sstate on eSDK for this script without success. I suggest to close as won't fix.
This bug is specific to a test case (eSDK) but we should profile all tests suites not just one. For the latter task, there is a wider task bug (bug 10874) and work will be done there, with the intention to find those big bottlenecks. *** This bug has been marked as a duplicate of bug 10874 ***