Steps: 1- clone poky 2- Download Manifest form public autobuilder http://autobuilder.yoctoproject.org/pub/releases/yocto 2.2_M3.rc1/machines/genericx86/core-image-sato-sdk-genericx86.manifest 3 - Put manifest under build/tmp/deploy/images/genericx86 4 - add below to conf/local.conf INHERIT += "testimage" TEST_TARGET = "simpleremote" TEST_SERVER_IP = "HOST ip" TEST_TARGET_IP = "DUT ip" TEST_SUITES = " auto" 5. Edit bblayers.conf, bblayers.conf PATH:~/poky/build/conf adn add meta-selftest 6. bitbake core-image-sato-sdk -c testimage Expected : testimage is executed correctly Actual : ERROR: core-image-sato-sdk-1.0-r0 do_testimage: Could not invoke smart. Command ' --log-level=warning --data-dir=/home/jgperezc/work/YoctoProject/poky-2.2M3/build/tmp/work/genericx86-poky-linux/core-image-sato-sdk/1.0-r0/testimage/install_tmp/socat/var/lib/smart config --set rpm-root=/home/jgperezc/work/YoctoProject/poky-2.2M3/build/tmp/work/genericx86-poky-linux/core-image-sato-sdk/1.0-r0/testimage/install_tmp/socat' returned 127: /bin/sh: 1: --log-level=warning: not found
Is it attempting to run createrepo, so was fixed by this commit: testimage: add createrepo-native to testimage depends The smart tests need createrepo to be in the native sysroot. It needs to be one of the depends for testimage. (From OE-Core rev: 2f1a44d33d9ecceef69d11ef92c5369314cb3bf5) (CCing Brian who wrote that commit in case he can answer this quickly)
This has been solved in commit 32b82672033013ad89b931aa1235415db699415e mentioned by Ross
Forgot to add the meta-selftest layer, the bug is still there
Sent patch to the mailing list: http://lists.openembedded.org/pipermail/openembedded-core/2016-September/126783.html
This has been solved in commit: http://git.openembedded.org/openembedded-core/commit/meta/classes/testimage.bbclass?id=cf548fd85297585cc5688eda45ee332200bbd4b7
Created attachment 3507 [details] testimage_selftest_socat_error.log
Created attachment 3508 [details] testimage_selftest_test-sdk_error.log
I tested again on 2.2.rc6 with environment genericx86 on MMAX32. However, after setting the environment and executing 'bitbake core-image-sato-sdk -c testimage', I received the following error: << ERROR: core-image-sato-sdk-1.0-r0 do_testimage: Unable to list available packages. Command '/home/jairdeje/source/testpoky2/build/tmp/sysroots/x86_64-linux/usr/bin/smart --log-level=warning --data-dir=/home/jairdeje/source/testpoky2/build/tmp/work/genericx86-poky-linux/core-image-sato-sdk/1.0-r0/testimage/install_tmp/socat/var/lib/smart info --urls socat' returned 1: Loading cache... Updating cache... ######################################## [100%] error: 'socat' matches no packages >> After that, I proceeded to build 'socat' using 'bitbake socat' and executed testimage again. Following this, I received the next error on test_sdk: << core-image-sato-sdk-1.0-r0 do_testimage: ====================================================================== core-image-sato-sdk-1.0-r0 do_testimage: FAIL: test_sdk (oeqa.runtime.selftest.Selftest) core-image-sato-sdk-1.0-r0 do_testimage: ---------------------------------------------------------------------- >> In both cases, I have attached the related logs for further reference.
Changing the status to reopened as a related issue still exists as explained on my previous comment.
Before the patch I submitted, bitbake's error was very obscure and it was difficult to understand what was happening. Now it says that socat is missing. Now the error itself is because the image was generated without the "meta-selftest" layer and runtime testing is being run with this layer enabled. The way that runtime and selftest discover the tests is at follows: 1. Get BBLAYERS var from bitbake. 2. Search for <layer>/lib/oeqa/<runtime|selftest> folder. 3. Import all tests from these folders for all the layers. This wasn't a problem before 2.2 M2 because runtime testing wasn't tested by selftest. Now that we have those test in selftest layer the directory meta-selftes/lib/oeqa/runtime exists and if meta-selftest layer is added in runtime test it will pick up those test. It is worth to mention that runtime tests in meta-selftest layers are used to test runtime testing functionality and not the actual image produced by bitbake/OE; so the tests will fail if the image wasn't created with very special parameters . Finally, the test_sdk that is failing is because this test is for a runexported functionality that will check if a sdk was exported along with runtime test to be run in a device that lacks some of the binaries needed to do the testing. So, my advise on this would be to remove meta-selftest layer when running runtime tests. TL;DR remove meta-selftest when running runtime test.
Mariano Thanks for your feedback on this bug, I'm agree with your comment, so the steps on the wiki to do the test image should be updated. Jair Are you able to update the wiki with mentioned changes.
As per Mariano's suggestion, I've updated the affected QA wiki pages by removing the step to add the selftest configuration entry to bblayers.conf from each one of the BSP and LSB automated testing workflows. For further reference, the changed pages are: https://wiki.yoctoproject.org/wiki/Quality_Assurance_yocto_project https://wiki.yoctoproject.org/wiki/Steps_for_Automated_Testing_on_QEMU
Moving the status to resolved after executing the change described on my previous comment.
Jose, could you please cross-verify the changes I made to the QA wiki pages regarding this issue? I'll assign the ticket to you to validate the bug fix if you're in agreement.
Infomration is now updated on wiki.