So I added _ptest to TEST_SUITES and ptest-pkgs to IMAGE_FEATURES, built and image and ran the tests. test_ptestrunner (oeqa.runtime._ptest.PtestRunnerTest) ... NOTE: Installing complementary packages ... ok Lies. It either failed to install ptest-runner and assumed it worked, or it's eating the numerous errors that should have happened: root@qemux86-64:/usr/lib/libpcre/ptest# ./run-ptest make[1]: Entering directory '/usr/lib/libpcre/ptest' PASS: pcre_stringpiece_unittest FAIL: RunTest PASS: RunGrepTest ============================================================================ Testsuite summary for PCRE 8.34 ============================================================================ # TOTAL: 3 # PASS: 2 # SKIP: 0 # XFAIL: 0 # FAIL: 1 # XPASS: 0 # ERROR: 0 ============================================================================ See ./test-suite.log ============================================================================ Makefile:73: recipe for target 'test-suite.log' failed make[1]: *** [test-suite.log] Error 1 make[1]: Leaving directory '/usr/lib/libpcre/ptest' Makefile:159: recipe for target 'check-TESTS' failed make: *** [check-TESTS] Error 2
During an *in progress* ptest run: root@qemux86-64:~# grep FAIL: /tmp/ptest.log | wc -l 399 An hour later, it finished: core-image-sato - Ran 24 tests in 5099.061s core-image-sato - OK - All required tests passed (skipped=1) Lies! :)
Hello Ross, To do a ptest succesfully I build a ptest capable image using these configurations in local.conf: IMAGE_FEATURES_append = " ptest-pkgs" IMAGE_INSTALL_append = " ptest-runner" INHERIT += "buildhistory" INHERIT += "testimage" TEST_SUITES = " _ptest" Buildhistory is mandatory and that test should fail without it. I also see that you skipped ptest-runner image install. But you are right, it seems that a check to see if ptest-runner actually ran succesfully is missing.
Also I need to underline here that is a difference between the ptest unittest running succesfully and the actual ptest output, that needs to be analyzed separately. Ex: tmp/work/genericx86_64-poky-linux/core-image-sato-sdk/1.0-r0/testimage/ptest-20141106084139.log That is then parsed into: tmp/work/genericx86_64-poky-linux/core-image-sato-sdk/1.0-r0/core-image-sato-sdk-1.0/results$ ls fail.acl fail.attr fail.perl fail.quilt For example we could make the unittest fail every time a "fail.*" file is resulted but I don't think that the test will ever pass with this condition.
Lucian, is "oeqa/ptest: Fixed complementary package install detection and added ptest-runner exit code check" the fix for this? (oe-core b214c7b901fe0fad081d7a97f9e91cc218c66bd6)
Yes, it is.