Created attachment 3571 [details] failures log_glib-2.0 Commit: 573c646d4cc62dcd0c230381df4940bdf314d495 Image: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg HW: NUC6 ----- Steps ---- 1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory) * we test it on NUC with genericx86-64 2. Install the image on DUT (using legacy boot) 3. Boot the image 4. In command line type $ ptest-runner > ptest.log EXPECTED RESULT All glib-2.0 test cases should be PASS ACTUAL RESULT test cases are failing as below: FAIL:unzip (subdir only) FAIL:unzip (bad archive) FAIL:wget--O-overrides--P FAIL:wget-handles-empty-path FAIL:wget-retrieves-google-index FAIL:wget-supports--P NOTE: full lg can be downloaded from here https://wiki.yoctoproject.org/wiki/images/8/81/Logs_2.3.M1.rc1.txt
Something went wrong with the test run here. There are indeed some FAIL lines within the glib-2.0 test output block. But the result line looks like this: --- SUMMARY: total=249; passed=249; skipped=0; failed=0; user=78.5s; system=68.4s; maxrss=5902652 --- The failures are also not actual glib-2.0 tests: I think they're busybox ones, they're just printed in the middle of the glib-2.0 tests. There's also something wrong with the encoding of the log file, editors don't like to open it.... How did the test run output get corrupted like this?
Aha, some time before the glib-2.0 tests this happens: ==TIMEOUT: /usr/lib/busybox/ptest ERROR: Exit status is 9 END: /usr/lib/busybox/ptest So I think there are two bugs here: * busybox tests try to do wget on google.com: that fails but apparently takes a long time to fail... * the testrunner can't cope with that and ends up with corrupt output
I'm renaming this "testrunner timeout can lead to garbage output": If testrunner timeouts tests it should also make sure those tests don't get to print anything into the log after that. Mariano: assigning to you as a first guess: just push it back to me if you aren't the right owner. Busybox test problems are bug 8831.
Anibal, can you check on this? This is related with ptest
(In reply to comment #4) > Anibal, can you check on this? This is related with ptest Yes i'll take care of this bug.
(In reply to comment #3) > I'm renaming this "testrunner timeout can lead to garbage output": If > testrunner timeouts tests it should also make sure those tests don't get to > print anything into the log after that. > > Mariano: assigning to you as a first guess: just push it back to me if you > aren't the right owner. > > Busybox test problems are bug 8831. If the garbage comes from the specific ptest i.e. busybox, i can't filter that data because the runner captures the output without modifications. But i will modify the runner to ensure that not data will be append when a ptest timeout.
(In reply to comment #2) > Aha, some time before the glib-2.0 tests this happens: > > ==TIMEOUT: /usr/lib/busybox/ptest > ERROR: Exit status is 9 > END: /usr/lib/busybox/ptest > > So I think there are two bugs here: > * busybox tests try to do wget on google.com: that fails but apparently > takes a long time to fail... > * the testrunner can't cope with that and ends up with corrupt output If you mean the ERROR: Exit status is 9 line the code handles in that way so i will print before the TIMEOUT, also i'll add a \n before print TIMEOUT. Regarding the busybox lines in wget ptests, is ok busybox doesn't corrupt wget what is happen is because wget uses the default shell in this case busybox. $ busybox wget someopts Is like, $ sh wget someopts
I sent a patch to review, https://lists.yoctoproject.org/pipermail/yocto/2017-April/035534.html
After ack the patch i will bump the version to 2.0.2 and update the recipe. https://lists.yoctoproject.org/pipermail/yocto/2017-April/035534.html
Recipe upgrade sent to review, http://lists.openembedded.org/pipermail/openembedded-core/2017-April/135451.html
Merge at, http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-support/ptest-runner/ptest-runner_2.0.2.bb
Verified on 2.3 rc2 are working correctly https://wiki.yoctoproject.org/wiki/WW17_-_2017-04-21_-_Full_Test_Cycle_2.3_rc2#results