10842 2016-12-21 22:30:55 +0000 testrunner test timeout can lead to garbage output 2017-04-28 17:34:25 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.3 x86 Multiple VERIFIED FIXED Medium normal 2.3 M4 1 jose.perez.carranza anibal.limon meta.mr.watcher meta.watcher ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 69296 0 3571 jose.perez.carranza 2016-12-21 22:30:55 +0000 Created attachment 3571 failures log_glib-2.0 Commit: 573c646d4cc62dcd0c230381df4940bdf314d495 Image: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg HW: NUC6 ----- Steps ---- 1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory) * we test it on NUC with genericx86-64 2. Install the image on DUT (using legacy boot) 3. Boot the image 4. In command line type $ ptest-runner > ptest.log EXPECTED RESULT All glib-2.0 test cases should be PASS ACTUAL RESULT test cases are failing as below: FAIL:unzip (subdir only) FAIL:unzip (bad archive) FAIL:wget--O-overrides--P FAIL:wget-handles-empty-path FAIL:wget-retrieves-google-index FAIL:wget-supports--P NOTE: full lg can be downloaded from here https://wiki.yoctoproject.org/wiki/images/8/81/Logs_2.3.M1.rc1.txt 69324 1 jku 2016-12-22 16:34:20 +0000 Something went wrong with the test run here. There are indeed some FAIL lines within the glib-2.0 test output block. But the result line looks like this: --- SUMMARY: total=249; passed=249; skipped=0; failed=0; user=78.5s; system=68.4s; maxrss=5902652 --- The failures are also not actual glib-2.0 tests: I think they're busybox ones, they're just printed in the middle of the glib-2.0 tests. There's also something wrong with the encoding of the log file, editors don't like to open it.... How did the test run output get corrupted like this? 69327 2 jku 2016-12-22 16:51:04 +0000 Aha, some time before the glib-2.0 tests this happens: ==TIMEOUT: /usr/lib/busybox/ptest ERROR: Exit status is 9 END: /usr/lib/busybox/ptest So I think there are two bugs here: * busybox tests try to do wget on google.com: that fails but apparently takes a long time to fail... * the testrunner can't cope with that and ends up with corrupt output 69329 3 jku 2016-12-22 17:02:56 +0000 I'm renaming this "testrunner timeout can lead to garbage output": If testrunner timeouts tests it should also make sure those tests don't get to print anything into the log after that. Mariano: assigning to you as a first guess: just push it back to me if you aren't the right owner. Busybox test problems are bug 8831. 69982 4 mariano.lopez 2017-01-20 21:09:58 +0000 Anibal, can you check on this? This is related with ptest 69983 5 anibal.limon 2017-01-20 21:19:00 +0000 (In reply to comment #4) > Anibal, can you check on this? This is related with ptest Yes i'll take care of this bug. 72159 6 anibal.limon 2017-04-06 20:25:50 +0000 (In reply to comment #3) > I'm renaming this "testrunner timeout can lead to garbage output": If > testrunner timeouts tests it should also make sure those tests don't get to > print anything into the log after that. > > Mariano: assigning to you as a first guess: just push it back to me if you > aren't the right owner. > > Busybox test problems are bug 8831. If the garbage comes from the specific ptest i.e. busybox, i can't filter that data because the runner captures the output without modifications. But i will modify the runner to ensure that not data will be append when a ptest timeout. 72161 7 anibal.limon 2017-04-06 20:38:08 +0000 (In reply to comment #2) > Aha, some time before the glib-2.0 tests this happens: > > ==TIMEOUT: /usr/lib/busybox/ptest > ERROR: Exit status is 9 > END: /usr/lib/busybox/ptest > > So I think there are two bugs here: > * busybox tests try to do wget on google.com: that fails but apparently > takes a long time to fail... > * the testrunner can't cope with that and ends up with corrupt output If you mean the ERROR: Exit status is 9 line the code handles in that way so i will print before the TIMEOUT, also i'll add a \n before print TIMEOUT. Regarding the busybox lines in wget ptests, is ok busybox doesn't corrupt wget what is happen is because wget uses the default shell in this case busybox. $ busybox wget someopts Is like, $ sh wget someopts 72165 8 anibal.limon 2017-04-06 21:02:39 +0000 I sent a patch to review, https://lists.yoctoproject.org/pipermail/yocto/2017-April/035534.html 72166 9 anibal.limon 2017-04-06 21:03:12 +0000 After ack the patch i will bump the version to 2.0.2 and update the recipe. https://lists.yoctoproject.org/pipermail/yocto/2017-April/035534.html 72215 10 anibal.limon 2017-04-10 17:11:32 +0000 Recipe upgrade sent to review, http://lists.openembedded.org/pipermail/openembedded-core/2017-April/135451.html 72391 11 anibal.limon 2017-04-12 17:20:22 +0000 Merge at, http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-support/ptest-runner/ptest-runner_2.0.2.bb 72946 12 jose.perez.carranza 2017-04-28 17:34:25 +0000 Verified on 2.3 rc2 are working correctly https://wiki.yoctoproject.org/wiki/WW17_-_2017-04-21_-_Full_Test_Cycle_2.3_rc2#results 3571 2016-12-21 22:30:55 +0000 2016-12-21 22:30:55 +0000 failures log_glib-2.0 log_glib-2.0 application/octet-stream 1566 jose.perez.carranza RkFJTDogdW56aXAgKHN1YmRpciBvbmx5KQotLS0gZXhwZWN0ZWQKKysrIGFjdHVhbApAQCAtMSAr MCwwIEBACi15ZXMKPT09PT09PT09PT09PT09PT09PT09PQplY2hvIC1uZSAnJyA+aW5wdXQKZWNo byAtbmUgJ2JlZ2luLWJhc2U2NCA2NDQgYmFkLnppcApVRXNEQkJRQUFna0lBQUFBSVFBNUFBQUFO d0FBQUR3QUFBQVFBQWNBWFROcXdyMXl3cXRKR3hKTExTVkoKZUNrQkQwQWRLQms4SnpRc0lqMDFK QzAvT1JKUVN3TUVGQUVDQ0FBQUFBQWhBRG9BQUFBUEFBQUFOZ0FBCkFBd0FBUUFTdzczQ3QxRENv a29oUFhRaU5qb1VOVFVpSFJ3Z0xUNFdIbEJMQVFJUUFCUUFBZ2dJQUFBQQpvUUE1QUFBQU53QUFB RHdBQUFBUVFBY0FEQUFBQUN3QU1nQ0FBQUFBQUFCZE0yckN2WExDcTBrYkVrc3QKSlVsNEtRRVBR QjBvR1NZNEN6NFFOZ0VuSlNZSVBWQkxBUUlBQUJRQUFnZ0FBQUFBSVFBcUFBQUFEd0FBCkJEWUFB QUFNQUFFQURRQUFBRElBRFFBQUFFRUFBQUFTdzczQ3QxREtva29oUFhRaU56QStGQUkxSENjVwpO eklUTkZCTEJRVUtBQzRKQUEwNEN3MEVPaFpRU3dVR0FRQUFCQUlBQWdDWkFBQUFlUUFBQUFJQUxo TT0KPT09PQonIHwgdXVkZWNvZGU7IHVuemlwIGJhZC56aXAgMj4mMTsgZWNobyAkPwpGQUlMOiB1 bnppcCAoYmFkIGFyY2hpdmUpCi0tLSBleHBlY3RlZAorKysgYWN0dWFsCkBAIC0xLDUgKzEsMiBA QAotQXJjaGl2ZTogIGJhZC56aXAKLSAgaW5mbGF0aW5nOiBdM2rDgsK9csOCwqtJXlteUkstJUl4 Ci11bnppcDogY29ycnVwdGVkIGRhdGEKLXVuemlwOiBpbmZsYXRlIGVycm9yCit1bnppcDogY2Fu J3Qgb3BlbiBiYWQuemlwWy56aXBdCiAxCkZBSUw6IHdnZXQtLU8tb3ZlcnJpZGVzLS1QCisgdGVz dCB4ICchPScgeAorIG1rZGlyIGZvbworIGJ1c3lib3ggd2dldCAtcSAtTyBpbmRleC5odG1sIC1Q IGZvbyBodHRwOi8vd3d3Lmdvb2dsZS5jb20vCndnZXQ6IGNhbid0IGNvbm5lY3QgdG8gcmVtb3Rl IGhvc3QgKDIxNi41OC4xOTQuMTY0KTogQ29ubmVjdGlvbiB0aW1lZCBvdXQKRkFJTDogd2dldC1o YW5kbGVzLWVtcHR5LXBhdGgKKyB0ZXN0IHggJyE9JyB4CisgYnVzeWJveCB3Z2V0IGh0dHA6Ly93 d3cuZ29vZ2xlLmNvbQpDb25uZWN0aW5nIHRvIHd3dy5nb29nbGUuY29tICgyMTYuNTguMTk0LjE2 NDo4MCkKd2dldDogY2FuJ3QgY29ubmVjdCB0byByZW1vdGUgaG9zdCAoMjE2LjU4LjE5NC4xNjQp OiBDb25uZWN0aW9uIHRpbWVkIG91dApGQUlMOiB3Z2V0LXJldHJpZXZlcy1nb29nbGUtaW5kZXgK KyB0ZXN0IHggJyE9JyB4CisgYnVzeWJveCB3Z2V0IC1xIC1PIGZvbyBodHRwOi8vd3d3Lmdvb2ds ZS5jb20vCndnZXQ6IGNhbid0IGNvbm5lY3QgdG8gcmVtb3RlIGhvc3QgKDIxNi41OC4xOTQuMTY0 KTogQ29ubmVjdGlvbiB0aW1lZCBvdXQKRkFJTDogd2dldC1zdXBwb3J0cy0tUAorIHRlc3QgeCAn IT0nIHgKKyBta2RpciBmb28KKyBidXN5Ym94IHdnZXQgLXEgLVAgZm9vIGh0dHA6Ly93d3cuZ29v Z2xlLmNvbS8Kd2dldDogY2FuJ3QgY29ubmVjdCB0byByZW1vdGUgaG9zdCAoMjE2LjU4LjE5NC4x NjQpOiBDb25uZWN0aW9uIHRpbWVkIG91dAoK