| Summary: | pTest - busybox tests fail for 2.1_M1.rc1 | ||
|---|---|---|---|
| Product: | [QA/Testing] Runtime Testing | Reporter: | Alexandru Georgescu <alexandru.c.georgescu> |
| Component: | general | Assignee: | dengke <dengke.du> |
| Status: | RESOLVED DUPLICATE | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | francisco.j.pedraza.gonzalez, jair.de.jesus.gonzalez.plascencia, jku, jose.perez.carranza, libertad.gonzalez.de.la.cruz, richard.purdie |
| Version: | 2.1 | ||
| Target Milestone: | Future | ||
| Hardware: | NUC | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
Alexandru Georgescu
2015-12-17 10:04:49 UTC
(In reply to comment #0) > Commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e > Previous Commit: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1 > https://wiki.yoctoproject.org/wiki/ > Ptest_5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e > > > Results for NUC ptest busybox 2015-12-10: > ******************************** > Tests that failed now and worked last time: 8 > unzip (bad archive) > echo-prints-slash_0041 > echo-prints-slash-zero > echo-prints-dash > echo-prints-slash_41 > echo-prints-non-opts > echo-prints-slash_041 > echo-prints-slash_00041 > Tests that passed now and and failed last time: 2 > ls-s-works > ls-l-works > Tests that were skipped now and worked last time: 1 > bzcat > Tests that were skipped last time and worked now: 20 > dc basic syntax (stdin multiple args) > glibc build sort > diff -B does not ignore non-blank single line change > dc complex with spaces (multiple args) > unzip (bad archive) > sed /regex/N{...} addresses work > sed /regex/+N{...} addresses work > glibc build sort unique > awk length() > dc complex without spaces (multiple args) > sed /regex/+0 -i wo rks > diff of empty file against stdin > dc complex without spaces (single arg) > sed /regex/+N{...} -i works > sed /regex/+0{...} -i works > dc complex with spaces (single arg) > diff -B ignores blank single line change > sort with non-default leading delim 4 > dc basic syntax (argv multiple args) > dc basic syntax (argv single arg) > Tests that failed now: 20 > wget-retrieves-google-index > unzip (bad archive) > cpio lists hardlinks > echo-prints-slash_0041 > echo-prints-slash-zero > echo-prints-dash > sed n (flushes pattern space terminates early) > wget-handles-empty-path > echo-prints-slash_41 > echo-prints-non-opts > hostname-d-works > echo-prints-slash_041 > sed append autoinserts newline > sed embedded NUL > wget-supports--P > sed NUL in command > echo-prints-slash_00041 > sed nonexistent label > wget--O-overrides--P > unzip (subdir only) Retested on NUC 5425WYK genericx86_64, 2016-01-28: poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c AB Link: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg Report: https://wiki.yoctoproject.org/wiki/Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c compared with commit 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e Results for NUC ptest quilt 2016-01-28: ******************************** Tests that failed now and worked last time: 0 Tests that passed now and and failed last time: 0 Tests that were skipped now and worked last time: 0 Tests that were skipped last time and worked now: 0 Tests that failed now: 1 faildiff.test (In reply to comment #1) > (In reply to comment #0) > > Commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e > > Previous Commit: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1 > > https://wiki.yoctoproject.org/wiki/ > > Ptest_5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e > > > > > > Results for NUC ptest busybox 2015-12-10: > > ******************************** > > Tests that failed now and worked last time: 8 > > unzip (bad archive) > > echo-prints-slash_0041 > > echo-prints-slash-zero > > echo-prints-dash > > echo-prints-slash_41 > > echo-prints-non-opts > > echo-prints-slash_041 > > echo-prints-slash_00041 > > Tests that passed now and and failed last time: 2 > > ls-s-works > > ls-l-works > > Tests that were skipped now and worked last time: 1 > > bzcat > > Tests that were skipped last time and worked now: 20 > > dc basic syntax (stdin multiple args) > > glibc build sort > > diff -B does not ignore non-blank single line change > > dc complex with spaces (multiple args) > > unzip (bad archive) > > sed /regex/N{...} addresses work > > sed /regex/+N{...} addresses work > > glibc build sort unique > > awk length() > > dc complex without spaces (multiple args) > > sed /regex/+0 -i wo rks > > diff of empty file against stdin > > dc complex without spaces (single arg) > > sed /regex/+N{...} -i works > > sed /regex/+0{...} -i works > > dc complex with spaces (single arg) > > diff -B ignores blank single line change > > sort with non-default leading delim 4 > > dc basic syntax (argv multiple args) > > dc basic syntax (argv single arg) > > Tests that failed now: 20 > > wget-retrieves-google-index > > unzip (bad archive) > > cpio lists hardlinks > > echo-prints-slash_0041 > > echo-prints-slash-zero > > echo-prints-dash > > sed n (flushes pattern space terminates early) > > wget-handles-empty-path > > echo-prints-slash_41 > > echo-prints-non-opts > > hostname-d-works > > echo-prints-slash_041 > > sed append autoinserts newline > > sed embedded NUL > > wget-supports--P > > sed NUL in command > > echo-prints-slash_00041 > > sed nonexistent label > > wget--O-overrides--P > > unzip (subdir only) > > > Retested on NUC 5425WYK genericx86_64, 2016-01-28: > > poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c > AB Link: > http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/ > machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg > > Report: > https://wiki.yoctoproject.org/wiki/ > Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c > compared with commit 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e > > Results for NUC ptest quilt 2016-01-28: > ******************************** > Tests that failed now and worked last time: 0 > > Tests that passed now and and failed last time: 0 > > Tests that were skipped now and worked last time: 0 > > Tests that were skipped last time and worked now: 0 > Tests that failed now: 1 > faildiff.test By mistake I added wrong results information, here you have the correct ones: Results for NUC ptest busybox 2016-01-28: ******************************** Tests that failed now and worked last time: 2 ls-s-works ls-l-works Tests that passed now and and failed last time: 7 echo-prints-slash_0041 echo-prints-slash-zero echo-prints-dash echo-prints-slash_41 echo-prints-non-opts echo-prints-slash_041 echo-prints-slash_00041 Tests that were skipped now and worked last time: 0 Tests that were skipped last time and worked now: 0 Tests that failed now: 15 wget-retrieves-google-index unzip (bad archive) ls-s-works cpio lists hardlinks ls-l-works sed n (flushes pattern space terminates early) wget-handles-empty-path hostname-d-works sed append autoinserts newline sed embedded NUL wget-supports--P sed NUL in command sed nonexistent label wget--O-overrides--P unzip (subdir only) Retested and passed, you can verify https://wiki.yoctoproject.org/wiki/Ptest_296dfbc9b3a3c0ed64b8e748c2868928b9c2aa8b Chen Qi, Please place this one to "Resolve" to be able to put in "Verified". Not reproduced again. Tested wit commit 4b94b498e21aeba945fe7e72a6b7c4bb0314fb83 on NUC and still seeing 7 failures hence reopening this bug **** Failures ******** FAIL:cpio lists hardlinks FAIL:ls-l-works FAIL:ls-s-works FAIL:sed embedded NUL FAIL:sed NUL in command FAIL:sed append autoinserts newline FAIL:sed nonexistent label ************************* https://wiki.yoctoproject.org/wiki/Ptest_4b94b498e21aeba945fe7e72a6b7c4bb0314fb83 Full log: https://wiki.yoctoproject.org/wiki/images/2/26/Ptestlog2.2_M4.txt Image : http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.iso 2.3 M1 has failing tests at least when run on a machine without an internet connection. https://wiki.yoctoproject.org/wiki/images/8/81/Logs_2.3.M1.rc1.txt What's worse, the tests timeout and that confuses the testrunner: it starts other tests and then at some point the busybox tests start spewing their results in the middle of some other recipes results. See comments on bug 10842. Tests were executed on 2.3 M3 rc2 observing 8 failures on the results: ********** Failures ******* FAIL: cpio lists hardlinks FAIL: hostname-d-works FAIL: ls-l-works FAIL: ls-s-works FAIL: sed embedded NUL FAIL: sed NUL in command FAIL: sed append autoinserts newline FAIL: sed nonexistent label *************************** Results: https://wiki.yoctoproject.org/wiki/Ptest_415b72ffcbd26e5f3664370d8b2a9b8105fb6342 |