Bug 8831 - pTest - busybox tests fail for 2.1_M1.rc1
Summary: pTest - busybox tests fail for 2.1_M1.rc1
Status: RESOLVED DUPLICATE of bug 5697
Alias: None
Product: Runtime Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.1
Hardware: NUC Multiple
: Medium normal
Target Milestone: Future
Assignee: dengke
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-12-17 10:04 UTC by Alexandru Georgescu
Modified: 2018-04-24 04:10 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandru Georgescu 2015-12-17 10:04:49 UTC
Commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
Previous Commit: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1
https://wiki.yoctoproject.org/wiki/Ptest_5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e


Results for NUC ptest busybox 2015-12-10:
********************************
Tests that failed now and worked last time: 8
unzip (bad archive)
echo-prints-slash_0041
echo-prints-slash-zero
echo-prints-dash
echo-prints-slash_41
echo-prints-non-opts
echo-prints-slash_041
echo-prints-slash_00041
Tests that passed now and and failed last time: 2
ls-s-works
ls-l-works
Tests that were skipped now and worked last time: 1
bzcat
Tests that were skipped last time and worked now: 20
dc basic syntax (stdin multiple args)
glibc build sort
diff -B does not ignore non-blank single line change
dc complex with spaces (multiple args)
unzip (bad archive)
sed /regex/N{...} addresses work
sed /regex/+N{...} addresses work
glibc build sort unique
awk length()
dc complex without spaces (multiple args)
sed /regex/+0 -i wo rks
diff of empty file against stdin
dc complex without spaces (single arg)
sed /regex/+N{...} -i works
sed /regex/+0{...} -i works
dc complex with spaces (single arg)
diff -B ignores blank single line change
sort with non-default leading delim 4
dc basic syntax (argv multiple args)
dc basic syntax (argv single arg)
Tests that failed now: 20
wget-retrieves-google-index
unzip (bad archive)
cpio lists hardlinks
echo-prints-slash_0041
echo-prints-slash-zero
echo-prints-dash
sed n (flushes pattern space terminates early)
wget-handles-empty-path
echo-prints-slash_41
echo-prints-non-opts
hostname-d-works
echo-prints-slash_041
sed append autoinserts newline
sed embedded NUL
wget-supports--P
sed NUL in command
echo-prints-slash_00041
sed nonexistent label
wget--O-overrides--P
unzip (subdir only)
Comment 1 Francisco Pedraza 2016-02-04 10:37:52 UTC
(In reply to comment #0)
> Commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
> Previous Commit: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1
> https://wiki.yoctoproject.org/wiki/
> Ptest_5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
> 
> 
> Results for NUC ptest busybox 2015-12-10:
> ********************************
> Tests that failed now and worked last time: 8
> unzip (bad archive)
> echo-prints-slash_0041
> echo-prints-slash-zero
> echo-prints-dash
> echo-prints-slash_41
> echo-prints-non-opts
> echo-prints-slash_041
> echo-prints-slash_00041
> Tests that passed now and and failed last time: 2
> ls-s-works
> ls-l-works
> Tests that were skipped now and worked last time: 1
> bzcat
> Tests that were skipped last time and worked now: 20
> dc basic syntax (stdin multiple args)
> glibc build sort
> diff -B does not ignore non-blank single line change
> dc complex with spaces (multiple args)
> unzip (bad archive)
> sed /regex/N{...} addresses work
> sed /regex/+N{...} addresses work
> glibc build sort unique
> awk length()
> dc complex without spaces (multiple args)
> sed /regex/+0 -i wo rks
> diff of empty file against stdin
> dc complex without spaces (single arg)
> sed /regex/+N{...} -i works
> sed /regex/+0{...} -i works
> dc complex with spaces (single arg)
> diff -B ignores blank single line change
> sort with non-default leading delim 4
> dc basic syntax (argv multiple args)
> dc basic syntax (argv single arg)
> Tests that failed now: 20
> wget-retrieves-google-index
> unzip (bad archive)
> cpio lists hardlinks
> echo-prints-slash_0041
> echo-prints-slash-zero
> echo-prints-dash
> sed n (flushes pattern space terminates early)
> wget-handles-empty-path
> echo-prints-slash_41
> echo-prints-non-opts
> hostname-d-works
> echo-prints-slash_041
> sed append autoinserts newline
> sed embedded NUL
> wget-supports--P
> sed NUL in command
> echo-prints-slash_00041
> sed nonexistent label
> wget--O-overrides--P
> unzip (subdir only)


Retested on NUC 5425WYK genericx86_64, 2016-01-28:

poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
AB Link: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg

Report: https://wiki.yoctoproject.org/wiki/Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
compared with commit 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e

Results for NUC ptest quilt 2016-01-28:
********************************
Tests that failed now and worked last time: 0

Tests that passed now and and failed last time: 0

Tests that were skipped now and worked last time: 0

Tests that were skipped last time and worked now: 0
Tests that failed now: 1
faildiff.test
Comment 2 Francisco Pedraza 2016-02-04 10:39:40 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > Commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
> > Previous Commit: fc45deac89ef63ca1c44e763c38ced7dfd72cbe1
> > https://wiki.yoctoproject.org/wiki/
> > Ptest_5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
> > 
> > 
> > Results for NUC ptest busybox 2015-12-10:
> > ********************************
> > Tests that failed now and worked last time: 8
> > unzip (bad archive)
> > echo-prints-slash_0041
> > echo-prints-slash-zero
> > echo-prints-dash
> > echo-prints-slash_41
> > echo-prints-non-opts
> > echo-prints-slash_041
> > echo-prints-slash_00041
> > Tests that passed now and and failed last time: 2
> > ls-s-works
> > ls-l-works
> > Tests that were skipped now and worked last time: 1
> > bzcat
> > Tests that were skipped last time and worked now: 20
> > dc basic syntax (stdin multiple args)
> > glibc build sort
> > diff -B does not ignore non-blank single line change
> > dc complex with spaces (multiple args)
> > unzip (bad archive)
> > sed /regex/N{...} addresses work
> > sed /regex/+N{...} addresses work
> > glibc build sort unique
> > awk length()
> > dc complex without spaces (multiple args)
> > sed /regex/+0 -i wo rks
> > diff of empty file against stdin
> > dc complex without spaces (single arg)
> > sed /regex/+N{...} -i works
> > sed /regex/+0{...} -i works
> > dc complex with spaces (single arg)
> > diff -B ignores blank single line change
> > sort with non-default leading delim 4
> > dc basic syntax (argv multiple args)
> > dc basic syntax (argv single arg)
> > Tests that failed now: 20
> > wget-retrieves-google-index
> > unzip (bad archive)
> > cpio lists hardlinks
> > echo-prints-slash_0041
> > echo-prints-slash-zero
> > echo-prints-dash
> > sed n (flushes pattern space terminates early)
> > wget-handles-empty-path
> > echo-prints-slash_41
> > echo-prints-non-opts
> > hostname-d-works
> > echo-prints-slash_041
> > sed append autoinserts newline
> > sed embedded NUL
> > wget-supports--P
> > sed NUL in command
> > echo-prints-slash_00041
> > sed nonexistent label
> > wget--O-overrides--P
> > unzip (subdir only)
> 
> 
> Retested on NUC 5425WYK genericx86_64, 2016-01-28:
> 
> poky: 3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
> AB Link:
> http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/ptest/
> machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg
> 
> Report:
> https://wiki.yoctoproject.org/wiki/
> Ptest_3d2c0f5902cacf9d8544bf263b51ef0dd1a7218c
> compared with commit 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
> 
> Results for NUC ptest quilt 2016-01-28:
> ********************************
> Tests that failed now and worked last time: 0
> 
> Tests that passed now and and failed last time: 0
> 
> Tests that were skipped now and worked last time: 0
> 
> Tests that were skipped last time and worked now: 0
> Tests that failed now: 1
> faildiff.test

By mistake I added wrong results information, here you have the correct ones:
Results for NUC ptest busybox 2016-01-28:
********************************
Tests that failed now and worked last time: 2
ls-s-works
ls-l-works
Tests that passed now and and failed last time: 7
echo-prints-slash_0041
echo-prints-slash-zero
echo-prints-dash
echo-prints-slash_41
echo-prints-non-opts
echo-prints-slash_041
echo-prints-slash_00041
Tests that were skipped now and worked last time: 0

Tests that were skipped last time and worked now: 0
Tests that failed now: 15
wget-retrieves-google-index
unzip (bad archive)
ls-s-works
cpio lists hardlinks
ls-l-works
sed n (flushes pattern space terminates early)
wget-handles-empty-path
hostname-d-works
sed append autoinserts newline
sed embedded NUL
wget-supports--P
sed NUL in command
sed nonexistent label
wget--O-overrides--P
unzip (subdir only)
Comment 3 Francisco Pedraza 2016-04-15 16:03:25 UTC
Retested and passed, you can verify
https://wiki.yoctoproject.org/wiki/Ptest_296dfbc9b3a3c0ed64b8e748c2868928b9c2aa8b
Chen Qi, Please place this one to "Resolve" to be able to put in "Verified".
Comment 4 Chen Qi 2016-04-18 01:11:11 UTC
Not reproduced again.
Comment 5 Jose Perez C 2016-10-20 19:19:28 UTC
Tested wit commit 4b94b498e21aeba945fe7e72a6b7c4bb0314fb83 on NUC and still seeing 7 failures  hence reopening this bug 

**** Failures ********

FAIL:cpio lists hardlinks
FAIL:ls-l-works
FAIL:ls-s-works
FAIL:sed embedded NUL
FAIL:sed NUL in command
FAIL:sed append autoinserts newline
FAIL:sed nonexistent label

*************************

https://wiki.yoctoproject.org/wiki/Ptest_4b94b498e21aeba945fe7e72a6b7c4bb0314fb83

Full log: 
https://wiki.yoctoproject.org/wiki/images/2/26/Ptestlog2.2_M4.txt

Image : 
http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.iso
Comment 6 Jussi Kukkonen 2016-12-22 16:57:55 UTC
2.3 M1 has failing tests at least when run on a machine without an internet connection.

https://wiki.yoctoproject.org/wiki/images/8/81/Logs_2.3.M1.rc1.txt

What's worse, the tests timeout and that confuses the testrunner: it starts other tests and then at some point the busybox tests start spewing their results in the middle of some other recipes results. See comments on bug 10842.
Comment 7 Jair Gonzalez 2017-04-06 19:51:23 UTC
Tests were executed on 2.3 M3 rc2 observing 8 failures on the results:

********** Failures *******

FAIL: cpio lists hardlinks
FAIL: hostname-d-works
FAIL: ls-l-works
FAIL: ls-s-works
FAIL: sed embedded NUL
FAIL: sed NUL in command
FAIL: sed append autoinserts newline
FAIL: sed nonexistent label

***************************

Results:
https://wiki.yoctoproject.org/wiki/Ptest_415b72ffcbd26e5f3664370d8b2a9b8105fb6342
Comment 8 Jair Gonzalez 2017-04-06 20:08:07 UTC
Marking issue as duplicate of 5697.

*** This bug has been marked as a duplicate of bug 5697 ***