Bug 12836

Summary: [2.6 M1 RC1] busybox ptest failed
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Manish Das <manish.das>
Component: coreAssignee: Manish Das <manish.das>
Status: RESOLVED NOTABUG QA Contact:
Severity: normal    
Priority: Medium+ CC: andrej.v, meta.mr.watcher, meta.watcher, ross.burton
Version: 2.6   
Target Milestone: 2.6 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: Regression (Found by QA test)
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
ptest rerun for 2.6M1 RC1
none
ptest result for busybox failure in 2.6M1RC1 none

Description Manish Das 2018-07-02 05:58:25 UTC
Performed diff and found that busybox tests that passed in 2.5 M4 RC1 are failing on 2.6 M1 RC1


-+ test x = x10.221.123.60
++ test x = x10.221.123.56


---- logfile.gnu        2018-06-15 10:53:17.335972490 +0000
-+++ logfile.bb 2018-06-15 10:53:17.338972490 +0000
+--- logfile.gnu        2018-04-27 21:35:49.693630414 +0000
++++ logfile.bb 2018-04-27 21:35:49.694630414 +0000


- -rw-r--r-- 1 root root  2052 Jun  7 08:11 README
- -rw-r--r-- 1 root root   983 Jun  7 08:11 TODO
- -rwxr-xr-x 1 root root  2703 Jun  7 08:11 all_sourcecode.tests
+ -rw-r--r-- 1 root root  2052 Apr 23 18:47 README
+ -rw-r--r-- 1 root root   983 Apr 23 18:47 TODO
+ -rwxr-xr-x 1 root root  2703 Apr 23 18:47 all_sourcecode.tests


---- logfile.gnu        2018-06-15 10:53:17.354972489 +0000
-+++ logfile.bb 2018-06-15 10:53:17.356972489 +0000
+--- logfile.gnu        2018-04-27 21:35:49.698630414 +0000
++++ logfile.bb 2018-04-27 21:35:49.699630414 +0000


-echo -ne '' | pidof pidof.tests | grep -o -w 18682
+echo -ne '' | pidof pidof.tests | grep -o -w 31361


-FAIL: wget--O-overrides--P
+PASS: wget--O-overrides--P

-FAIL: wget-handles-empty-path
+PASS: wget-handles-empty-path

-FAIL: wget-retrieves-google-index
+PASS: wget-retrieves-google-index


-FAIL: wget-supports--P
+PASS: wget-supports--P
Comment 1 Ross Burton 2018-07-05 14:47:42 UTC
Andrej, will you be able to look at this?  That it is solely wget tests all failing suggests to me that either:

1) the QA had a transient network failure or
2) we're missing some dependency, or maybe the real wget is/was being used

First step would be running the busybox ptest in a core-image-minimal to see what happens with current master.
Comment 2 Andrej Valek 2018-07-08 17:39:54 UTC
(In reply to comment #1)
> Andrej, will you be able to look at this?  That it is solely wget tests all
> failing suggests to me that either:
> 
> 1) the QA had a transient network failure or
> 2) we're missing some dependency, or maybe the real wget is/was being used
> 
> First step would be running the busybox ptest in a core-image-minimal to see
> what happens with current master.

Yes, I will take a look on it. First I will build the mentioned core-image-minimal and run ptest there.
Comment 3 Andrej Valek 2018-07-09 08:07:04 UTC
I have tested it with core-image-minimal.

FAIL: wget--O-overrides--P
+ test x '!=' x
+ mkdir foo
+ busybox wget -q -O index.html -P foo http://www.google.com/
wget: bad address 'www.google.com'
FAIL: wget-handles-empty-path
+ test x '!=' x
+ busybox wget http://www.google.com
wget: bad address 'www.google.com'
FAIL: wget-retrieves-google-index
+ test x '!=' x
+ busybox wget -q -O foo http://www.google.com/
wget: bad address 'www.google.com'
FAIL: wget-supports--P
+ test x '!=' x
+ mkdir foo
+ busybox wget -q -P foo http://www.google.com/
wget: bad address 'www.google.com'

It failed, because of missing connection/routing to internet. So I have added proxy routing via http/s_proxy variables. After that, it works, without any problems.

PASS: wget--O-overrides--P
PASS: wget-handles-empty-path
PASS: wget-retrieves-google-index
PASS: wget-supports--P

So the questions is, how the connection is set-up on testing machine.
Comment 4 Manish Das 2018-07-09 10:07:19 UTC
The connection is set up following the steps in the wiki:
https://wiki.yoctoproject.org/wiki/BSP_Test_Plan#Test_Process_2

I can run the ptest again if a network issue has caused this failure and update the result of the re-run.
Comment 5 Ross Burton 2018-07-09 11:04:31 UTC
Manish: please do rerun the test, as it appears that this was due to either a transient network problem, or bad configuration.
Comment 6 Manish Das 2018-07-10 10:26:00 UTC
Created attachment 4343 [details]
ptest rerun for 2.6M1 RC1

ptest rerun for 2.6M1 RC1 (10-Jul-18)
Comment 7 Manish Das 2018-07-10 10:30:37 UTC
Hi Ross, as suggested, we ran the ptest for 2.6 m1 rc1 without any intervention. The issue is reproducible. Logs are attached from today's run.

*****************************************************************
FAIL: wget--O-overrides--P
+ test x '!=' x
+ mkdir foo
+ busybox wget -q -O index.html -P foo http://www.google.com/
wget: can't connect to remote host (172.217.31.196): Connection timed out
FAIL: wget-handles-empty-path
+ test x '!=' x
+ busybox wget http://www.google.com
Connecting to www.google.com (172.217.31.196:80)
wget: can't connect to remote host (172.217.31.196): Connection timed out
FAIL: wget-retrieves-google-index
+ test x '!=' x
+ busybox wget -q -O foo http://www.google.com/
wget: can't connect to remote host (172.217.31.196): Connection timed out
FAIL: wget-supports--P
+ test x '!=' x
+ mkdir foo
+ busybox wget -q -P foo http://www.google.com/
wget: can't connect to remote host (216.58.196.4): Connection timed out
PASS: which-uses-default-path
PASS: xargs-works
*****************************************************************
Comment 8 Ross Burton 2018-07-10 10:37:13 UTC
And Andrej has just demonstrated that it works for him.  Please verify that connectivity works normally in the image before blaming the test.
Comment 9 Andrej Valek 2018-07-12 09:05:24 UTC
I have checkout tag '2.6_M1' today. It works without any problems. 

So, please verify your connection before ptest running.
$ export http_proxy="proxy":"port"
$ wget http://google.com
If the google server is not resolved, you have to set-up a dns server (/etc/resolv.conf).

When you successfully download a index.html, you can try to run ptest.
You don't need to wait for whole ptest. You can run only the busybox, $ ptest-runner busybox
Comment 10 Manish Das 2018-07-13 04:01:26 UTC
Created attachment 4345 [details]
ptest result for busybox failure in 2.6M1RC1
Comment 11 Manish Das 2018-07-13 04:05:10 UTC
Hi Andrej, after setting up the proxy 
(export http_proxy=http://proxy.png.intel.com:911) and running only the ptest for busybox ($ ptest-runner busybox), the issue got resolved.

PASS: wget--O-overrides--P
PASS: wget-handles-empty-path
PASS: wget-retrieves-google-index
PASS: wget-supports--P

Logs attached for reference. Closing the bug.
Comment 12 Manish Das 2018-07-13 04:06:45 UTC
Workaround suggested resolved the issue.