Bug 12143 - Ptest-runner does not run python, in 2.4rc1
Summary: Ptest-runner does not run python, in 2.4rc1
Status: RESOLVED FIXED
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.4
Hardware: NUC x86_64
: High normal
Target Milestone: 2.4 M4
Assignee: Juro Bystricky
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-09-29 20:37 UTC by oscar lopez arandas
Modified: 2017-10-10 21:25 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
2.4.rc1.log (1.37 MB, application/octet-stream)
2017-09-29 20:37 UTC, oscar lopez arandas
no flags Details
ptest_resullts.png (813.87 KB, image/png)
2017-10-05 17:48 UTC, oscar lopez arandas
no flags Details
full_log (5.32 MB, text/x-log)
2017-10-10 21:25 UTC, oscar lopez arandas
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description oscar lopez arandas 2017-09-29 20:37:12 UTC
Created attachment 4038 [details]
2.4.rc1.log

python does not run anything in the ptest-runner automatic test, nor does it do it manually with ptest-runner python.
 
poky : 592c3449697ce39bf07864fafd7f0a61ca230561
2.4 rc1 
HW: NUC  6i5SYH 
image:
https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc1/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg

https://wiki.yoctoproject.org/wiki/Ptest_592c3449697ce39bf07864fafd7f0a61ca230561
STEPS: automatic
 1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)

* we test it on NUC with genericx86-64

2. Install the image on DUT (using legacy boot)

3) Configure the network so it be able to work externally:

- Export the proxy using "export http_proxy=<add your proxy link>" command eg
$ export http_proxy=http://proxy-chain.intel.com:911

And check if you have internet connection, typing on terminal:
$ wget www.google.com

4. In command line type
$ ptest-runner > ptest.log 

manually:

I can you run the tests manually, i.e.
# ptest-runner python


EXPECTED OUTCOME:
python run correctly

ACTUAL OUTCOME:
================================================================================
BEGIN: /usr/lib/python/ptest
find /usr/lib/python2.7 -name '*.py[co]' -print | xargs rm -f
python -Wd -3 -E -tt  /usr/lib/python2.7/test/regrtest.py -l -v 
python -Wd -3 -E -tt  /usr/lib/python2.7/test/regrtest.py -l -v 
END: /usr/lib/python/ptest
============================================================================
NOTES:
full log in attachment
Comment 1 oscar lopez arandas 2017-10-05 17:47:39 UTC
Recently it was discovered that before 2.4 rc1  ptest analysis and comparisons are not reliable due to a bug in the script that was used. 
I am attaching the new results obtain by comparing again the results of the ptest execution. 
2.3 rc1  2.4 M1
2.4 m1  2.4 m2 rc1
2.4 m2 rc1  2.4 m2rc3
2.4 m2 rc3 2.4 m3
2.4 m3  2.4 rc1

you will clearly see that python stoped being executed on 2.4 m2 rc3
Comment 2 oscar lopez arandas 2017-10-05 17:48:32 UTC
Created attachment 4055 [details]
ptest_resullts.png
Comment 3 Juro Bystricky 2017-10-05 18:21:17 UTC
python-ptest needs to have python test suite installed. The suite is in the package python-tests, so this package needs to be installed in the image.
The fix is simple, just add {PN}-tests into RDEPENDS.

 RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests"

With new image built with the updated recipe for python_2.7.13.bb I was able to run core-image-sato-sdk-ptest in qemu:

# ptest-runner python

with the results:

PASS: 6632
FAIL:26
SKIP: 28
Comment 4 oscar lopez arandas 2017-10-10 21:24:13 UTC
python does present run in the ptest-runner automatic test

POKY : 2.4_M4_r2 COMMIT:da05a01423a60a23cd1073eb0ad3776e5bf4f2a8
HW: NUC  6i5SYH 
IMAGES:
https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc2/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg

Results  Chart table:
https://wiki.yoctoproject.org/wiki/Ptest_da05a01423a60a23cd1073eb0ad3776e5bf4f2a8

EXPECTED OUTCOME:
python run correctly

ACTUAL OUTCOME:
python run correctly
 
NOTES:
full log in attachment
Comment 5 oscar lopez arandas 2017-10-10 21:25:04 UTC
Created attachment 4070 [details]
full_log