Bug 12146

Summary: ptest-e2fsprog cases failed in 2.4rc1
Product: [QA/Testing] Build Testing Reporter: oscar lopez arandas <oscarx.lopez.arandas>
Component: generalAssignee: Juro Bystricky <juro.bystricky>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: jose.perez.carranza
Version: 2.4   
Target Milestone: 2.4 M4   
Hardware: NUC   
OS: x86_64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
full-log
none
log file of e2fsprogs testsuite.
none
2.4_M4_rc2 none

Description oscar lopez arandas 2017-09-29 21:47:37 UTC 
Created attachment 4041 [details]
full-log

tests were executed and The pass rate goes back the percentage, observing 1 failures on the -e2fsprog 

poky : 592c3449697ce39bf07864fafd7f0a61ca230561
2.4 rc1 
HW: NUC  6i5SYH 
image:
https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc1/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg

https://wiki.yoctoproject.org/wiki/Ptest_592c3449697ce39bf07864fafd7f0a61ca230561
STEPS: automatic
 1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)

* we test it on NUC with genericx86-64

2. Install the image on DUT (using legacy boot)

3) Configure the network so it be able to work externally:

- Export the proxy using "export http_proxy=<add your proxy link>" command eg


And check if you have internet connection, typing on terminal:
$ wget www.google.com

4. In command line type
$ ptest-runner > ptest.log 

EXPECTED OUTCOME:
The pass rate goes up the percentage


ACTUAL OUTCOME:
The pass rate goes back the percentage
==========================================
BEGIN: /usr/lib/e2fsprogs/ptest
FAIL: e2fsprogs
END: /usr/lib/e2fsprogs/ptest
=============================================
full log in attachment
Comment 1 Juro Bystricky 2017-10-06 20:46:35 UTC 
Created attachment 4063 [details]
log file of e2fsprogs testsuite.
Comment 2 Juro Bystricky 2017-10-06 20:47:03 UTC 
e2fsprog test suit logs everything into its own log file /usr/lib/e2fsprogs/ptest/test.log, so no test output actually makes it into the "main" ptest-log file.
So the evaluation script counts it as a single test.
So any comparison with previous results is meaningless, as no-one analyzed the results in test.log file. Pleas see a sample attached, at the bottom it says: 

258 tests succeeded	51 tests failed.

The remedy would be to ensure the e2fsprogs test output makes it into ptest-log file and also performing spme kind of conversion of "ok" and "failed" to more canonical "PASS:" FAIL:"

Of course, the individual tests that fail should be analyzed for causes.
Comment 3 oscar lopez arandas 2017-10-10 21:54:28 UTC 
tests were executed and The pass rate goes up the percentage, observing 44 failures on the e2fsprog 

POKY : 2.4_M4_r2 COMMIT:da05a01423a60a23cd1073eb0ad3776e5bf4f2a8
HW: NUC  6i5SYH 
IMAGES:
https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc2/machines/genericx86-64/core-image-sato-sdk-ptest-genericx86-64.hddimg

Results  Chart table:
https://wiki.yoctoproject.org/wiki/Ptest_da05a01423a60a23cd1073eb0ad3776e5bf4f2a8

EXPECTED OUTCOME:
The pass rate goes up the percentage


ACTUAL OUTCOME:
The pass rate goes up the percentage
==========================================
FAIL: d_fallocate: fallocate sparse files and big files: failed
FAIL: d_fallocate_bigalloc: fallocate sparse files and big files with bigalloc: failed
FAIL: d_fallocate_blkmap: fallocate sparse files and big files on a blockmap fs: failed
FAIL: d_punch: punch sparse files and big files: failed
FAIL: d_punch_bigalloc: punch sparse files and big files with bigalloc: failed
=============================================
full log in attachment
Comment 4 oscar lopez arandas 2017-10-10 21:55:13 UTC 
Created attachment 4074 [details]
2.4_M4_rc2