Bug 13465

Summary: [QA 2.8 M2 RC1] quilt and libevent failure in ptest
Product: [QA/Testing] Package Testing (ptest) Reporter: Sudhir Sharma <sharmasudhir>
Component: ptestAssignee: Trevor Gamblin <tvgamblin>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: randy.macleod, richard.purdie
Version: 0.0.0   
Target Milestone: 2.8 M3   
Hardware: x86   
OS: Multiple   
Whiteboard: NEWCOMER
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
ptest regression results
none
current ptest report
none
Previous ptest results
none
quilt log
none
libevent log none

Description Sudhir Sharma 2019-08-01 03:47:43 UTC 
Created attachment 4544 [details]
ptest regression results

We took ptest results from autobuilder and did ptest regression. 

1. Download ptest results for current and previous release.
2. Run resulttool regression for ptest
3. Identify test that passed previous release and failed  this release

Previous release 
https://autobuilder.yocto.io/pub/releases/yocto-2.8_M1.rc2/testresults/qemux86-64-ptest/

Current release
https://autobuilder.yocto.io/pub/releases/yocto-2.8_M2.rc1/testresults/qemux86-64-ptest/

We have observed failure in ptest regression results.

Quilt
ptestresult.quilt.remember-locations.test: FAILED -> PASSED
ptestresult.quilt.project-settings.test: FAILED -> PASSED
ptestresult.quilt.nolink.test: FAILED -> PASSED
ptestresult.quilt.import.test: FAILED -> PASSED
ptestresult.quilt.import2.test: FAILED -> PASSED
ptestresult.quilt.add-filename-check.test: FAILED -> PASSED

libevent
ptestresult.libevent.libevent: FAILED -> PASSED

Attaching failure log for quilt and libevent.
Comment 1 Sudhir Sharma 2019-08-01 03:48:56 UTC 
Created attachment 4545 [details]
current ptest report
Comment 2 Sudhir Sharma 2019-08-01 03:49:26 UTC 
Created attachment 4546 [details]
Previous ptest results
Comment 3 Sudhir Sharma 2019-08-01 03:49:59 UTC 
Created attachment 4547 [details]
quilt log
Comment 4 Sudhir Sharma 2019-08-01 03:50:30 UTC 
Created attachment 4548 [details]
libevent log
Comment 5 Sudhir Sharma 2019-08-01 06:40:34 UTC 
We have re-run the quilt and libevent module and found the same issue re-ocurring. (attached logs)
Comment 6 Randy MacLeod 2019-08-15 14:57:07 UTC 
libevent has been updated so check it again.
Comment 7 Trevor Gamblin 2019-08-23 13:35:04 UTC 
libevent works after update.

quilt patches at:

https://patchwork.openembedded.org/patch/164208/
https://patchwork.openembedded.org/patch/164209/
Comment 9 Randy MacLeod 2019-09-02 20:29:26 UTC 
https://autobuilder.yocto.io/pub/non-release/20190829-10/testresults/testresult-report.txt

Recipe                       | Passed      | Failed   | Skipped   | Time(s)   

qemuarm64
libevent                     | 0           | 1        | 0         | 117
quilt                        | 60          | 0        | 0         | 49

qemux86-64
libevent                     | 0           | 1        | 0         | 116
quilt                        | 60          | 0        | 0         | 29

There is still one failed ptest for libevent.
Comment 10 Trevor Gamblin 2019-09-04 12:44:26 UTC 
Unable to recreate the libevent failures. Is more detail on the autobuilder configuration available?
Comment 12 Trevor Gamblin 2019-09-12 15:09:31 UTC 
Submitted patches https://patchwork.openembedded.org/patch/164784/ and https://patchwork.openembedded.org/patch/164806/ to increase ptest granularity and show that libevent isn't outright failing. Closing.