Bug 13765

Summary: [QA 3.1 M2 RC1] strace failure in ptest
Product: [QA/Testing] Package Testing (ptest) Reporter: Sangeeta Jain <sangeeta.jain>
Component: ptestAssignee: Randy MacLeod <randy.macleod>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: akuster808, raj.khem, randy.macleod, richard.purdie
Version: 3.1   
Target Milestone: 3.1 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
overall ptest regression report - 3.1 M2 RC1 Vs 3.1 M1 RC8
none
details of 257 strace ptest failure none

Description Sangeeta Jain 2020-01-31 03:00:05 UTC
Created attachment 4623 [details]
overall ptest regression report - 3.1 M2 RC1 Vs 3.1 M1 RC8

previous release:

https://autobuilder.yocto.io/pub/releases/yocto-3.1_M1.rc8/testresults/qemux86-64-ptest/testresults.json

Current release:

https://autobuilder.yocto.io/pub/releases/yocto-3.1_M2.rc1/testresults/qemux86-64-ptest/testresults.json


Regression results shows 257 failures in strace ptest.

Attached :

1. Regression results
2. Details of strace failures
Comment 1 Sangeeta Jain 2020-01-31 03:01:13 UTC
Created attachment 4624 [details]
details of 257 strace ptest failure
Comment 2 Khem Raj 2020-02-05 01:31:00 UTC
There has been a bunch of fixes since 5.4 release which is relevant to us see [1]

especially glibc 2.31 and kernel 5.5/syscall related. I wonder if we could try
a testrun with strace master and see if these are fixed.

[1] https://github.com/strace/strace/blob/master/NEWS#L4-L28
Comment 3 Richard Purdie 2020-02-06 15:38:06 UTC
We believe fix came in after M2, current test report looks ok: https://autobuilder.yocto.io/pub/non-release/20200204-12/testresults/testresult-report.txt
Comment 4 Randy MacLeod 2020-02-06 16:35:20 UTC
There are no failures in current autobuilder tests:

--------------------------------------------------------------------------
Recipe                       | Passed       | Failed    | Skipped     | Time(s)   
--------------------------------------------------------------------------
qemux86-64
strace                       | 560          | 0         | 161         | 540
--------------------------------------------------------------------------
qemuarm64
strace                       | 515          | 0         | 206         | 726

but I'll double check and get the commit id that fixed this.
Also there are some odd warnings issued by the kernel(?) that I should create another defect to track if I still see them when testing.
Comment 5 Randy MacLeod 2020-02-06 23:07:49 UTC
Fixed by: http://git.openembedded.org/openembedded-core/commit/?id=3d6bf58c7080c1cacf3ed1f270ff5acf4858c790

Most recent manual ptest with core-image-minimal qemux86-64/kvm:

============================================================================
Testsuite summary for strace 5.4
============================================================================
# TOTAL: 721
# PASS:  562
# SKIP:  159
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make: Leaving directory '/usr/lib/strace/ptest/tests'
DURATION: 590

The warnings are tracked by a new defect:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13785