Bug 13765 - [QA 3.1 M2 RC1] strace failure in ptest
Summary: [QA 3.1 M2 RC1] strace failure in ptest
Status: RESOLVED FIXED
Alias: None
Product: Package Testing (ptest)
Classification: QA/Testing
Component: ptest (show other bugs)
Version: 3.1
Hardware: x86 Multiple
: High normal
Target Milestone: 3.1 M3
Assignee: Randy MacLeod
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2020-01-31 03:00 UTC by Sangeeta Jain
Modified: 2020-02-06 23:07 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
overall ptest regression report - 3.1 M2 RC1 Vs 3.1 M1 RC8 (441.63 KB, text/plain)
2020-01-31 03:00 UTC, Sangeeta Jain
no flags Details
details of 257 strace ptest failure (15.63 KB, application/octet-stream)
2020-01-31 03:01 UTC, Sangeeta Jain
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sangeeta Jain 2020-01-31 03:00:05 UTC
Created attachment 4623 [details]
overall ptest regression report - 3.1 M2 RC1 Vs 3.1 M1 RC8

previous release:

https://autobuilder.yocto.io/pub/releases/yocto-3.1_M1.rc8/testresults/qemux86-64-ptest/testresults.json

Current release:

https://autobuilder.yocto.io/pub/releases/yocto-3.1_M2.rc1/testresults/qemux86-64-ptest/testresults.json


Regression results shows 257 failures in strace ptest.

Attached :

1. Regression results
2. Details of strace failures
Comment 1 Sangeeta Jain 2020-01-31 03:01:13 UTC
Created attachment 4624 [details]
details of 257 strace ptest failure
Comment 2 Khem Raj 2020-02-05 01:31:00 UTC
There has been a bunch of fixes since 5.4 release which is relevant to us see [1]

especially glibc 2.31 and kernel 5.5/syscall related. I wonder if we could try
a testrun with strace master and see if these are fixed.

[1] https://github.com/strace/strace/blob/master/NEWS#L4-L28
Comment 3 Richard Purdie 2020-02-06 15:38:06 UTC
We believe fix came in after M2, current test report looks ok: https://autobuilder.yocto.io/pub/non-release/20200204-12/testresults/testresult-report.txt
Comment 4 Randy MacLeod 2020-02-06 16:35:20 UTC
There are no failures in current autobuilder tests:

--------------------------------------------------------------------------
Recipe                       | Passed       | Failed    | Skipped     | Time(s)   
--------------------------------------------------------------------------
qemux86-64
strace                       | 560          | 0         | 161         | 540
--------------------------------------------------------------------------
qemuarm64
strace                       | 515          | 0         | 206         | 726

but I'll double check and get the commit id that fixed this.
Also there are some odd warnings issued by the kernel(?) that I should create another defect to track if I still see them when testing.
Comment 5 Randy MacLeod 2020-02-06 23:07:49 UTC
Fixed by: http://git.openembedded.org/openembedded-core/commit/?id=3d6bf58c7080c1cacf3ed1f270ff5acf4858c790

Most recent manual ptest with core-image-minimal qemux86-64/kvm:

============================================================================
Testsuite summary for strace 5.4
============================================================================
# TOTAL: 721
# PASS:  562
# SKIP:  159
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make: Leaving directory '/usr/lib/strace/ptest/tests'
DURATION: 590

The warnings are tracked by a new defect:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13785