Bug 13666

Summary: [QA 3.0.1 RC3] valgrind.drd failure in ptest
Product: [QA/Testing] Package Testing (ptest) Reporter: Sudhir Sharma <sharmasudhir>
Component: ptestAssignee: Stacy Gaikovaia <stacy.gaikovaia>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: akuster808, alex.kanavin, randy.macleod, stacy.gaikovaia
Version: 3.1   
Target Milestone: 3.2 M4   
Hardware: x86   
OS: x86_64   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13838
Whiteboard: Backport to 3.0.2
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
ptest regression results
none
ptest resulttool 3.0.1 RC3 none

Description Sudhir Sharma 2019-12-03 04:03:55 UTC 
Created attachment 4591 [details]
ptest regression results

ptest regression results

We took ptest results from autobuilder and did ptest regression. 

1. Download ptest results for current and previous release.
2. Run resulttool regression for ptest
3. Identify test that passed previous release and failed  this release

Previous release 
https://autobuilder.yocto.io/pub/releases/yocto-3.0.rc2/testresults/qemux86-64-ptest/

Current release
https://autobuilder.yocto.io/pub/releases/yocto-3.0.1.rc3/testresults/qemux86-64-ptest/

We have observed failure in ptest regression results.

Valgrid
ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED
Comment 1 Sudhir Sharma 2019-12-03 04:05:02 UTC 
Created attachment 4592 [details]
ptest resulttool 3.0.1 RC3
Comment 2 Armin Kuster 2019-12-03 15:45:10 UTC 
this is the command I ran:

resulttool regression https://autobuilder.yocto.io/pub/releases/yocto-3.0.1.rc3/testresults/qemux86-64-ptest/testresults.json https://autobuilder.yocto.io/pub/releases/yocto-3.0.rc2/testresults/qemux86-64-ptest/testresults.json


I see:

ptestresult.gstreamer1.0.gst/gstsystemclock: PASSED -> FAILED

I take that to mean it is now passed but failed in the initial release.

anything with:
ptestresult.python3.test_value_extra_dquote__test.test_email.test__header_value_parser.Test_parse_mime_parameters_: PASSED -> None

is no change

and
ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED

it is now failing

Am I understanding the results correctly?
Comment 3 akuster 2019-12-18 16:22:28 UTC 
This is failing in 3.1 M1 too
Comment 4 akuster 2020-01-16 16:32:13 UTC 
a patch has been posted to help in some areas.

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=a3bbdc6bc687f39220941f08422bdf94435bd094
Comment 5 Randy MacLeod 2020-01-29 02:04:06 UTC 
In oe-core as of:
9031b66c7c (HEAD -> master, origin/master, origin/HEAD) grep: Fix shell after autotools changes

There are the following failures:
$ grep FAIL /tmp/valgrind-ptest-jan-28.log 
FAIL: gdbserver_tests/hgtls
FAIL: memcheck/tests/linux/with-space
FAIL: memcheck/tests/overlap
FAIL: drd/tests/pth_detached3
FAIL: drd/tests/std_list
FAIL: none/tests/fdleak_cmsg
FAILED: 6

so Alex's improvements did not fix:
   ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED

I'll start debugging this failure and likely others tomorrow.
Comment 6 Alexander Kanavin 2020-02-04 19:34:48 UTC 
I have done additional fixing so that list of failures is already outdated :)

The latest ptest results are here:
x86_64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/667/steps/8/logs/step1c

arm64:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/448/steps/8/logs/step1c

So you need to replace Khem's patch for drd/tests/pth_detached3 with one from upstream (which also adjusts the expected test outcome), and look into hgtls failure on arm, and then we're sorted.
Comment 7 Randy MacLeod 2020-02-04 20:05:34 UTC 
Thanksk Alex. I was trying to determine when the patch that I added stopped being needed but I guess there's no point. I'll leave this defect open to track the remaining valgrind ptest problems.
Comment 8 Randy MacLeod 2020-10-14 08:48:55 UTC 
Patch sent to oe-core for backported commit.