Created attachment 4591 [details] ptest regression results ptest regression results We took ptest results from autobuilder and did ptest regression. 1. Download ptest results for current and previous release. 2. Run resulttool regression for ptest 3. Identify test that passed previous release and failed this release Previous release https://autobuilder.yocto.io/pub/releases/yocto-3.0.rc2/testresults/qemux86-64-ptest/ Current release https://autobuilder.yocto.io/pub/releases/yocto-3.0.1.rc3/testresults/qemux86-64-ptest/ We have observed failure in ptest regression results. Valgrid ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED
Created attachment 4592 [details] ptest resulttool 3.0.1 RC3
this is the command I ran: resulttool regression https://autobuilder.yocto.io/pub/releases/yocto-3.0.1.rc3/testresults/qemux86-64-ptest/testresults.json https://autobuilder.yocto.io/pub/releases/yocto-3.0.rc2/testresults/qemux86-64-ptest/testresults.json I see: ptestresult.gstreamer1.0.gst/gstsystemclock: PASSED -> FAILED I take that to mean it is now passed but failed in the initial release. anything with: ptestresult.python3.test_value_extra_dquote__test.test_email.test__header_value_parser.Test_parse_mime_parameters_: PASSED -> None is no change and ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED it is now failing Am I understanding the results correctly?
This is failing in 3.1 M1 too
a patch has been posted to help in some areas. http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=a3bbdc6bc687f39220941f08422bdf94435bd094
In oe-core as of: 9031b66c7c (HEAD -> master, origin/master, origin/HEAD) grep: Fix shell after autotools changes There are the following failures: $ grep FAIL /tmp/valgrind-ptest-jan-28.log FAIL: gdbserver_tests/hgtls FAIL: memcheck/tests/linux/with-space FAIL: memcheck/tests/overlap FAIL: drd/tests/pth_detached3 FAIL: drd/tests/std_list FAIL: none/tests/fdleak_cmsg FAILED: 6 so Alex's improvements did not fix: ptestresult.valgrind.drd/tests/std_list: FAILED -> PASSED I'll start debugging this failure and likely others tomorrow.
I have done additional fixing so that list of failures is already outdated :) The latest ptest results are here: x86_64: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/667/steps/8/logs/step1c arm64: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/448/steps/8/logs/step1c So you need to replace Khem's patch for drd/tests/pth_detached3 with one from upstream (which also adjusts the expected test outcome), and look into hgtls failure on arm, and then we're sorted.
Thanksk Alex. I was trying to determine when the patch that I added stopped being needed but I guess there's no point. I'll leave this defect open to track the remaining valgrind ptest problems.
Patch sent to oe-core for backported commit.
DRD ptest bugs fixed as of https://git.openembedded.org/openembedded-core/commit/?id=6a74caa115298e594ae22a9de91b132db62e4b5d