OVERVIEW: After ptest is executed, it gets compared with the previous release. The script is not comparing correctly since in 2.3.2 rc1 the python test ran with a pass rate of 99+ and in the comparison with 2.3.3 rc1, the results show that passrate was 0% in 2.3.2 rc1. Compare the ptest report on both wikis. https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-20-_Full_Test_Cycle_-_2.3.3_rc1#pTest_for_genericx86-64_on_NUC https://wiki.yoctoproject.org/wiki/WW38_-_2017-09-20-_Full_Test_Cycle_-_2.3.2_rc1#pTest_for_genericx86-64_on_NUC
Created attachment 4180 [details] ptest_2.3.3.rc1_compared_2.3.2.rc1
Executed the ptest_parser.py, could not reproduce the issue. Attached the ptest result that show the python result is working as expected.
ptest comparison script is not showing correct results. For QA 2.7 M1 rc1, the ptest summary from ptest.log (https://wiki.yoctoproject.org/wiki/Ptest_fe43e599327f26b399c1883205e988236a5e51bd) show: === Test Summary === TOTAL: 579 PASSED: 316 FAILED: 251 SKIPPED: 12 END: /usr/lib/valgrind/ptest But the qa-tools/scripts/ptest-parser.py show: TOTAL (Test Ran): 567 PASSED: 304 Need to fix this an rerun ptest comparison script.
QA team was working on this.
Fixed developed by Sudhir and patch pushed to qa-tools http://git.yoctoproject.org/cgit/cgit.cgi/qa-tools/commit/?id=e1a81ebc332014b71a680df2742bf950ce542295