12181 2017-10-04 13:22:36 +0000 QA: Archive ptest results in git 2018-11-01 08:14:28 +0000 1 1 10 QA/Testing Build Testing general 2.4 x86 Multiple RESOLVED FIXED Medium+ enhancement 4.99 1 joshuagloe ee.peng.yeoh ee.peng.yeoh jair.de.jesus.gonzalez.plascencia liezhi.yang randy.macleod richard.purdie --- --- Don't know oldest_to_newest 77338 0 joshuagloe 2017-10-04 13:22:36 +0000 Per the test plan[1] we currently store pTest results in the wiki[2] and generate comparisons using the ptest-parser in qa-tools[3]. We're increasingly moving QA data to git and using associated scripts that can generate reports or perform comparisons on the data. Examples of this trend are buildhistory and performance data. It would be desirable to store pTest results this way also and be able to generate comparative reports based on the results archived in git. Ideally we'd have: a) a yp-qa-ptest-results (or similarly named) repo on git.yoctoproject.org b) automated execution of the ptests on a dut (triggered by QA autobuilder) c) scripts to generate reports comparing the ptest results from two builds b and c should probably have separate bugs 1. https://wiki.yoctoproject.org/wiki/BSP_Test_Plan#pTest 2. https://wiki.yoctoproject.org/wiki/Ptest/archive 3. https://wiki.yoctoproject.org/wiki/Qa-tools#Ptest_Parser 77387 1 richard.purdie 2017-10-05 15:14:31 +0000 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5f6945f5031e1a4ca116cc1eccf4c2f9dc228547 ? 77389 2 randy.macleod 2017-10-05 15:16:12 +0000 Added Robert since he worked on the commit that adds ptest results to buildhistory that Richard just referenced above. 77394 3 joshuagloe 2017-10-05 16:00:17 +0000 (In reply to comment #1) > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > ?id=5f6945f5031e1a4ca116cc1eccf4c2f9dc228547 ? This is definitely part of the solution, it means we don't need a separate repository. I need to look closer, some extra tooling may or may not be required. There were more patches in that series which don't appear to have been merged, they will be useful to resolve this bug. https://patchwork.openembedded.org/series/8033/# 77395 4 joshuagloe 2017-10-05 16:02:29 +0000 (In reply to comment #3) > (In reply to comment #1) > > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > > ?id=5f6945f5031e1a4ca116cc1eccf4c2f9dc228547 ? > > This is definitely part of the solution, it means we don't need a separate > repository. I need to look closer, some extra tooling may or may not be > required. > > There were more patches in that series which don't appear to have been > merged, they will be useful to resolve this bug. > > https://patchwork.openembedded.org/series/8033/# Or I'm an idiot, looks like they were all merged 78308 5 jair.de.jesus.gonzalez.plascencia 2017-11-21 23:36:35 +0000 I've created bug 12372 to track the progress in the implementation of automated execution of pTest. 81989 6 ee.peng.yeoh 2018-11-01 08:14:28 +0000 As point out by Richard and Randy, Robert had enabled ptest from testimage to archive test result to buildhistory git repository. Had verified that this was working with below configuration, ptest result from testimage was stored into buildhistory git repository. TEST_SUITES = "ping ssh xorg ptest" INHERIT += "buildhistory" BUILDHISTORY_COMMIT = "1" Closing this ticket.