Bug 12789

Summary:	qa-tools: enhance script to analyze ptest log to identify test case that passing in precious cycle but failing in current cycle
Product:	[QA/Testing] Test Plans/Suite	Reporter:	yeoh ee peng <ee.peng.yeoh>
Component:	test-plans/suite	Assignee:	pankaj yadav <pankaj-y>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	High	CC:	pankaj-y
Version:	2.4.3
Target Milestone:	2.6 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description yeoh ee peng 2018-06-14 07:28:36 UTC

Current qa-tools that was used to analyze ptest log focus on compiling overall % of passing, while this overall passing statistic was serving as a high level indicator, this does not pin point the real problem.  

Need to enhance qa-tools to identify specific test case where it was passing in previous QA release but failing on current QA release, so that this information will be used to open bugzilla ticket to investigate ptest failure.

Comment 1 yeoh ee peng 2018-07-06 01:56:19 UTC

I am working on this enhancement.

Comment 2 pankaj yadav 2018-07-13 07:37:44 UTC

I have identified the use cases and creating the script.

Comment 3 yeoh ee peng 2018-07-13 07:41:06 UTC

(In reply to comment #2)
> I have identified the use cases and creating the script.

Perhaps you could change the Status: IN PROGRESS IMPLEMENTATION if you are already started the implementation.

Comment 4 pankaj yadav 2018-07-19 09:55:01 UTC

Script for comparison of Ptest result has been developed.

Comment 5 pankaj yadav 2018-07-30 02:17:33 UTC

Code implementation is done in qa-tools at below mentioned path :

http://git.yoctoproject.org/cgit/cgit.cgi/qa-tools/commit/?id=54071af421f91e34e84adafd435f18d686a0af8e