Bug 9559

Summary:	test reporting tool plan
Product:	[QA/Testing] Test Plans/Suite	Reporter:	Benjamin Esquivel <benjamin.esquivel>
Component:	test-plans/suite	Assignee:	Benjamin Esquivel <benjamin.esquivel>
Status:	VERIFIED FIXED	QA Contact:	Jose Perez C <jose.perez.carranza>
Severity:	enhancement
Priority:	High	CC:	alexandru.c.georgescu, anibal.limon, belen.barros.pena, bill.randle, corneliux.stoicescu, francisco.j.pedraza.gonzalez, jose.perez.carranza, joshuagloe, leonardo.sandoval.gonzalez, libertad.gonzalez.de.la.cruz, markus.lehtonen, richard.purdie, sgw
Version:	2.2
Target Milestone:	2.2 M2
Hardware:	x86
OS:	Multiple
URL:	https://wiki.yoctoproject.org/wiki/QA/Test_Reporting_Tool
See Also:	https://bugzilla.yoctoproject.org/show_bug.cgi?id=9954 https://bugzilla.yoctoproject.org/show_bug.cgi?id=9956 https://bugzilla.yoctoproject.org/show_bug.cgi?id=9957 https://bugzilla.yoctoproject.org/show_bug.cgi?id=9958
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Benjamin Esquivel 2016-04-29 17:15:24 UTC

A test reporting tool is necessary for the various components that produce results. An example requirements of why such tool is needed are:

i. the ability to store results from multiple sources (performance, manual, automated)
ii. the ability to compare results with a tool that is designed for that purpose. Including comparing performance results which are trickier than functional testing
iii. the centralization of the test result storage to allow a number of things like back up, replication, extension, advanced filtering, etc.

These are just some of the benefits of having a tool for reporting. A wiki is proposed for the plan description:

https://wiki.yoctoproject.org/wiki/QA/Test_Reporting_Tool

Note that this is not intended to replace testopia as a test manager but to extend the testing capabilities by having a one stop shop for test reporting.

Comment 1 Alexandru Georgescu 2016-05-19 07:46:34 UTC

Hi Benjamin,
The current work on the project mentioned in this enhancement is available in GitHub at: https://github.com/fullvlad/CustomReports

Comment 2 Benjamin Esquivel 2016-05-19 15:38:53 UTC

Thanks Alex, I believe that would be a valuable input

Comment 3 Joshua Lock 2016-05-25 13:55:08 UTC

Ideally this tool would be able to consume, and compare, other quality related datasources such as the buildhistory from the autobuilder[1] and performance statistics.

1. http://git.yoctoproject.org/cgit/cgit.cgi/poky-buildhistory/

Comment 4 Benjamin Esquivel 2016-07-01 17:59:43 UTC

an initial set of requirements are written at this section: 

https://wiki.yoctoproject.org/wiki/QA/Test_Reporting_Tool#General_Expectations

Comment 5 Benjamin Esquivel 2016-07-15 17:16:35 UTC

The plan was written in the following bugs:

- bug 9954,
- bug 9956,
- bug 9957,
- bug 9958

Comment 6 Jose Perez C 2016-07-29 14:43:56 UTC

Test Plan Created at https://wiki.yoctoproject.org/wiki/QA/Test_Reporting_Tool