Bug 3380

Summary:	[QA/Test] Integrated Automation framework catering to All automation needs
Product:	[QA/Testing] Runtime Testing	Reporter:	venkata ramana g <venkatarg>
Component:	general	Assignee:	Stefan Stanacar <sstncr>
Status:	VERIFIED WONTFIX	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	richard.purdie, sanil.kumar, sgw, veerabrahmamvr, venkatarg, yi.zhao
Version:	1.4
Target Milestone:	1.4
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	---

Description venkata ramana g 2012-11-02 12:46:57 UTC

There should be single automation framework(ex: auto test), where all the automated testcases can be kept,maintained and mapped to testcases.
This enables automation to be effectively reused.

It has to cater to and support
1. Host testing - mainly for build framework, tool chain or ADT testing
2. Should able to handle bring up and bring down physical/qemu target for deployment, mainly for target bring up related testcases automation.
3. Should support target testcase execution and result fetching.
4. setup, tear down and cleanup to be supported.
5. Distributing the tests over network
6. Should allow the automated testcases to be triggered manually on target on need basis.
7. Should allow to organize the testcases(ex: Sanity, functional, Tool chain->sanity) and execute required based on type and component.
8. Should suit embedded systems where target might have limited space or even read only i.e., not have enough space to copy all the testcases one shot and execute.

"Auto test" is widely used and can be considered for meeting the above requirements.

Comment 1 Laurentiu Serban 2012-11-23 11:42:39 UTC

Hello Venkata,

We will start working on this.
Auto test is also on our radar for this kind activity.
We will get you involved in this.

Br,
Laurentiu

Comment 2 Laurentiu Serban 2012-11-27 21:05:12 UTC

Stefan Stanacar is the owner for this

Comment 3 Stefan Stanacar 2013-03-27 22:11:44 UTC

The new Autobuilder code can provide the necessary infrastructure for more test automation (together with better runtime/qemuimagetest scripts) and supporting yet another framework/tool isn't always the best idea. 
Also the use of autotest means some overlaping functionality with AB (duplicating build and runtime tests which are already on the AB), so the team decided to focus on doing more QA with AB for the 1.5 cycle.

Comment 4 Stefan Stanacar 2013-04-03 16:09:32 UTC

As per my last comment I filed #4137.
Now just setting the status to Verified.

Comment 5 venkata ramana g 2013-04-04 05:37:48 UTC

I agree that we need not support a new tool, if existing framework can handle the same.But we can check if current AB QA framework meets the above mentioned automation needs.