| Summary: | QA test results management to replace Testopia | ||
|---|---|---|---|
| Product: | [QA/Testing] Test Plans/Suite | Reporter: | yeoh ee peng <ee.peng.yeoh> |
| Component: | test-plans/suite | Assignee: | yeoh ee peng <ee.peng.yeoh> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | High | CC: | randy.macleod, stephano |
| Version: | 5.99 | ||
| Target Milestone: | 2.7 M3 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
| Bug Depends on: | |||
| Bug Blocks: | 12651, 13012 | ||
|
Description
yeoh ee peng
2018-04-04 19:05:08 UTC
Developing for #1: Completed: 1. Developed the script to parse oeqa testcases to capture test case name 2. Developed the script to define test result and log data structure for each test case captured above 3. Developed the script to output the test result and log data structure to json format file In progress: 1. Integrate with oe-git-archive to store test result and log data to git repository Developing for #1: Completed: 1. Developed the script to parse oeqa testcases to capture test case name 2. Developed the script to define test result and log data structure for each test case captured above 3. Developed the script to output the test result and log data structure to json format file In progress: 1. Integrate with oe-git-archive to store test result and log data to git repository Completed development for create and update. The “create” will create the test plan for a particular test component (eg. runtime), create the directory structure based on the test environments (eg. os type, board, host machine) defined by user, create the test result json template file, finally store these directory and files into local git repository. The “update” will update test result status (Pass/Fail) from a test log file created from executing oeqa (eg. runtime) into the local git repository created in “create”. OEQA test result logging http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156990.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156991.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156992.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156993.html Test result logging storage http://lists.openembedded.org/pipermail/openembedded-core/2018-November/157403.html Richard shared his ideas on test result managements, where the results need to be collected from the autobuilder, and analysed. Currently, OEQA automated test will output testresults.json to LOG_DIR, proposing a script to store the multiple testresults.json to a common git repository. Currently this script served as the single entry point to result managements, including storage, analyzing, reporting, & manual test execution. Initial patch submitted as below. http://lists.openembedded.org/pipermail/openembedded-core/2018-November/157403.html Bugzilla ticket below was for developing the script as a plugin to the main test-case-mgmt script, where this new plugin will handle manual test execution based on manual test cases available inside OEQA/manual, and then output test results in the standard testresults.json format. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12651 Revised the "store" and "report" codebase. http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278238.html Testopia main roles was to store test result and provide data for the test summary report. The resultstool scripts were developed to provide the "store" (storing test results into git repository) and "report" (test summary report). Patches submitted to enable "store" and "report" that will replace testopia. http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278487.html |