| Summary: | Allow oe-selftest to run custom test suites based on different criteria | ||
|---|---|---|---|
| Product: | [QA/Testing] Build Testing | Reporter: | Daniel Istrate <daniel.alexandrux.istrate> |
| Component: | general | Assignee: | Daniel Istrate <daniel.alexandrux.istrate> |
| Status: | VERIFIED FIXED | QA Contact: | Bogdan Alexandru Voiculescu <bogdanx.a.voiculescu> |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | alexandru.c.georgescu, bluelightning, bogdanx.a.voiculescu, richard.purdie, ross.burton |
| Version: | 2.0 | ||
| Target Milestone: | 2.1 M1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | New (Never tested) |
| Verified: | Documentation change: | Yes (doc changes required) | |
| Bug Depends on: | |||
| Bug Blocks: | 8647 | ||
Patch sent to ML: http://lists.openembedded.org/pipermail/openembedded-core/2015-December/113637.html This patch covers: 1. Can run custom lists of tests based on different criteria: --run-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags> eg: --run-tests-by module imagefeatures signing recipetool --run-tests-by id 1377 1273 935 --run-tests-by tag wic sstate bitbake 2. Can list tests based on different criteria: --list-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags> eg: --list-tests-by module imagefeatures signing recipetool --list-tests-by id 1377 1273 935 --list-tests-by tag wic sstate bitbake 3. Can list all tags that have been set to test cases: --list-tags The list of tags should be kept as minimal as possible. This helps preview the tags used so far. To take advantage of the 'tag' feature: - add @tag(feature=<>) to testcases eg: @tag(feature='signing') for a single tag @tag(feature=(('signing', 'sstate')) or @tag(feature=['signing', 'sstate']) for multiple tags Moved to M1 since Daniel already submitted a patch for it in M1. Patches submitted. As of b3d6872bf169a13e572638e3809c50a868246616: patch merged into master. Verified with m2.rc1 |
oe-selftest should be able to run custom test suites based on different criteria like: id (test case id (@testcase)), name (test case name), class, module and tag (@tag(feature=(feature list))). eg: oe-selftest --run-tests-by tag wic bitake recipetool pr-service oe-selftest --run-tests-by name test_x test_y test_z oe-selftest --run-tests-by id 1001 1002 185 1902 oe-selftest --run-tests-by class Signing ImageFeatures oe-selftest --run-tests-by module wic signing