9129 2016-02-17 18:17:21 +0000 Enhance Test Suite scanning and execution 2016-10-27 21:17:16 +0000 1 1 10 QA/Testing Test Plans/Suite test-plans/suite unspecified All Multiple RESOLVED DUPLICATE 10230 Medium enhancement 2.3 8410 1 alejandro.franco leonardo.sandoval.gonzalez benjamin.esquivel corneliux.stoicescu leonardo.sandoval.gonzalez --- --- Yes (doc changes required) oldest_to_newest 59163 0 alejandro.franco 2016-02-17 18:17:21 +0000 Derived from bug 8410 1.- The usage of "auto" in TEST_SUITES. When "auto" is listed in TEST_SUITES, a function ran after do_testimage scans the directories '<layer>/lib/oeqa/runtime/', and any tests found are added to the test run list, in whatever order they are found. 2.- How tests are skipped Once the tests in TEST_SUITES and the ones scanned if "auto" was present are in the run test list, they are ran one at a time, in order of appearance. First all the ones in TEST_SUITES in the order they were added there, then the ones scanned if "auto" was present, in order of scan. If any of these tests should be skipped, then those tests themselves will issue their own skip action, either using unittest.skipTest or any of the skip decorators specified in utils/decorators.py, like skipUnlessPassed. These decorators only work if the test list is ordered by dependencies, which is not the case sometimes, and results in errors. Proposed solution: Improving how test suites are scanned (2) in order to have an *ordered* test list should guarantee (3) works as designed. 59166 1 alejandro.franco 2016-02-17 20:43:08 +0000 In short, the goal of this is making sure testimage test suite dependencies are met when testing 59167 2 benjamin.esquivel 2016-02-17 20:48:20 +0000 correct, because currently the dependencies are not guaranteed to run in order. 62457 3 leonardo.sandoval.gonzalez 2016-05-25 20:46:00 +0000 Due to limited resources, moving this to 2.3 67677 4 leonardo.sandoval.gonzalez 2016-10-27 21:17:16 +0000 This work is covered by the new implementation of oeqa, called oeqa2, so marking as duplicate. *** This bug has been marked as a duplicate of bug 10230 ***