There is request to run specific test cases at runtime based on user input, since for different test cycle, test set is different. E.g. run priority 1 cases only, run cases in a specific component only , run cases with specific tag, etc. We'll have patch to Yocto to support this.
Test runner can provide a decorator: addtag. when a case was decorated by this decorator, a tag will be added to this case. then test runner will filter by these tags to run specific cases: for example: #foo.py from oeqa.oetest import oeRuntimeTest from oeqa.utils.decorators import addtag @addtag("class") class TestClass(oeRuntimeTest): @addtag("foo") def testfoo(self): print "testfoo" @addtag("bar1","bar2") def testbar(self): print "testbar" @addtag("bar1") def testbaz(self): print "testbaz" If user use a tag into conf/local.conf: TEST_SUITES_TAGS = "class", runner will run all test cases: testfoo, testbar, testbaz. If user use a tag: TEST_SUITES_TAGS = "foo", runner will run test cases: testfoo. If user use a tag: TEST_SUITES_TAGS = "foo bar1", runner will run two test cases: testfoo,testbar,testbaz. If user use a tag: TEST_SUITES_TAGS = "bar1&bar2", runner will run test case: testbar. A test case can add many tags, and user also can use many tags to filter test cases. If run testcases by an exported test, use -a or --tag to add filter tag(e.g. -a "foo bar1&bar2").
Hi Junhui, The idea looks good. Costin
To support complicated situation, we change TEST_SUITES_TAGS is a python expression, It means that multiple tags like "foo bar1" should be "foo or bar1", or is a python keyword. details: Test runner provides a decorator: tag. when a case was decorated by this decorator, tags will be added to this case. then test runner will filter cases by these tags. Runner follows below rules: 1. tags can be added to a test class or a test method, test method will inherit tags form its class, also can overwrite those tags. 2. tags can be valued, if not valued, the runner will give it a default value: True 3. test runner follows python expression, when this expression's result is True for a test case, this case will be executed. 4. If the case doesn't have a specific tag, this variable is None. Some examples: #foo.py from oeqa.oetest import oeRuntimeTest from oeqa.utils.helper import tag @tag(TestType="Functional Positive") class TestClass(oeRuntimeTest): @tag("foo", TestType="Stability", FeatureID="IOTOS-211", debug=5) def testfoo(self): print "testfoo" @tag("bar", FeatureID="IOTOS-212", debug=10) def testbar(self): print "testbar" @tag("baz", FeatureID="IOTOS-213", debug=15) def testbaz(self): print "testbaz" If user use a tag into conf/local.conf: TEST_SUITES_TAGS = "TestType == 'Functional Positive'", runner will run two test cases: testbar, testbaz. If user use a tag: TEST_SUITES_TAGS = "foo", runner will run test cases: testfoo. If user use a tag: TEST_SUITES_TAGS = "debug < 11", runner will run two test cases: testfoo,testbar. If user use a tag: TEST_SUITES_TAGS = "TestType == 'Functional Positive' and debug <= 10", runner will run test cases: testbar.
This patch is merged on poky.
Hello, Verified with 75bad1b64bd485a50af69ed1f8663a0dadbb5f9e patch was merged with: fac0d67ec25f357684bcd0c80be282e1bca3adef
marking as verified