7849 2015-06-05 03:46:30 +0000 Filter proper test cases from the test case manifest file and run the filtered ones 2015-09-14 11:22:56 +0000 1 1 10 QA/Testing Runtime Testing general 1.8.1 x86 Multiple VERIFIED FIXED Medium+ enhancement 1.9 M3 1 cathy.shen costin.c.constantin alexandru.c.georgescu junhuix.zhang cathy.shen --- --- Don't know oldest_to_newest 51518 0 cathy.shen 2015-06-05 03:46:30 +0000 There is request to run specific test cases at runtime based on user input, since for different test cycle, test set is different. E.g. run priority 1 cases only, run cases in a specific component only , run cases with specific tag, etc. We'll have patch to Yocto to support this. 51988 1 junhuix.zhang 2015-06-29 08:30:59 +0000 Test runner can provide a decorator: addtag. when a case was decorated by this decorator, a tag will be added to this case. then test runner will filter by these tags to run specific cases: for example: #foo.py from oeqa.oetest import oeRuntimeTest from oeqa.utils.decorators import addtag @addtag("class") class TestClass(oeRuntimeTest): @addtag("foo") def testfoo(self): print "testfoo" @addtag("bar1","bar2") def testbar(self): print "testbar" @addtag("bar1") def testbaz(self): print "testbaz" If user use a tag into conf/local.conf: TEST_SUITES_TAGS = "class", runner will run all test cases: testfoo, testbar, testbaz. If user use a tag: TEST_SUITES_TAGS = "foo", runner will run test cases: testfoo. If user use a tag: TEST_SUITES_TAGS = "foo bar1", runner will run two test cases: testfoo,testbar,testbaz. If user use a tag: TEST_SUITES_TAGS = "bar1&bar2", runner will run test case: testbar. A test case can add many tags, and user also can use many tags to filter test cases. If run testcases by an exported test, use -a or --tag to add filter tag(e.g. -a "foo bar1&bar2"). 52232 2 costin.c.constantin 2015-07-09 10:54:31 +0000 Hi Junhui, The idea looks good. Costin 53405 3 junhuix.zhang 2015-08-26 09:31:22 +0000 To support complicated situation, we change TEST_SUITES_TAGS is a python expression, It means that multiple tags like "foo bar1" should be "foo or bar1", or is a python keyword. details: Test runner provides a decorator: tag. when a case was decorated by this decorator, tags will be added to this case. then test runner will filter cases by these tags. Runner follows below rules: 1. tags can be added to a test class or a test method, test method will inherit tags form its class, also can overwrite those tags. 2. tags can be valued, if not valued, the runner will give it a default value: True 3. test runner follows python expression, when this expression's result is True for a test case, this case will be executed. 4. If the case doesn't have a specific tag, this variable is None. Some examples: #foo.py from oeqa.oetest import oeRuntimeTest from oeqa.utils.helper import tag @tag(TestType="Functional Positive") class TestClass(oeRuntimeTest): @tag("foo", TestType="Stability", FeatureID="IOTOS-211", debug=5) def testfoo(self): print "testfoo" @tag("bar", FeatureID="IOTOS-212", debug=10) def testbar(self): print "testbar" @tag("baz", FeatureID="IOTOS-213", debug=15) def testbaz(self): print "testbaz" If user use a tag into conf/local.conf: TEST_SUITES_TAGS = "TestType == 'Functional Positive'", runner will run two test cases: testbar, testbaz. If user use a tag: TEST_SUITES_TAGS = "foo", runner will run test cases: testfoo. If user use a tag: TEST_SUITES_TAGS = "debug < 11", runner will run two test cases: testfoo,testbar. If user use a tag: TEST_SUITES_TAGS = "TestType == 'Functional Positive' and debug <= 10", runner will run test cases: testbar. 53857 4 junhuix.zhang 2015-09-09 07:26:45 +0000 This patch is merged on poky. 54010 5 costin.c.constantin 2015-09-14 11:22:20 +0000 Hello, Verified with 75bad1b64bd485a50af69ed1f8663a0dadbb5f9e patch was merged with: fac0d67ec25f357684bcd0c80be282e1bca3adef 54011 6 costin.c.constantin 2015-09-14 11:22:56 +0000 marking as verified