Run a specific test case list flexibly by an manifest file, but not put the case names in TEST_SUITES value. There are two solutions, but the first one is much flexible and we'll have patch submitted to Yocto. 1. TEST_SUITES = "${@read_testlist("/some/file")}" 2. bitbake -r mytests.conf <other options>
For "bitbake -r mytests.conf <other options>", the limitation is it depend on bitbake, and the mytests.conf is a bitbake specific format, we'd like to run exported test in the environment which might not have bitbake environment with more generic format, like manifest or json. So we prefer the solution "TEST_SUITES = "${@read_testlist("/some/file")}"" One more additional request for this ticket: Is it possible to support multiple manifest file as the input? We'd like this capability as well. Thanks!
User can specify TESTSUITE by external files in local.conf, e.g. TEST_SUITES_MANIFEST = "test1.manifest test2.manifest". The files lie in build/conf/. TEST_SUITES_MANIFEST supports relative path(which relative to build/conf/) and absolute path. Although the way like TEST_SUITES = "${@read_testlist(d, "test1.manifest")" is also workable, per testing, it has performance issue because running a couple of times during BB parsing stage.
Merged to poky master branch. Commit ID: 245899cc98881d2be784f8e7302120765c2bc7f
verified with poky master branch.