| Summary: | get_tests_from_module() shouldn't assume oeSelfTest | ||
|---|---|---|---|
| Product: | [QA/Testing] Runtime Testing | Reporter: | Ross Burton <ross.burton> |
| Component: | general | Assignee: | Aníbal Limón <anibal.limon> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | anibal.limon, benjamin.esquivel, leonardo.sandoval.gonzalez |
| Version: | 2.2 | ||
| Target Milestone: | 2.4 M2 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Ross Burton
2016-12-20 15:32:31 UTC
This is a good point, these tests were enabled as OeSelfTest given the requirement to be executed along with the automation but with bug oeqa2, all the tests that do not require/use the bitbake -e info should inherit instead from OeTest which is lighter. Regarding your question about using unittest.TestLoader instead, it's something we are not targeting right now for oeqa2. We are targeting a complementary feature (another step towards keeping unittest compatibility) which is the ability to run any of the tests with a python -m unittest call. Such execution would not include features like: Test dependencies, selective execution, etc. But at least it would not make the OeTest (and derived classes) useless outside our own loader. Perhaps later we could include this basic test formatting as well. Marking it for M2 as this comes along with the rest of the test migration. When migration of these tests is done we need to switch their inheritance from OeSelfTest to OeTest. Then come back with a measurement of the improvement in the time of execution to see if it satisfies this bug. oeqa2 is now on master, so moving to current milestone (M3). selftest migration will start in 2.4 release, so moving milestone (In reply to comment #0) > oeqa shouldn't make the hard assumption that all test cases inherit > oeSelfTest, as that has quite a lot of overhead for a test case that doesn't > need to use it. > > (for example the oe/lib/ tests). > > Why can't oe-selftest just use unittest.TestLoader? We just finish the migration of selftest to the new OEQA framework and make sense what you said to support base unittest TestCases, i created a patch for do this but needs some testing to be sure that don't brake current codebase. http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/simple_patches&id=168a929b3c39a55efcdc44104197070f4be31d42 Cheers, Anibal (In reply to comment #5) > (In reply to comment #0) > > oeqa shouldn't make the hard assumption that all test cases inherit > > oeSelfTest, as that has quite a lot of overhead for a test case that doesn't > > need to use it. > > > > (for example the oe/lib/ tests). > > > > Why can't oe-selftest just use unittest.TestLoader? > > We just finish the migration of selftest to the new OEQA framework and make > sense what you said to support base unittest TestCases, i created a patch > for do this but needs some testing to be sure that don't brake current > codebase. > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/ > simple_patches&id=168a929b3c39a55efcdc44104197070f4be31d42 > > Cheers, > Anibal I just change the oelib cases to use unittest.TestCase and fire a AB build to see the results, http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/simple_patches&id=cb7f0a9dcd9d5038ec5f41755f8fac5347f1355c Anibal Review at, http://lists.openembedded.org/pipermail/openembedded-core/2017-June/137780.html http://lists.openembedded.org/pipermail/openembedded-core/2017-June/137781.html |