9756 2016-06-10 16:10:40 +0000 Plan clean up and improvements for oeqa 2017-12-06 16:56:24 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.2 x86 Multiple RESOLVED FIXED Medium+ enhancement 2.2 M3 1 joshuagloe anibal.limon anibal.limon benjamin.esquivel bluelightning leonardo.sandoval.gonzalez meta.mr.watcher meta.watcher --- --- Don't know oldest_to_newest 62900 0 joshuagloe 2016-06-10 16:10:40 +0000 oeqa has several deficiencies which make it difficult to work with and maintain, including: * dependencies are scattered around the code base * test-loading only supports one folder deep * test dependency order is non-deterministic and doesn't work well * module relations/dependencies are unclear We'd like to move to a new design which extends the python unittest module and better supports test dependencies and ordering and provides a clear high-level API addressing the concerns above. In order to do that we first need to perform a review of the oeqa codebase and plan the work involved in moving forward with the new design. This task will be complete when a new set of tickets are filed covering the tasks involved in improving oeqa. 63695 1 anibal.limon 2016-07-06 21:58:08 +0000 This plan needs to take into account XML output see, https://bugzilla.yoctoproject.org/show_bug.cgi?id=9682 63872 2 benjamin.esquivel 2016-07-12 20:24:29 +0000 Re-assigning to Edwin and moving to M3 as this will not make it for 2.2 M2. 64010 3 anibal.limon 2016-07-14 20:46:17 +0000 I finish the first draft of the plan, some tasks related to migration needs to define a set of step for do it but the other depend tasks needs to be complete for provide details of migration, i will load the bugs for cover the next tasks: Runner: - Implement OETestRunner with the ability for output logs in JUnit XML and RAW text based on configuration parameter. - Implement OEQemuTestRunner extending OETestRunner, this will start, stop qemu with certain configuration. Decorators: - Implement OETimeout decorator, this will send a SIGALARM when some time expires. - Implement OETag decorator, like OETestID this will hold a string for identify some test case, also the ability to list OETestCase's by tag. TestContext: - Refactor/split oeqa/oetest.py: Currently this class have all the TestContext implementations SDK, Image, SDKExt, the refactor needs to take into account don't depend on BItbake datastore. - Implement/Review the support for load test suites in different layers. Structure: - Restructure code for avoid direct dependencies on Bitbake data store, the QA framework needs to be implemented in a way to made useful outside Bitbake env. - Audit and move/merge general helper modules outside oeqa/utils like httpserver, logparser, git, ftools, network to avoid duplcated functionality, may be already exists functions/method's in oe/lib or bitbake/lib. Migration (steps to be defined): - test{image,sdk,sdkext): Support the new oeqa framework. - testexport: With changes of made independent oeqa framework of Bitbake, refactor current testexport functionality. - oe-selftest: Support the new oeqa framework. 65843 4 anibal.limon 2016-09-06 22:40:37 +0000 Master bug filed [1], i'll continue to load the bugs and add as dependants of the master bug. [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=10230