7865 2015-06-09 07:16:35 +0000 oe-selftest to detect test cases in other layers else than OE-Core 2016-02-29 10:27:20 +0000 1 1 10 QA/Testing Build Testing general unspecified x86 Multiple RESOLVED FIXED Medium enhancement 2.1 1 sharron.liu costin.c.constantin alexandru.c.georgescu anibal.limon benjamin.esquivel cathy.shen francisco.j.pedraza.gonzalez jing.j.wang richard.purdie sharron.liu sharron.liu --- --- No (bug/feature does not impact docs) oldest_to_newest 51568 0 sharron.liu 2015-06-09 07:16:35 +0000 Hi Alex, and Yocto team, I'm working on a project using Yocto as build tool. I developed my own build-time test cases, and I wanted to use oe-selftest to run my tests. I have my test cases locating in my own layer. And I found oe-selftest failed to detect my test cases, when typing the command "--list-modules". I don't want to put them under "poky/meta/" since they're not for Yocto testing and they're for my project specific build testing. Any other project that use Yocto tool may face this issue, also. Could I expect oe-selftest to detect and launch test cases locating in other layers? For more details, here's my folder structure: poky meta/lib/oeqa/ meta-selftest scripts/oe-selftest meta-mylayer (added to "BBPATH") lib/oeqa/selftest (I want test cases here detected by oe-selftest) I tried to add minor changes in oe-selftest, and that patch will detect any cases under "lib/oeqa/selftest" from all layers that are added to "BBPATH". I would share that to you. Thanks, Sharron 51569 1 sharron.liu 2015-06-09 08:11:31 +0000 Alex, My test cases need to be organized into sub folders indeed, each sub folder represents a single test suite. meta-mylayer (added to "BBPATH") lib/oeqa/selftest (I want test cases here detected by oe-selftest) lib/oeqa/selftest/cfg/mytest.py ==> expect "--list-modules" list it as "cfg.mytest" lib/oeqa/selftest/cfg/files/mytest_data.json, mytest_dependency.py ==> These are test data and dependency, don't expect "--list-modules" list it is as test case. I also modify the oe-selftest a few lines for this purpose. Let me know what do you think :) Thanks, Sharron 51578 2 costin.c.constantin 2015-06-09 14:52:19 +0000 Hello Liu, Please have a look on 7833 and let me know if it suffices your needs. Costin 51610 3 costin.c.constantin 2015-06-10 11:49:35 +0000 Hi Liu, I think i did a mistake in my last comment. 7833 is not referring to something similar to this feature. However, i posted a comment with some steps to perform in order to achieve multi layer test inclusions. I will come back shortly with a patch for this request Costin 51628 4 sharron.liu 2015-06-11 08:40:32 +0000 Costin, Thanks for your prompt reply! You're right, 7833 is related to runtime test, while this 7865 is for build-time test. Here I would provide more detail steps for how-to-reproduce my issue :) 1. follow step 1~3 (of 7833) to add my test layer. meta-mylayer (added to "BBLAYERS", added to "BBPATH") lib/oeqa/selftest lib/oeqa/selftest/__init__.py lib/oeqa/selftest/cfg/__init__.py lib/oeqa/selftest/cfg/mytest.py lib/oeqa/selftest/cfg/files/mytest_data.json 2. from poky, setup build env, and from "BUILDDIR" type oe-selftest --list-modules 3. expect mytest being listed: <here's test modules from "meta/lib/oeqa/selftest/*.py", like buildhistory etc.> cfg.mytest 4. run mytest from oe-selftest oe-selftest --run-tests cfg.mytest 5. expect tests in "cfg.mytest" being run. Thanks, Sharron 51680 5 costin.c.constantin 2015-06-13 20:19:34 +0000 Hi Sharron, I managed to add few features to oe-selftest, allowing it to support tests from any meta*/lib/oeqa/selftest directories plus subdirectories to them. Still left to do: - considering an external cfg.mytest file for tests. Will be done shortly. I will upload the patch upstream. Costin 51681 6 costin.c.constantin 2015-06-14 20:07:33 +0000 Hi Sharron, For whatever reason, i misunderstood your needs, so please disregard comment #5. I'll come back with a solution shortly. Costin 51736 7 costin.c.constantin 2015-06-16 12:49:52 +0000 Hello, Patch submitted to be merged. This is Sharron's contribution. Costin 52160 8 cathy.shen 2015-07-07 08:21:07 +0000 Costin, is it merged? 52695 9 costin.c.constantin 2015-07-29 08:15:00 +0000 Hi Cathy, Just had a chat with Sharron and she proposed a better formatted version of the patch. I will update you after having a look on it 52743 10 costin.c.constantin 2015-07-31 11:24:20 +0000 Hello, As with the http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d2023c34e4c3735d919c35d80daf991aab579500 and http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=51d760e2a7e22f1b07608fc925da343e56a9ae74 oe-selftest considers test files in other layers present in bblayers.conf having lib/oeqa/selftest directory structure. 53863 11 cathy.shen 2015-09-09 08:10:21 +0000 Thanks for the information, Costin. I'd like to close this ticket considering it's already supported in Yocto. 54346 12 costin.c.constantin 2015-09-21 06:43:26 +0000 Thank you Cathy for the update. I delayed this comment because i wanted to clarify few things with Sharron. I will not close this feature until she will update it with her comment if the patches solve her needs. Until then marking it as resolved. 54717 13 alexandru.c.georgescu 2015-09-28 11:58:48 +0000 (In reply to comment #12) > Thank you Cathy for the update. I delayed this comment because i wanted to > clarify few things with Sharron. I will not close this feature until she > will update it with her comment if the patches solve her needs. > Until then marking it as resolved. Hi Sharron, can you update with the info Costin needs to close the enhancement? Thanks, AlexG 54718 14 alexandru.c.georgescu 2015-09-28 11:59:17 +0000 Changed the status to NEEDINFO 54802 15 sharron.liu 2015-09-30 06:18:13 +0000 Verified in latest poky (commit 4bc3f09 Sep 23 2015) meta-mylayer (added to "BBLAYERS", added to "BBPATH") lib/oeqa/selftest lib/oeqa/selftest/__init__.py lib/oeqa/selftest/cfg/__init__.py lib/oeqa/selftest/cfg/mytest.py lib/oeqa/selftest/cfg/files/mytest_data.json oe-selftest --list-modules >> here "cfg.mytest" is not listed ===> failed oe-selftest --run-tests cfg.mytest >> here "cfg.mytest" run as expected ==> passed. 54803 16 sharron.liu 2015-09-30 06:19:42 +0000 https://bugzilla.yoctoproject.org/show_bug.cgi?id=7865#c15 56643 17 sjolley.yp.pm 2015-11-19 15:57:35 +0000 This was REOPENED 57791 18 benjamin.esquivel 2016-01-13 16:09:49 +0000 CCing Anibal and Paco 58784 19 costin.c.constantin 2016-02-09 13:36:50 +0000 Hello, Patch submitted for review 59513 20 costin.c.constantin 2016-02-29 10:07:17 +0000 merged into master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d2a563c32c62908a8c8eb9618f86a3fc9e5fe5bc Marking as resolved