Bug 7376

Summary: oe-selftest doesn't run lib/oe/tests/*
Product: [QA/Testing] Runtime Testing Reporter: Ross Burton <ross.burton>
Component: generalAssignee: Jose Perez C <jose.perez.carranza>
Status: VERIFIED FIXED QA Contact: Juan Ramos <juan.fernandox.ramos.frayle>
Severity: enhancement    
Priority: Medium+ CC: alexandru.c.georgescu, benjamin.esquivel, bluelightning, bogdanx.a.voiculescu, corneliux.stoicescu, mariano.lopez, meta.mr.watcher, meta.watcher
Version: 1.8   
Target Milestone: 2.3 M1   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on:    
Bug Blocks: 10727    
Attachments:
Description Flags
core-image-sato-sdk_x86-64MMax64.log none

Description Ross Burton 2015-02-27 12:05:58 UTC
And it should.
Comment 1 Paul Eggleton 2015-03-03 10:32:42 UTC
Alternatively, should those tests be migrated over to meta/lib/oeqa/selftest/ ?
Comment 2 Costin 2015-07-27 16:59:03 UTC
 it doesn't run tests in meta/lib/oe ... with or without subdirs. It does however running those from lib/oeqa/selftest. There were quite some mods lately for oe-selftest, so i guess moving them to oeqa/selftest would be a good approach.
Comment 3 Ross Burton 2015-07-28 15:10:34 UTC
Migrating the tests is probably the right thing to do.

Reassigning to Alex, can someone in QA review the tests and scope out how many of those could be moved to selftest?
Comment 4 Paul Eggleton 2015-07-28 15:41:13 UTC
FYI, as of just recently in master oe-selftest now sets up the python path so that both bb and oe modules can be imported, so we can now do more unit test type tests in oe-selftest (which appears to be what meta/lib/oe/tests was meant to be).
Comment 5 Alexandru Georgescu 2015-07-28 16:20:58 UTC
(In reply to comment #4)
> FYI, as of just recently in master oe-selftest now sets up the python path
> so that both bb and oe modules can be imported, so we can now do more unit
> test type tests in oe-selftest (which appears to be what meta/lib/oe/tests
> was meant to be).

Seems interesting we'll have a look
Comment 6 Costin 2015-07-31 09:56:55 UTC
Hi, 

In a way, this is related to 7865. I guess we should modify in the first place scriptpath.add_oe_lib_path() to add also other layers present in BBPATH to the oe-selftest python's sys path.
Also for this to take effect oe-selftest's get_tests() method should consider dir structure different than oeqa/selftest
Comment 7 Paul Eggleton 2015-07-31 10:00:23 UTC
I don't think we can simply make scriptpath.add_oe_lib_path() look in BBPATH because it would then require us to query the value of BBPATH (which is not a problem in oe-selftest context, but would be in others). I wouldn't be opposed to adding a parameter that let you pass in the BBPATH though, or a separate function which did the query using tinfoil (see what scripts/devtool does to get it).
Comment 8 Costin 2015-07-31 11:25:53 UTC
I'll have a look on tests from lib/oe to move them in oeqa/selftest
Comment 9 Benjamin Esquivel 2016-11-03 22:01:39 UTC
is this still happening Mariano?
Comment 10 Jose Perez C 2016-11-03 22:14:05 UTC
I'm working on this bug assigning it back to me.
Comment 11 Mariano Lopez 2016-11-03 22:34:38 UTC
Selftest will look for test in <layer>/lib/oeqa/selftest for every layer in BBLAYERS. In this case the tests are in meta/lib/oe/tests, so selftest won't find them
Comment 13 Mariano Lopez 2016-11-11 22:49:47 UTC
Checking these tests all of them check for functions/classes located in meta/lib/oe; I have some comments regarding for these tests.

1. The tests don't check the functionality of the build system. Is the right place to put them under selftest?
2. If you want to run them in the autobuilder, it is possible to hardcode the path in oe-selftest.
Comment 14 Paul Eggleton 2016-11-13 20:29:25 UTC
I appreciate they are not the same kinds of tests we normally do within oe-selftest, however they are testing library functions used by the build system.

The reason I am suggesting moving them to oe-selftest is because we really don't need yet another place where we define and run tests.
Comment 15 Benjamin Esquivel 2016-11-15 00:23:42 UTC
(In reply to comment #14)
> I appreciate they are not the same kinds of tests we normally do within
> oe-selftest, however they are testing library functions used by the build
> system.
> 
> The reason I am suggesting moving them to oe-selftest is because we really
> don't need yet another place where we define and run tests.
I think you're right, these tests should be alright in seltest as they don't add more than a couple of minutes to the execution. I'll reply something at the review.
Comment 16 Jose Perez C 2016-11-15 16:19:56 UTC
Patch V2 sent to mailing list 

http://lists.openembedded.org/pipermail/openembedded-core/2016-November/128815.html
Comment 18 Juan Ramos 2017-05-08 19:08:16 UTC
Created attachment 3760 [details]
core-image-sato-sdk_x86-64MMax64.log
Comment 19 Juan Ramos 2017-05-08 19:08:45 UTC
(In reply to comment #18)
> Created attachment 3760 [details]
> core-image-sato-sdk_x86-64MMax64.log

Verified, tested in commit:381897c64069ea43d595380a3ae913bcc79cf7e1