Bug 7524

Summary: oeqa: tests in additional layer not found
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Patrick Ohly <patrick.ohly>
Component: devtools / tool chainAssignee: Scott Rifenbark <srifenbark>
Status: VERIFIED FIXED QA Contact: Bogdan Alexandru Voiculescu <bogdanx.a.voiculescu>
Severity: normal    
Priority: Medium CC: bogdanx.a.voiculescu, meta.mr.watcher, meta.watcher, srifenbark
Version: 1.7   
Target Milestone: 1.9 M1   
Hardware: x86   
OS: Multiple   
Whiteboard: 30 April 2015: IN PROGRESS REVIEW
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)

Description Patrick Ohly 2015-03-26 19:26:22 UTC
I created the following dummy.py in meta-oe/lib/oeqa/runtime/dummy.py. meta-oe is in my bblayer.conf. My local.conf has:

INHERIT += "testimage"
TEST_SUITES="dummy"

dummy.py:
-------------------
import unittest
from oeqa.oetest import oeRuntimeTest, skipModule
from oeqa.utils.decorators import *

class DummyTest(oeRuntimeTest):

    def test_dummy(self):
        pass
-------------------


According to the comments in the documentation and meta/lib/oeqa/runtime/__ini__.py, I would expect my test to be found.

But it fails:

> bitbake -c testimage core-image-minimal 
...
NOTE: Executing RunQueue Tasks
ERROR: Loading tests failed:
Traceback (most recent call last):
  File "testimage.bbclass", line 48, in testimage_main
  File "/data2/yocto/iot-os/meta/lib/oeqa/oetest.py", line 29, in loadTests
    suite = testloader.loadTestsFromNames(tc.testslist)
  File "/usr/lib64/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib64/python2.7/unittest/loader.py", line 100, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'dummy'

It works once I move the file to meta/lib/oeqa/runtime.

I'm using a not-quite-up-to-date master branch of OE-core (rev 5f74262cabb414a1a6000ab6b2d3ba2c92e2a7ff).
Comment 1 Patrick Ohly 2015-03-27 14:40:44 UTC
strace -f shows that the source file of the test is read when located inside the additional layer. So the file search path is okay, it's just loading/running which somehow fails.
Comment 2 Patrick Ohly 2015-04-07 16:53:48 UTC
I figured it out: the lib/oeqa/runtime directory of a layer must contain an __init__.py file. The file can be empty, but it must exist.

So this is a documentation bug after all, for http://www.yoctoproject.org/docs/1.7.1/dev-manual/dev-manual.html#qemu-image-writing-new-tests

I suggest to rephrase the last sentence of the first paragraph as "Just remember that filenames need to map directly to test (module) names, that you do not use module names that collide with existing core tests, and that there must be at least an empty __init__.py file in the runtime directory."
Comment 3 Scott Rifenbark 2015-04-30 15:38:29 UTC
Hi, 

Checkout http://www.yoctoproject.org/docs/1.9/dev-manual/dev-manual.html#qemu-image-writing-new-tests for the changes.  Because the list was a bit long, I reformatted into a bulleted list.  

I have this in IN PROGRESS REVIEW now.  Let me know of any issues. 

Thanks,
Scott
Comment 4 Patrick Ohly 2015-05-21 09:51:07 UTC
Looks good to me. Not sure whether that means that I should change the status to "RESOLVED".
Comment 5 Scott Rifenbark 2015-05-21 13:52:45 UTC
Hi Patrick, 

Since you filed the bug and it was assigned to me and you approved the fix, I will set it to RESOLVED and mark the doc flag done.  QA will then move it from RESOLVED to VERIFIED. 

Thanks!
Scott
Comment 6 Bogdan Alexandru Voiculescu 2015-09-03 09:25:23 UTC
verified; documentation looks good.