reproduce condition: at least a error or a fail test case.

Another issue: If this happens, It will block posterior test cases.
It because an if sentence in oeRuntimeTest's teawDown:
   if (self._resultForDoCleanups.failures or
                self._resultForDoCleanups.errors):
       self.dump_target_logs()

In this sentence we must notice that self._resultForDoCleanups.failures and self._resultForDoCleanups.errors were lists, these two lists contain test cases have already failed or in error, so when an error(or fail) happens, this if is always true, and self.dump_target_logs() will always be executed.

reproduce condition: at least a error or a fail test case.

Another issue: If this happens, It will block posterior test cases.
It because an if sentence in oeRuntimeTest's teawDown:
   if (self._resultForDoCleanups.failures or
                self._resultForDoCleanups.errors):
       self.dump_target_logs()

In this sentence we must notice that self._resultForDoCleanups.failures and self._resultForDoCleanups.errors were lists, these two lists contain test cases have already failed or in error, so when an error(or fail) happens, this if is always true, and self.dump_target_logs() will always be executed.

I'm afraid this issue is easy to be reproduced and impact test a lot once there is fail cases, pls. prioritize the fixing.

Hi,

This issue seem to be similar with the one seen in bug 8203.
Please take a look at that and apply those patches and see if it works.

I tested with TEST_TARGET = "simpleremote", I am not so sure about fake target.


Thanks,
--Daniel

As Daniel mentioned in the last comment the patches in bug 8203 fix the simple remote bug, unfortunately they introduced another one when generating the exported test. I'm currently working on this.

Are you able to run exported tests? I'm not able to run them at all. For me it can't find module bb. If you are able to run it please let me know how.

I performed more testing. Trying with current master I get this traceback when I try to run the runexported.py script:

$ ./runexported.py testdata.json 
Traceback (most recent call last):
  File "./runexported.py", line 33, in <module>
    from oeqa.oetest import runTests
  File "/home/mlopezva/otc/repo/poky/bug8118/tmp/testimage/core-image-sato/oeqa/oetest.py", line 14, in <module>
    import bb
ImportError: No module named bb


Using bb for logging was introduced in 	058e75d60e98841c8532a05006ae46725e3cdfd1.
If I try before that commit I get this trace:

$ ./runexported.py testdata.json
SSH log file: /home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/ssh_target_log.20150826075658
Traceback (most recent call last):
  File "./runexported.py", line 135, in <module>
    ret = main()
  File "./runexported.py", line 129, in main
    runTests(tc)
  File "/home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/oeqa/oetest.py", line 35, in runTests
    suite = loadTests(tc, type)
  File "/home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/oeqa/oetest.py", line 29, in loadTests
    suite = testloader.loadTestsFromNames(tc.testslist)
  File "/usr/lib64/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
AttributeError: 'module' object has no attribute 'buildcvs'


That is why I need to know how do you test using FakeTarget. I have a patch that I think solves the issue that will be introduced with the patches of bug 8203 but I don't have a way to testing it.

Bug 8212 opened for the exported case test.

(In reply to comment #7)
> I performed more testing. Trying with current master I get this traceback
> when I try to run the runexported.py script:
> 
> $ ./runexported.py testdata.json 
> Traceback (most recent call last):
>   File "./runexported.py", line 33, in <module>
>     from oeqa.oetest import runTests
>   File
> "/home/mlopezva/otc/repo/poky/bug8118/tmp/testimage/core-image-sato/oeqa/
> oetest.py", line 14, in <module>
>     import bb
> ImportError: No module named bb
> 
> 
> Using bb for logging was introduced in 
> 058e75d60e98841c8532a05006ae46725e3cdfd1.
> If I try before that commit I get this trace:
> 
> $ ./runexported.py testdata.json
> SSH log file:
> /home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/ssh_target_log.
> 20150826075658
> Traceback (most recent call last):
>   File "./runexported.py", line 135, in <module>
>     ret = main()
>   File "./runexported.py", line 129, in main
>     runTests(tc)
>   File
> "/home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/oeqa/oetest.py",
> line 35, in runTests
>     suite = loadTests(tc, type)
>   File
> "/home/mlopezva/repo/poky/temp/tmp/testimage/core-image-sato/oeqa/oetest.py",
> line 29, in loadTests
>     suite = testloader.loadTestsFromNames(tc.testslist)
>   File "/usr/lib64/python2.7/unittest/loader.py", line 130, in
> loadTestsFromNames
>     suites = [self.loadTestsFromName(name, module) for name in names]
> AttributeError: 'module' object has no attribute 'buildcvs'
> 
> 
> That is why I need to know how do you test using FakeTarget. I have a patch
> that I think solves the issue that will be introduced with the patches of
> bug 8203 but I don't have a way to testing it.

Yes, we had found this issue before, But we use our scripts to export test suites, so we fixed this issue in our scripts

Created attachment 2691 [details]
This patch will allow to runexported script compatible with host dump

This is a patch allows to work exported test with host dump. Notice that I won't fix current broken export tests feature. I think you will need to rebase to patch to work with your scripts. Please note this patch requires http://lists.openembedded.org/pipermail/openembedded-core/2015-August/109376.html

I will mark as duplicate of bug 8203 because the same series of patches broke the hardware test functionality.

*** This bug has been marked as a duplicate of bug 8203 ***

Close this ticket since it's fixed.