Bug 10050 - oeqa runner generate wrong test results in log file when running dynamic test cases
Summary: oeqa runner generate wrong test results in log file when running dynamic test...
Status: RESOLVED OBSOLETE
Alias: None
Product: Runtime Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.4
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: Future
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-07-29 07:01 UTC by dawei
Modified: 2020-04-09 15:28 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
Test case (2.52 KB, text/plain)
2016-07-29 07:01 UTC, dawei
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description dawei 2016-07-29 07:01:59 UTC
Created attachment 3333 [details]
Test case

Details:
The results is correct printed on screen but not correct in log file

Step:

Use yocto oeqa runner to run the attach test cases

*Expected Results:*
See pass/fail/skip/error 4 results in log file

*Actual Results:*
All passed in log file
Comment 1 dawei 2016-07-29 07:08:26 UTC
From Zhang,Junhui:
there is a workaround in line 19 , if comment it out , this issue is not reproduced. You can see more explanation there.
Comment 2 Benjamin Esquivel 2016-07-29 15:00:00 UTC
Mariano, can you replicate this test?
Comment 3 Mariano Lopez 2016-07-29 21:17:04 UTC
In the test cases attached it creates dynamically the class and the for every new class it creates a TextTestResult, so this can lead to the behaviour observed.

OE doesn't use/require dynamic generated test.
Comment 4 dawei 2016-08-02 08:07:20 UTC
For us , we need this function to generate logs for this case:

One oe runner static *Total* case includes many real cases , so to the runner, it just only generate one result for those many real cases in the log file, which doesn't reflect the real results.
Comment 5 Jair Gonzalez 2017-08-29 21:53:19 UTC
Moving to 2.5, due to limited resources.
Comment 6 Jair Gonzalez 2017-08-29 22:03:10 UTC
Returning version to 2.4.
Comment 7 Joshua Lock 2017-12-07 15:12:54 UTC
Much of the oeqa code has been rewritten since this issue was filed, need to test whether the issue is still present.
Comment 8 Randy MacLeod 2020-04-09 15:28:27 UTC
If this problem shows up again, reopen.