Bug 11622

(In reply to comment #0)
> oe-selftest is now printing out the same kind of output it used to log to a
> separate file, which I believe is meant as input to our other processes.
> Consistency is all well and good, but this is a regression in terms of
> manual runs - there's no summary for example so I have to scroll through the
> entire output to be confident that no test failed.
> 
> Can we please restore the original output format or at least something very
> similar?

Yes the new format is different and is the same used by runtime and {e,}sdk, if you look at end the summary, there is the same value [old 1][new 2]:

 ----------------------------------------------------------------------
Ran 282 tests in Ns

The only difference i see is that now we print a summary with the test case id's [2], this information is needed to implement automatic reporting to testopia it has the case id's and the PASSED/FAIL status.

I suggest to move [2]:

2017-06-07 07:12:18,190 - oe-selftest - INFO - SUMMARY:
2017-06-07 07:12:18,190 - oe-selftest - INFO - oe-selftest () - Ran 282 tests in 25955.740s
2017-06-07 07:12:18,191 - oe-selftest - INFO - oe-selftest - OK - All required tests passed

To the end of the RESULTS details,

Any feedback?

Cheers,
Anibal

[1] https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/903/steps/Running%20oe-selftest/logs/stdio
[2] https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/910/steps/Running%20oe-selftest/logs/stdio

OK, so I scrolled back again in the oe-selftest output and you're right, there is a summary - I'd missed it because it isn't at the end anymore, it's now followed by the large number of RESULTS lines. Do we really need to see those RESULTS lines in the console output? If the answer is yes, then can we print the summary after those instead of before?

(In reply to comment #2)
> OK, so I scrolled back again in the oe-selftest output and you're right,
> there is a summary - I'd missed it because it isn't at the end anymore, it's
> now followed by the large number of RESULTS lines. Do we really need to see
> those RESULTS lines in the console output? If the answer is yes, then can we
> print the summary after those instead of before?

Yes we need those line results to automatically parse/load into Testopia, QA Team is working on it.

And for sure we will move the summary at end of the output.

Cheers,
Anibal

Those results used to be made available via a separate log file, is that no longer workable? My concern is as far as manual running is concerned they are mostly just noise.

(In reply to comment #4)
> Those results used to be made available via a separate log file, is that no
> longer workable? My concern is as far as manual running is concerned they
> are mostly just noise.

That change could be possible, but currently we use a unique logger for the output and the file and if we want to differ between manual and automatic run, the addition of command line option is needed. 

If we display different information in the output and log file then when some person look at the output and the file the information will differ and could cause some questions about the content.

Cheers,
Anibal

Review at,

http://lists.openembedded.org/pipermail/openembedded-core/2017-June/137683.html

0.05 working days = 24 minutes!

Anibal, I think you overestimate this bug.

Merge at,

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e9a6fd9e9779757773a172475d9725823dd3d5ff

(In reply to comment #8)
> Merge at,
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/
> ?id=e9a6fd9e9779757773a172475d9725823dd3d5ff

I will send other patch to don't log twice the detail information about test failure, this will improve the display of information.

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/oe_selftest_threaded&id=d3d3bacbd6fe6f612237cd36bcab7c045304b8d1

Cheers,
Anibal