Bug 9752 - ptest results do not include all the components tested
Summary: ptest results do not include all the components tested
Status: RESOLVED FIXED
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium+ major
Target Milestone: 2.2 M4
Assignee: Plauchu Edwin
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-06-09 21:30 UTC by Benjamin Esquivel
Modified: 2016-09-28 14:50 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Benjamin Esquivel 2016-06-09 21:30:09 UTC 
There is one component that has been identified that runs as part of the ptest execution and gets no report out of it. 

The flex component is enabled and run as part of the execution but it is presenting a bogus behavior that prevents it from reporting a PASS/FAIL result.

The script that is parsing the ptest log searching for results is skipping the flex component reports even though it presented the following lines:

START: ptest-runner
BEGIN: /usr/lib/flex/ptest

I would propose that the parser fails instead of just ignoring the lack of results. It is required at this moment to see if there are other components in this situation.
Comment 1 Leonardo Sandoval Gonzalez 2016-09-19 14:56:23 UTC 
Edwin, in case you want to easily reproduced this issue, just revert

66aebcc: Leonardo Sandoval: Wed Jun 15 18:05:22 2016 +0100: flex: Fix top_builddir path for flex's test-driver script

And run the ptest-runner. This patch is the one correcting the flex issue, so reverting it would allow to reproduce what Benjamin observed.
Comment 2 Plauchu Edwin 2016-09-20 15:24:43 UTC 
Added error coverage to support when launching child process fails.

Patch sent to Anibal Limon
Comment 5 Plauchu Edwin 2016-09-22 00:50:51 UTC 
Patch Version 4 (It is additionally including unit test code)
https://lists.yoctoproject.org/pipermail/yocto/2016-September/032104.html
Comment 6 Aníbal Limón 2016-09-22 22:45:58 UTC 
(In reply to comment #5)
> Patch Version 4 (It is additionally including unit test code)
> https://lists.yoctoproject.org/pipermail/yocto/2016-September/032104.html

Edwin,

I just integrated your code into and created a tag v2.0.1 since is a minor fix,

http://git.yoctoproject.org/cgit/cgit.cgi/ptest-runner2/log/

Please update the recipe and send the patch to oe-core.
Comment 7 Plauchu Edwin 2016-09-23 20:14:29 UTC 
the patch to oe-core
http://lists.openembedded.org/pipermail/openembedded-core/2016-September/126887.html