Bug 9752

Summary: ptest results do not include all the components tested
Product: [QA/Testing] Build Testing Reporter: Benjamin Esquivel <benjamin.esquivel>
Component: generalAssignee: Plauchu Edwin <edwin.plauchu.camacho>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: Medium+ CC: anibal.limon, leonardo.sandoval.gonzalez
Version: 2.2   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=9721
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Benjamin Esquivel 2016-06-09 21:30:09 UTC 
There is one component that has been identified that runs as part of the ptest execution and gets no report out of it. 

The flex component is enabled and run as part of the execution but it is presenting a bogus behavior that prevents it from reporting a PASS/FAIL result.

The script that is parsing the ptest log searching for results is skipping the flex component reports even though it presented the following lines:

START: ptest-runner
BEGIN: /usr/lib/flex/ptest

I would propose that the parser fails instead of just ignoring the lack of results. It is required at this moment to see if there are other components in this situation.
Comment 1 Leonardo Sandoval Gonzalez 2016-09-19 14:56:23 UTC 
Edwin, in case you want to easily reproduced this issue, just revert

66aebcc: Leonardo Sandoval: Wed Jun 15 18:05:22 2016 +0100: flex: Fix top_builddir path for flex's test-driver script

And run the ptest-runner. This patch is the one correcting the flex issue, so reverting it would allow to reproduce what Benjamin observed.
Comment 2 Plauchu Edwin 2016-09-20 15:24:43 UTC 
Added error coverage to support when launching child process fails.

Patch sent to Anibal Limon
Comment 5 Plauchu Edwin 2016-09-22 00:50:51 UTC 
Patch Version 4 (It is additionally including unit test code)
https://lists.yoctoproject.org/pipermail/yocto/2016-September/032104.html
Comment 6 Aníbal Limón 2016-09-22 22:45:58 UTC 
(In reply to comment #5)
> Patch Version 4 (It is additionally including unit test code)
> https://lists.yoctoproject.org/pipermail/yocto/2016-September/032104.html

Edwin,

I just integrated your code into and created a tag v2.0.1 since is a minor fix,

http://git.yoctoproject.org/cgit/cgit.cgi/ptest-runner2/log/

Please update the recipe and send the patch to oe-core.
Comment 7 Plauchu Edwin 2016-09-23 20:14:29 UTC 
the patch to oe-core
http://lists.openembedded.org/pipermail/openembedded-core/2016-September/126887.html