Bug 6260

Summary: ptests requirements often not met
Product: [Build System, Metadata & Runtime] Other YP Layers Reporter: Volker <lists.volker>
Component: layersAssignee: Maxin B. John (inactive) <maxin.baby.john>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: alexandru.c.georgescu, poky.bs.watcher, poky.watcher, sgw
Version: unspecified   
Target Milestone: 1.8   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Volker 2014-05-01 17:49:23 UTC 
A lot of ptests failed in my case because of unmet RDEPENDS settings.

Examples:
- ptests does not run correctly if the rootfs is read-only. But the read-only rootfs provides writeable space.
- sed from busybox does not provide all options as GNU sed, so another test failed (perl)
- some tests try to execute bash, which is not installed on my system (e.g. strace)
- files not found (python)
- wget (I think the busybox wget) tests failed because it did not check for networking (and runqemu did not come up with a proper /etc/resolv.conf).


I didn't find much documentation about ptests. And none of them stated these problems, nor how to test your ptest by using only a minimal image (e.g. without bash and GNU sed).

Maybe there could be a quick warning message echo'd in 'ptest-runner' that it is not that stable yet and work in progress.

I see that the Yocto build system also runs some tests. Maybe it would be possible to create some additional ones that also executes ptest-runner?
Comment 1 Volker 2014-05-01 19:58:47 UTC 
Regarding the automatic run of ptests I found #4114
Comment 2 Saul Wold 2014-05-05 21:04:35 UTC 
The -ptest packages should have the correct RDEPENDS, so the perl-ptest should 
RDEPENDS on sed.  and so forth.

Regarding the read-only, it may not make sense to have these kinds of tests on 
read-only because we just don't know where each test is trying to write to, if 
you can enumerate that further, we can look at what fixes are required there.

We are working on incorporating more testing (including ptest) into the test 
automation infrastructre.
Comment 3 Maxin B. John (inactive) 2014-05-06 14:10:26 UTC 
I will start addressing the ptest related problems. 

Here are some documentation about ptest framework:
https://wiki.yoctoproject.org/wiki/Ptest
http://daniel.haxx.se/blog/2012/08/31/ptest-because-make-test-is-insufficient/
Comment 4 Maxin B. John (inactive) 2014-05-09 08:22:05 UTC 
Updated the perl ptest: 
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=b537c2fbf980122f25c9842f04915dd962f3f112
Comment 5 Saul Wold 2014-11-14 18:45:12 UTC 
This needs further enumeration as to what additional ptest packages have RDEPENDS issues or possibly close this one and open individual bugs for each issue.
Comment 6 Maxin B. John (inactive) 2014-11-25 10:23:48 UTC 
Hi,

Agree with Saul. We can open individual bugs for dependency issues when we encounter it.

Best Regards,
Maxin