| Summary: | oe-selftest: measure test coverage | ||
|---|---|---|---|
| Product: | [QA/Testing] Build Testing | Reporter: | Paul Eggleton <bluelightning> |
| Component: | general | Assignee: | Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez> |
| Status: | RESOLVED FIXED | QA Contact: | Daniel Istrate <daniel.alexandrux.istrate> |
| Severity: | enhancement | ||
| Priority: | Medium | CC: | daniel.alexandrux.istrate, eduard.bartosh, humberto.ibarra.lopez |
| Version: | unspecified | ||
| Target Milestone: | 2.1 M2 | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
| Attachments: | |||
|
Description
Paul Eggleton
2015-11-11 16:38:44 UTC
yes, I briefly looked at it and put it to my TODO list. I can work on it if needed after 22nd of Nov when I'm back from vacation. (In reply to comment #1) > yes, I briefly looked at it and put it to my TODO list. I can work on it if > needed after 22nd of Nov when I'm back from vacation. Ed. I already have a patch and some measurements. I will post them today on this page. In case more work is needed, just let me know. BTW, all coverage has been done with python coverage 4.0.2. Created attachment 2851 [details]
proposed patch to enable coverage on unit tests
Created attachment 2852 [details]
provision patch to list classes (and its tests methods) without format
Once this patch is applied, it can used to get the unit tests and executed each one with coverage:
$oe-selftest --list-classes > list-classes.txt
$ # remove any tests you do want to execute
$ xargs -a list-classes.txt -n1 oe-selftest --coverage --run-tests
Lots of .coverage.* files are created, having coverage PER unit test.
Created attachment 2854 [details]
oe-selftest --run-all-tests including coverage
This is a report created by coverage at the end of the run 'oe-selftest --run-all-tests'. In resume, the current coverage for the Poky system is around 42%.
Created attachment 2855 [details]
Coverage per unit test
This tarball contains coverage output data PER unit tests. The unit tests taken into account are indicated on list-classes.txt and the coverage configuration is defined on .coveragerc. The coverage was done under the following commit:
fc45dea: Richard Purdie: Tue Nov 3 14:03:03 2015 +0000: build-appliance-image: Update to jethro head revision
Patch sent to the ML. Waiting for comments. Initial set of patches are already on master, enabling source code coverage from oe-selftest. This work has been done by Humberto Ibarra. |