Bug 8921

Summary:	Code coverage: add ability to create missing 'coverage area' reports
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Benjamin Esquivel <benjamin.esquivel>
Component:	Scripts and Tools	Assignee:	Unassigned <unassigned>
Status:	RESOLVED OBSOLETE	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	joshuagloe, leonardo.sandoval.gonzalez, mhalstead, richard.purdie, ross.burton
Version:	5.99
Target Milestone:	Future
Hardware:	x86
OS:	Multiple
Whiteboard:	QA REVIEW
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Benjamin Esquivel 2016-01-13 22:54:53 UTC

Let's start by saying that a testing coverage area is a defined code section that can be executed by a test. In the source code, a coverage area can be measured by function then by branch(if/else/case/etc). A given function can present 3 testing coverage statuses:

 - The function is completely un-exercised (zero coverage)
 - The function is partially exercised (Some branches covered, some coverage)
 - Completely executed (all branches covered, 100% coverage)

The objective of this bug is to obtain this information in an automated fashion along with the code coverage report to be able to identify which functions have zero coverage which in turn means that it should be analyzed why tests are missing to exercise that part of the code.

Comment 1 Leonardo Sandoval Gonzalez 2016-02-18 21:26:08 UTC

Currently, python coverage module is being used to measure code execution. The code granularity AFAIK is a module (a complete file) and not a function. With this in mind, we should do some post-processing on annotated code (coverage's annotated code) to get the functions and then the counting of executed, excluded and missing lines.

Comment 2 Leonardo Sandoval Gonzalez 2016-12-06 17:29:40 UTC

This is low priority for 2.3 and there is no window time to work on this area during current release.

Comment 3 Ed Bartosh 2016-12-07 10:58:09 UTC

As this issue got assigned to me I'd like to know why do we need this and is this functionality can be implemented using python-coverage?

Comment 4 Leonardo Sandoval Gonzalez 2016-12-07 14:37:02 UTC

(In reply to comment #3)
> As this issue got assigned to me I'd like to know why do we need this and is
> this functionality can be implemented using python-coverage?

I moved to 2.4 and for some reason it got assigned to you.

For your question, oe-selftest is already using python-coverage (version 4.x), so we can produce data. The main task here is the analysis of this data. No plans to do the later during 2.3

Comment 5 Armin Kuster 2019-12-22 19:32:09 UTC

I believe we are doing this via the resulttool.

Comment 6 Richard Purdie 2020-01-09 16:45:21 UTC

I don't believe we ever figured out correct tools for code coverage or that they are practical with the system. If anyone ever does want to do this that is fine however we have other areas QA work would likely be focused on, marking as obsolete, can be reopened if anyone is willing to progress this.