Bug 9448

Summary: Add mechanism for running image level QA checks
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Joshua Lock <joshuagloe>
Component: coreAssignee: Joshua Lock <joshuagloe>
Status: RESOLVED FIXED QA Contact: Francisco Pedraza <francisco.j.pedraza.gonzalez>
Severity: enhancement    
Priority: Medium+ CC: jose.perez.carranza, meta.mr.watcher, meta.watcher, ross.burton
Version: 2.2   
Target Milestone: 2.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Joshua Lock 2016-04-12 16:10:28 UTC 
The Ostro project are implementing various image QA/sanity checks as postfuncs on do_image. 

We'd like to make use of similar functionality in meta-swupd to test for known possible issues when constructing images in a swupd-based OS (i.e. the issue in https://bugzilla.yoctoproject.org/show_bug.cgi?id=9320)

It would be useful to support an extension mechanism in OE-Core where we can define various methods which sanity-check a constructed image and have them all run before failing the task and reporting any issues to the user.
Comment 1 Joshua Lock 2016-04-13 21:10:05 UTC 
We added an implementation of this to meta-swupd in order to provide a sanity check for an easily produced problem when developing swupd-based OS.

See https://bugzilla.yoctoproject.org/show_bug.cgi?id=9320 and http://git.yoctoproject.org/cgit/cgit.cgi/meta-swupd/commit/?id=9ffc69b4954f9fb21b9572a0599f685242a941d9
Comment 3 Joshua Lock 2016-06-07 21:45:16 UTC 
Chris Larson rightly pointed out that the proposed patch doesn't allow for shell check functions (as well as abusing a Python exception class).

http://lists.openembedded.org/pipermail/openembedded-core/2016-June/122579.html
Comment 4 Joshua Lock 2016-07-07 15:32:32 UTC 
Working on a revised solution which uses the bb.build.FuncFailed exception, as raised when bitbake functions fail, along with a subclass of the same which adds an extra field for more detailed error messages when implementing a python image QA function.
Comment 5 Joshua Lock 2016-07-14 13:45:58 UTC 
v2 patch submitted: http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123884.html
Comment 6 Ross Burton 2016-07-20 13:43:34 UTC 
Fixed in oe-core c9bef2ecf1a30159d11781184829f41844a58c13.