Bug 11757

Summary: selftests: do not touch existing build artifacts
Product: [Yocto Project Subprojects] IoT Reference OS Kit Reporter: Patrick Ohly <patrick.ohly>
Component: intel-iot-refkit-toolsAssignee: Patrick Ohly <patrick.ohly>
Status: RESOLVED FIXED QA Contact: yeoh ee peng <ee.peng.yeoh>
Severity: normal    
Priority: Medium+    
Version: 2.4   
Target Milestone: 2.5 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Patrick Ohly 2017-07-05 07:08:33 UTC
At the moment, some tests rebuild "normal" recipes with different settings:
- system update: refkit-image-common
- secure boot: ovmf (and others?)

This causes extra work in the CI testing because:
- system update runs and modifies refkit-image-common
- image installer needs to rebuild the original refkit-image-common,
  which wouldn't be needed if system update hadn't modified it

Changing the test order could be used to work around that but that is merely a workaround and doesn't work in practice because of bug #11756.

All tests should be written so that they either use the original recipes unmodified, or use their own test-specific recipes. For example, system update should have its own refkit-system-update.bb which just includes refkit-image-common.bb.

We probably need some infrastructure for creating test recipes. OE-core does it via meta-selftest, but that doesn't scale to other layers.

Another aspect involves ovmf: running a virtual machine with "runqemu ovmf" potentially modifies the ovmf.qcow2 file when allowing image writes to persist. Each test which needs to do that (image installer!) should make a copy of ovmf.qcow2 first and use the copy.
Comment 1 Patrick Ohly 2017-07-05 07:09:37 UTC
I've filed this because I don't want to forget about it, and because perhaps someone else than myself can work on this?
Comment 2 Patrick Ohly 2017-07-11 14:10:17 UTC
Partly fixed by https://github.com/intel/intel-iot-refkit/pull/238/commits/01641909eaafc51941cec27ef2d830582a0f886c (for system update).

Let's re-evaluate the impact on the current testing.
Comment 3 Patrick Ohly 2017-11-07 15:21:05 UTC
I believe this has been addressed sufficiently well.