At the moment, some tests rebuild "normal" recipes with different settings: - system update: refkit-image-common - secure boot: ovmf (and others?) This causes extra work in the CI testing because: - system update runs and modifies refkit-image-common - image installer needs to rebuild the original refkit-image-common, which wouldn't be needed if system update hadn't modified it Changing the test order could be used to work around that but that is merely a workaround and doesn't work in practice because of bug #11756. All tests should be written so that they either use the original recipes unmodified, or use their own test-specific recipes. For example, system update should have its own refkit-system-update.bb which just includes refkit-image-common.bb. We probably need some infrastructure for creating test recipes. OE-core does it via meta-selftest, but that doesn't scale to other layers. Another aspect involves ovmf: running a virtual machine with "runqemu ovmf" potentially modifies the ovmf.qcow2 file when allowing image writes to persist. Each test which needs to do that (image installer!) should make a copy of ovmf.qcow2 first and use the copy.
I've filed this because I don't want to forget about it, and because perhaps someone else than myself can work on this?
Partly fixed by https://github.com/intel/intel-iot-refkit/pull/238/commits/01641909eaafc51941cec27ef2d830582a0f886c (for system update). Let's re-evaluate the impact on the current testing.
I believe this has been addressed sufficiently well.