Bug 10937

Summary: remove from oe-selftest buildsets the step for building an image prior the test session
Product: [Infrastructure] AutoBuilder Reporter: Benjamin Esquivel <benjamin.esquivel>
Component: autobuilderAssignee: Joshua Lock <joshuagloe>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: infras.ab.watcher, Infras.watcher, mariano.lopez, monserratx.sedeno.bustos, pidge, sonia.leon.bautista
Version: 2.2   
Target Milestone: 2.4 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on: 11118    
Bug Blocks: 10035    

Description Benjamin Esquivel 2017-01-17 17:34:17 UTC 
since the tests should be self-sufficient in their dependencies for images built prior testing, having a step prior the testing for bitbake core-image-minimal makes no sense.

Removing this step would improve the testing time for over half an hour and should have zero impact over the testing. If there are tests failing then those should be fixed to be self-sufficient.
Comment 1 Joshua Lock 2017-01-26 11:34:54 UTC 
M2 rc1 has been built, moving this to M3
Comment 2 Joshua Lock 2017-02-23 16:57:03 UTC 
We've seen multiple hangs when testing this change, it's clear there's more work to be done than just removing the BuildImage step. It's not yet clear what that work is.
Comment 3 Joshua Lock 2017-03-13 14:36:27 UTC 
This didn't make it for 2.3 M3, as it's an enhancement push to 2.4 M1
Comment 4 Joshua Lock 2017-08-01 18:59:14 UTC 
Only Failure was test_checkpkg (distrodata.Distrodata), which is known to be flaky.
Comment 5 Joshua Lock 2017-08-04 15:55:11 UTC 
morty builds fail with this change, they seem to require the BuildImages step have run before oe-selftest is invoked.

I replicated this locally, with clean build directories and a checkout of the morty branch:
i) invoking oe-selftest on the clean build directory results in the build being "stuck" at the "Running tests..." output of oe-selftest
ii) building core-image-minimal in a clean build directory, then invoking oe-selftest things proceed as expected.
Comment 6 Joshua Lock 2017-08-08 15:16:25 UTC 
We'll achieve this by replacing the "core-image-minimal" BuildImages target with a placeholder value, for releases older than Pyro we'll replace the placeholder with core-image-minimal and for newer release we'll clear the placeholder.