We are seeing failures in the testing of https://github.com/intel/intel-iot-refkit/pull/97 that look like they are caused by setting BUILD_ID=CI_BUILD_ID only in build.sh, but not the selftest jobs: there's a delay during selftest that implies that some real image building takes place, and the resulting images after that end up having the wrong filename.
I've added a fix to https://github.com/intel/intel-iot-refkit/pull/97
Yes, makes sense to define same BUILD_ID in all CI scripts, but it's somewhat disturbing that CI tester tries to test something that gets built in pre/post build stages. Means, it's probably good that tester failed as it was clearly trying to test something not built by builder script. Which brings back question, perhaps it is good after all to have different BUILD_ID used in pre/post scripts, pointing out overriden builds happening by some side effect, that we need start investigate.
I looked more closely to this job log (refkit CI PR build #539). Based on logs, images existence and timestamps, it seems that post-build script wiped images that were built by build script, then built images again with different BUILD_ID. This is clearly unwanted chain of events, which was caught thanks to differently set BUILD_ID, so I dont support idea of this bug title after all, as same BUILD_ID would make such hiddne wipe-and-replace possible. Post-build test must not affect build stage result. We may want to re-think our publishing flow, which was created before adding pre/post testing. We publish all result areas in all paths now, this serves idea that some results/logs are helpful even after failed/incomplete build stage. Current flow: pre-build,build,post-build,publish Perhaps publish should happen immediately after build phase, to make sure we publish what we build.
I overlooked idea that post-build test uses BUILD_ID to select built images, means my analysis was not entirely correct, and we need it set same way. But this case still points to publishing problem, as now there are mixed results from both build and post-build-test stages, which we publish together. We should not publish artifacts from post-build-test stage in same location. If these need publishing, another location should be used.
PR#100 created, adds BUILD_ID setting, plus restores some cleanroom principles: - dont keep build/ from after pre-build, - publish images right after build phase, so post-build additions dont get out.
PR#100 is merged