Created attachment 5072 [details] Screenshot of the page in error https://reproducible-builds.org/citests/ points to https://www.yoctoproject.org/reproducible-build-results/ which displays an error message: "Error fetching test results" and an empty template I'm no webdev but trying to debug : - The fetched URL is https://git.yoctoproject.org/yocto-testresults/plain/oeselftest/testresults.json?h=master - This is rejected with "CORS Missing Allow Origin" (as my browser says) - The URL is wrong anyway : The repo does exist but the file does not exist in the master branch (but did at some point).
This commit removed almost every test results: https://git.yoctoproject.org/yocto-testresults/commit/?id=c3db215cd0045a1b9eaa79f3d68fa716bc2bafa0 This is from branch: master commit: 9ad32cec8e045d580563631ac59f49dff4cae274 date: 2024-09-11 06:05:20 +0000 => This might be more a bug in AB result production
The webteam and I repaired the CORS issues back in March of this year. I can take another look at that once testresults.json is available again. Alexis, can you tell why the latest commits from the autobuilder are not adding a testresults.json file?
Alexis, could you answer Michael's question?
Sorry Michael for the answer delay, I am not actively monitoring Bugzilla, so thanks Tim for the notification. I'll take a look at it
Small update after some investigation: - for some reason, the a-full/a-quick builds have started detecting two results revisions to store, with one being bogus, leading systematically to a second commit removing the freshly versioned test result - the issue has started appearing on September, 5th, but I did not narrow exactly the corresponding change yet, despite the changelog being very small in Poky/oe - however I have spotted that the issue is somehow related to qemuarm-oecore: this one has started to generate some testresults starting from this date, and with a bogus "commit" parameter in the corresponding testresults.json
Some additions: - I still do not have a clue about why qemuarm-oecore has started to generate some testresults.json files a month ago, I did not find the relevant change either in oecore or in yocto-autobuilder-helper. qemuarm-oecore was already part of a-quick way before the issue appeared - the deeper issue is that this new testresults.json is wrong: the test results metadata bears an oecore revision in the LAYERS->meta field, which confuses resulttool (and so we end up pushing a commit erasing plenty of results on master branch in yocto-testresults). This field is expected to get only poky revisions
The extra testresults.json file is generated since commit ebcd355a32e2711263e22d9b45b502696ecbb4d2 in openembedded-core. It is no longer generated if we add back the `TMPDIR .= "-${TCLIBC}"` line in defaultsetup.conf. I need to continue the investigations to understand what really happens behind this.
Some quick news: - The commit in my previous commit merely remove the renaming of the tmp dir to tmp-glibc. - Before this commit, the testresults.json file already contained a reference oe-core instead of poky. Getting a closer look, it seems this has always been the case. - Before this commit, the results of qemuarm-oecore were never grabbed, probably because the tmp dir was renamed. https://valkyrie.yocto.io/pub/non-release/20240830-2/testresults/ - So actually it seems this commit FIX an issue we had, but also reveal another issue. - qemuarm-oecore is a bit special. While other builders rely on poky, this is the only one relying on oe-core. So referencing a commit of oe-core instead of poky in testresults.json makes sense, at least with my current understanding of how things work here. So, if my understanding is correct, there is no bug in the generation of testresults.json . So far I can see two solutions to this problem: - Either we modify the builder configuration to rely on poky git, as for the other builders. If we only add "meta" to our bblayers, the situation would mostly be the same. Still, I don't really like this solution, as it kind of goes against the test purpose of testing oe-core alone. - Or we acknowledge the qemuarm-oecore build is not the same as the others, and handle its testresults.json in a different way (ignore the oe-core revision or link it in some way to a poky revision). I need to think a bit about all of this. If anybody has any thought, feel free to share.
I've just merged a patch which filters the results being stored to a specific revision: https://git.yoctoproject.org/poky/commit/?id=f0d4814d4d1e198e7b6bb1a04d88125159e33d90 and I then enabled this in autobuilder-helper. With those two changes, only poky revisions should be stored in the testresults git repo and this reproducibility issue should be resolved. Quite where/why this changed behaviour remains unclear.
Issue has been solved by the change described in the previous comment.
Closing the bug