Whilst the outcome of tests like testimage, testsdk or oe-selftest are stored in testresult.json, there is no storage of "build" results like core-image-sato being built for qemuarm. We'd like to use testresults.json as the way to document testing for Yocto Project Compatible status of a layer. To do that we need to document the build tests completed as well as the runtime and selftests tests.
Hi Richard, I would like to propose my approach for this bug and seek for your opinions towards it. I created a patch which added a function to enable the system to write the build outputs into a testresults.json file and I would like to add those changes into the buildhistory.bbclass. Reason of applying those changes to buildhistory is because I had go through the buildhistory class and I found out that this class actually quite satisfied the goal of this bug, it is just that buildhistory doesn't write the build outputs into json files. So I created this patch to document the build outputs into a testresult.json file. This json file will be stored under a folder called 'buildresult'.
Created attachment 4824 [details] Sample output This is how the final testresults.json file will be generated
Any comment or feedback?
Richard, Can you look at the attachment and reply to Teoh?
Sorry, I didn't see the reply until now, Randy pinged me on irc since I get a lot (too much) email from bugzilla. I think the information we need in the testresults file needs to be tweaked a bit from this. For example, I think the image name should be a field in the json output in it's own right, not as part of a string of text. If we need that string, we should be able to generate it from the individual pieces of data. I also think the branch and revision of the layer should be separate and specified for each later rather than the ' " ' used in your example. We also need to think about which variables the autobuilder varies in it's tests such as package format, init system and so on as these need to be recorded too. The exact list of variables to be recoded probably needs to be configurable. So yes, buildhistory does recode some information but I think what we should put into testresult.json should be different.
Teoh, can you work on this for M1 please.
(In reply to comment #6) > Teoh, can you work on this for M1 please. Sure Randy. I will start working on it.
Teoh, if you can get to this, great but it's now unassigned.
Mistakenly moved this to M4. Since it's an enhancement, it should be 5.0 M1.
Bulk move of 52 5.0-M1 bugs to M2.