11963 2017-08-22 23:35:10 +0000 QA: Create automated testing to cover ESDK artifacts downloaded from the autobuilder 2024-06-06 15:13:10 +0000 1 1 10 QA/Testing Manual Testing manual-testing 2.4 x86 Multiple RESOLVED WORKSFORME Medium normal Future 1 libertad.gonzalez.de.la.cruz unassigned bluelightning brian.avery jose.perez.carranza joshuagloe libertad.gonzalez.de.la.cruz randy.macleod richard.purdie ross.burton --- --- Yes (doc changes required) oldest_to_newest 76066 0 libertad.gonzalez.de.la.cruz 2017-08-22 23:35:10 +0000 Currently ESDK testplan is missing scenarios where ESDK is installed from a previously built artifacts. Please add testcases that cover this scenarios 76067 1 libertad.gonzalez.de.la.cruz 2017-08-22 23:37:08 +0000 also build esdk for more than one image type. Currently it is only built for core-image-minimal 76500 2 jose.perez.carranza 2017-09-04 16:15:26 +0000 Patch sent to the list https://patchwork.openembedded.org/patch/143640/ Steps to execute test cases and the artifacts that should be used are doemnted here: https://wiki.yoctoproject.org/wiki/Extensible_SDK_Test_Plan_(eSDK)#Test_automation 77217 3 brian.avery 2017-09-29 00:19:25 +0000 I think this is getting a little too complex. At a high level we want to test 2 things 1)improve coverage of the eSDK testing on the autobuilder to catch regressions early. 2) exercise real word usage outside of the autobuilder to make sure our users don’t get any surprises This bug addresses #2. #1 is basically adding more tests to the testsd.bbclass #2 should be downloading the sdk to a machine outside of the AB and running a few devtool tests , the more coverage here the better. The linux kernel workflow is a good test since it is how a number of users will leverage the esdk in the next release given our new kernel documentation. #2 could be done by a script in scripts/contrib if desired. Furthermore, once we start copying things into a build dir and changing testdata.json paths (as the proposed patch does), we are not really testing end user usage. 77219 4 bluelightning 2017-09-29 00:53:27 +0000 Also, using devtool sdk-install -s isn't testing installing artifacts from the autobuilder, so we shouldn't be using -s. 77616 5 libertad.gonzalez.de.la.cruz 2017-10-16 19:59:07 +0000 I am starting investigation on this. 78609 6 joshuagloe 2017-12-08 14:46:23 +0000 Definition of a manual test case to test this functionality has been pulled out into a separate issue; #12411 Automation of the test case is now the scope of this issue. It's somewhat more complicated because as Brian stated in comment #3 we should perform this testing outside of the AB environment. Improving coverage of the eSDK by adding further automated test cases for use by testsdk should probably also be a separate bugzilla entry. 87295 7 richard.purdie 2020-05-21 08:27:17 +0000 This bug is about "post-release" tests to ensure the eSDK actually works. It can;'t be run until the release is actually available at its final location so has to be done post-release as part of the release process. 99147 8 randy.macleod 2024-06-06 15:13:10 +0000 Most issues identified in this bug have been resolved. The post-release tests while a good idea, are beyond our current resources.