| Summary: | QA: Create automated testing to cover ESDK artifacts downloaded from the autobuilder | ||
|---|---|---|---|
| Product: | [QA/Testing] Manual Testing | Reporter: | Libertad <libertad.gonzalez.de.la.cruz> |
| Component: | manual-testing | Assignee: | Unassigned <unassigned> |
| Status: | RESOLVED WORKSFORME | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | bluelightning, brian.avery, jose.perez.carranza, joshuagloe, libertad.gonzalez.de.la.cruz, randy.macleod, richard.purdie, ross.burton |
| Version: | 2.4 | ||
| Target Milestone: | Future | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
|
Description
Libertad
2017-08-22 23:35:10 UTC
also build esdk for more than one image type. Currently it is only built for core-image-minimal Patch sent to the list https://patchwork.openembedded.org/patch/143640/ Steps to execute test cases and the artifacts that should be used are doemnted here: https://wiki.yoctoproject.org/wiki/Extensible_SDK_Test_Plan_(eSDK)#Test_automation I think this is getting a little too complex. At a high level we want to test 2 things 1)improve coverage of the eSDK testing on the autobuilder to catch regressions early. 2) exercise real word usage outside of the autobuilder to make sure our users don’t get any surprises This bug addresses #2. #1 is basically adding more tests to the testsd.bbclass #2 should be downloading the sdk to a machine outside of the AB and running a few devtool tests , the more coverage here the better. The linux kernel workflow is a good test since it is how a number of users will leverage the esdk in the next release given our new kernel documentation. #2 could be done by a script in scripts/contrib if desired. Furthermore, once we start copying things into a build dir and changing testdata.json paths (as the proposed patch does), we are not really testing end user usage. Also, using devtool sdk-install -s isn't testing installing artifacts from the autobuilder, so we shouldn't be using -s. I am starting investigation on this. Definition of a manual test case to test this functionality has been pulled out into a separate issue; #12411 Automation of the test case is now the scope of this issue. It's somewhat more complicated because as Brian stated in comment #3 we should perform this testing outside of the AB environment. Improving coverage of the eSDK by adding further automated test cases for use by testsdk should probably also be a separate bugzilla entry. This bug is about "post-release" tests to ensure the eSDK actually works. It can;'t be run until the release is actually available at its final location so has to be done post-release as part of the release process. Most issues identified in this bug have been resolved. The post-release tests while a good idea, are beyond our current resources. |