| Summary: | Exporting Tests section can lead to failed tests without proper specification | ||
|---|---|---|---|
| Product: | [Documentation] Mega Manual | Reporter: | Benjamin Esquivel <benjamin.esquivel> |
| Component: | mega-manual | Assignee: | Scott Rifenbark <srifenbark> |
| Status: | RESOLVED FIXED | QA Contact: | Benjamin Esquivel <benjamin.esquivel> |
| Severity: | major | ||
| Priority: | Medium | CC: | mariano.lopez, srifenbark |
| Version: | 2.2 | ||
| Target Milestone: | 2.3 M2 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | 14 Nov 2016: RESOLVED | ||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Done (doc changes complete) | |
Suggestion:
"""
You can export tests so that they can run independently of the build system. Exporting tests is required if you want to be able to hand the test execution off to a scheduler. You can only export tests that are defined in TEST_SUITES.
If your image is already built (assuming core-image-sato for further instructions), make sure the following variables are set in your local.conf file:
INHERIT +="testexport"
TEST_TARGET_IP = "192.168.7.2"
TEST_SERVER_IP = "192.168.7.1"
Take into account that these are example IP addresses, it is required to be check these addresses correspond to the server and the target. The correct setup can be verified executing the ping command from host to target and from target to host.
You can then export the tests with the following:
$ bitbake core-image-sato -c testexport
Exporting the tests places them in the Build Directory in tmp/testexport/core-image-sato, which can be controlled via the TEST_EXPORT_DIR variable.
"""
Benjamin, See http://www.yoctoproject.org/docs/2.2/dev-manual/dev-manual.html#exporting-tests and let me know what you think. Thanks, Scott It looks good to me. I'd appreciate Mariano's input as well since he wrote the feature. Added myself to the CC list. It is clearer specifying sato as the image for the example, but now it can be confusing with this part: """ Take into account that these are example IP addresses, it is required to be check these addresses correspond to the server and the target. The correct setup can be verified executing the ping command from host to target and from target to host. """ I would add: """ The IP address here will be kept as the default IPs in the configuration file generated to run the test, but can be override using the script to run the test (runexported.py) """ Mariano, Are you looking at http://www.yoctoproject.org/docs/2.2/dev-manual/dev-manual.html#exporting-tests for the new updates? I don't see the following in the current version of the manual: """ Take into account that these are example IP addresses, it is required to be check these addresses correspond to the server and the target. The correct setup can be verified executing the ping command from host to target and from target to host. """ Is your last comment still valid about adding the following: """ The IP address here will be kept as the default IPs in the configuration file generated to run the test, but can be override using the script to run the test (runexported.py) """ Thanks, Scott (In reply to comment #6) > Mariano, > > Are you looking at > http://www.yoctoproject.org/docs/2.2/dev-manual/dev-manual.html#exporting- > tests for the new updates? I don't see the following in the current version > of the manual: > I was referring to Benjamin's comment on how should end the documentation, not the current manual. Setting to RESOLVED and putting the doc flag to "done." Scott |
The mega manual section[1] states: """ If your image is already built, make sure the following are set in your local.conf file. Be sure to provide the IP address you need: INHERIT +="testexport" TEST_TARGET_IP = "192.168.7.2" TEST_SERVER_IP = "192.168.7.1" You can then export the tests with the following: $ bitbake core-image-sato -c testexport Exporting the tests places them in the Build Directory in tmp/testexport/core-image-sato, which is controlled by the TEST_EXPORT_DIR variable. """ This can lead to failures or confusion, at the first paragraph it is accepted that any image can be used then at further instructions it is locked-up to a particular image, in this case core-image-sato. The failure is at the test list exporting because no matter what image is selected for testing, the core-image-sato test list will be obtained always. I will place a suggestion in the next comment. [1] http://www.yoctoproject.org/docs/2.2/mega-manual/mega-manual.html#exporting-tests