for this my memory may be vague. I recalled some place may contain this information, but didn't find it. Anyway, it may be worthy of a complete doc for how to run this automated test and also how to develop sanity test cases
(In reply to comment #0) > for this my memory may be vague. I recalled some place may contain this > information, but didn't find it. Anyway, it may be worthy of a complete doc for > how to run this automated test and also how to develop sanity test cases Yes. I used to write a wiki for sanity test how to, https://wiki.pokylinux.org/wiki/Enabling_Automation_Test_in_Poky. I can enhance the content in it since there are some changes these days. But I think for formal documentation, it's better to have ScottR write one.
I need the raw information for this to assess how much of a doc this amounts to.
Kevin, Jiajun: Do you have the info Scott asked? Thanks, Alex.
This is an old one :) Basically all the information that exists at the moment on automated sanity tests for qemu targets is contained in that wiki page. I believe Kevin was looking for that. More information I have on the subject: -current setup works only for qemu image testing -all tests are bash scripts -all test scripts use functions defined in /scripts/qemuimage-testlib (these are handy) -current tests are located in /scripts/qemuimage-tests/* excluding /scripts/qemuimage-tests/scenario -scenarios are groups of tests categorized by architecture and target image type How it works: -after the qemu image is built, it is started using runqemu -it uses a fixed wait timer to estime how long it takes the image to boot (I belive it's 60 seconds or so) -after image is booted, a ssh connection is setup as root user -the shell scripts are copied on the target machine and the output is captured -using the output, a PASS or FAIL tag is added to that specific test -the the tests are conducted and the complete result is stored in /opt/test/ on the target machine Please note that we are planning to build on this. We would add more tests and ability to test BSP as well, though we don;t have a time frame at the moment.
Here is how we will distribute the revised BB manual. * the "made" manuals will live alongside the current YP manuals on the yoctoproject.org server. Before release time I will make a current version and an in progress version from the OE git repository. I will publish these to the server, respectively. Users will be able to access the manuals from the list of YP documentation. * the repository for the BB manual will continue to live where it does with OE so that will not disrupt people that understand where it is. * when Bill Traynor has Richard Purdie pull his area (wmat) into the OE repo, the result will be a yocto-ized structure that has multiple chapters, make file, etc. this will be done before the 1.4 release at some point. Bill is targeting end of March 2013 to lock it up. At that point, he will tag the repo so we have a good 1.4 commit for the manual. Work can continue on the tip where it can be incorporated into the "in progress" result just as we do with the YP manuals. Scott
Ignore Comment 5. was meant for a different bug... sorry.
I seem to recall discussing this bug with Richard Purdie and it was decided to wait for full blown documentation until all tests were flushed out (see the last bit in comment 4 for how these are maybe not yet complete). In the meantime, there are two classes in the YP Reference Manual that reference sanity testing in general: * Host System Sanity Checks (http://www.yoctoproject.org/docs/1.4/ref-manual/ref-manual.html#ref-sanity-checks) * Generated Output Quality Assurance Checks (http://www.yoctoproject.org/docs/1.4/ref-manual/ref-manual.html#ref-classes-insane) What I was considering doing for now is to include two notes (one for each section) that point to the wiki page referenced in Comment 1 for for additional information on running sanity checks. If a user is going to search for "sanity" in the mega-manual he will find this area and can learn more from the wiki at this point. Let me know if this solution is good for 1.4. Scott
*** Bug 3680 has been marked as a duplicate of this bug. ***
Since we are working on more automation and possibly changing the test scripts, lets move this to 1.5
Pushing out to post 1.5.
Please look at my comment 7 and see if this is still a valid solution for this bug. Scott
Hi, I consulted Paul Eggleton and we have this issue documented now. He has updated the wiki (https://wiki.yoctoproject.org/wiki/Enabling_Automation_Test_in_Poky) page such that it references back into our YP documentation. I am marking the bug RESOLVED and updating the doc flag to "Done". Scott
Documentation is in place. Setting to VERIFIED.