| Summary: | CI: capture and display results of oe-selftest as JUnit XML | ||
|---|---|---|---|
| Product: | [Yocto Project Subprojects] IoT Reference OS Kit | Reporter: | Patrick Ohly <patrick.ohly> |
| Component: | intel-iot-refkit-tools | Assignee: | Olev Kartau <olev.kartau> |
| Status: | RESOLVED FIXED | QA Contact: | yeoh ee peng <ee.peng.yeoh> |
| Severity: | normal | ||
| Priority: | Medium | CC: | mikko.ylinen |
| Version: | 2.4 | ||
| Target Milestone: | 2.4 M2 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Patrick Ohly
2017-06-23 07:00:56 UTC
python3-unittest-xml-reporting is pre-packaged for opensuse, adding it to Docker image is now a commit in PR#205 test run of #205 where python3-unittest-xml-reporting was added, is here: https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1668/consoleFull but I don't see any directory called "TestResults_<date>_<number>" on worker, did I miss something about enabling this? Where exactly should this directory appear, on main level, under build/ or somewhere else? explanation is, python3-unittest-xml-reporting is not enough: ImportError: No module named 'six' with python3-six added as well, import xmlrunner works. I will create a new PR (as 205 got dedicated) and try. 1st part verified, PR build created TestResults_ directory with 5 xml-files. Test results parsed and published by jenkins: https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1781/testReport/ (result of testing PR#210) The TEST-*.xml files are also accessible under "Build Artifacts" section of that job. (In reply to comment #5) > Test results parsed and published by jenkins: > > https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1781/ > testReport/ > > (result of testing PR#210) > > The TEST-*.xml files are also accessible under "Build Artifacts" > section of that job. Looks nice! Is it possible to add the results in the email summary too? PR#210 now has email report part. I was thinking, what to write as name of image for these tests, initially I used "post-build", but that's not really image name, please propose better if yo have ideas. I guess Device=qemu is correctly expressed. In current run of PR#210 that part of report shows like: Image: post-build Device: qemu Total:16 Pass:15 Fail:0 Skip:1 Error:0 N/A:0 Run rate:93% Pass rate of total:93% Pass rate of exec:100% This part will be shown as first, as post-build testing is first to complete. (In reply to comment #7) > I was thinking, what to write as name of image for these tests, > initially I used "post-build", but that's not really image name, > please propose better if yo have ideas. > I guess Device=qemu is correctly expressed. We don't necessarily have to follow the same pattern. Some of the self-tests do use qemu but not all so Device=qemu is not 100% correct. How about simply oe-selftest: [pre|post]-build Total:16 Pass:15 Fail:0 Skip:1 Error:0 N/A:0 Run rate:93% Pass rate of total:93% Pass rate of exec:100% I had overlooked pre-test totally, there are XML files produced also. https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1893/console (near end of output) is example of how Summary will look like, if PR#210 gets merged as it is now. PR#210 was merged and this is now functional whats the known method to get python3 xmlrunner on Ubuntu 16.04.2 ? package-install based solution is preferred, but I dont find such packaged. There is python-xmlrunner package but it only provides xmlrunner for python2. Installing python3-junitxml and/or python3-unittest2 does not help. (In reply to comment #11) > whats the known method to get python3 xmlrunner on Ubuntu 16.04.2 ? Install the python3-pip package, then run (as root) "pip3 install unitest-xml-reporting". |