11708 2017-06-23 07:00:56 +0000 CI: capture and display results of oe-selftest as JUnit XML 2017-07-13 06:25:58 +0000 1 1 6 Yocto Project Subprojects IoT Reference OS Kit intel-iot-refkit-tools 2.4 x86 Multiple RESOLVED FIXED Medium normal 2.4 M2 1 patrick.ohly olev.kartau mikko.ylinen ee.peng.yeoh --- --- No (bug/feature does not impact docs) oldest_to_newest 74507 0 patrick.ohly 2017-06-23 07:00:56 +0000 When python3 unittest-xml-reporting is installed (i.e. "import xmlrunner" works), then oe-selftest produces a directory called "TestResults_<date>_<number>" with individual .xml files. We should enable that in our CI system by installing unittest-xml-reporting for Python3 and publish the resulting .xml files so that Jenkins shows the same way as DAFT test results. unittest-xml-reporting wasn't packaged for Debian, but installation with "sudo pip3 install unittest-xml-reporting" worked fine. 74541 1 olev.kartau 2017-06-26 11:57:08 +0000 python3-unittest-xml-reporting is pre-packaged for opensuse, adding it to Docker image is now a commit in PR#205 74544 2 olev.kartau 2017-06-26 14:21:33 +0000 test run of #205 where python3-unittest-xml-reporting was added, is here: https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1668/consoleFull but I don't see any directory called "TestResults_<date>_<number>" on worker, did I miss something about enabling this? Where exactly should this directory appear, on main level, under build/ or somewhere else? 74569 3 olev.kartau 2017-06-27 10:42:56 +0000 explanation is, python3-unittest-xml-reporting is not enough: ImportError: No module named 'six' with python3-six added as well, import xmlrunner works. I will create a new PR (as 205 got dedicated) and try. 74576 4 olev.kartau 2017-06-27 16:00:41 +0000 1st part verified, PR build created TestResults_ directory with 5 xml-files. 74606 5 olev.kartau 2017-06-28 20:14:58 +0000 Test results parsed and published by jenkins: https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1781/testReport/ (result of testing PR#210) The TEST-*.xml files are also accessible under "Build Artifacts" section of that job. 74620 6 mikko.ylinen 2017-06-29 04:57:59 +0000 (In reply to comment #5) > Test results parsed and published by jenkins: > > https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1781/ > testReport/ > > (result of testing PR#210) > > The TEST-*.xml files are also accessible under "Build Artifacts" > section of that job. Looks nice! Is it possible to add the results in the email summary too? 74688 7 olev.kartau 2017-06-30 18:10:30 +0000 PR#210 now has email report part. I was thinking, what to write as name of image for these tests, initially I used "post-build", but that's not really image name, please propose better if yo have ideas. I guess Device=qemu is correctly expressed. In current run of PR#210 that part of report shows like: Image: post-build Device: qemu Total:16 Pass:15 Fail:0 Skip:1 Error:0 N/A:0 Run rate:93% Pass rate of total:93% Pass rate of exec:100% This part will be shown as first, as post-build testing is first to complete. 74702 8 mikko.ylinen 2017-07-03 06:36:35 +0000 (In reply to comment #7) > I was thinking, what to write as name of image for these tests, > initially I used "post-build", but that's not really image name, > please propose better if yo have ideas. > I guess Device=qemu is correctly expressed. We don't necessarily have to follow the same pattern. Some of the self-tests do use qemu but not all so Device=qemu is not 100% correct. How about simply oe-selftest: [pre|post]-build Total:16 Pass:15 Fail:0 Skip:1 Error:0 N/A:0 Run rate:93% Pass rate of total:93% Pass rate of exec:100% 74722 9 olev.kartau 2017-07-03 20:24:36 +0000 I had overlooked pre-test totally, there are XML files produced also. https://ostroproject.org/jenkins/job/intel-iot-refkit_pull-requests/1893/console (near end of output) is example of how Summary will look like, if PR#210 gets merged as it is now. 74804 10 olev.kartau 2017-07-06 12:04:45 +0000 PR#210 was merged and this is now functional 74990 11 olev.kartau 2017-07-13 06:13:09 +0000 whats the known method to get python3 xmlrunner on Ubuntu 16.04.2 ? package-install based solution is preferred, but I dont find such packaged. There is python-xmlrunner package but it only provides xmlrunner for python2. Installing python3-junitxml and/or python3-unittest2 does not help. 74991 12 patrick.ohly 2017-07-13 06:25:58 +0000 (In reply to comment #11) > whats the known method to get python3 xmlrunner on Ubuntu 16.04.2 ? Install the python3-pip package, then run (as root) "pip3 install unitest-xml-reporting".