Bug 9334

Summary: oe-selftest: error output shown with --coverage when coverage module not installed
Product: [QA/Testing] Build Testing Reporter: Humberto Ibarra <humberto.ibarra.lopez>
Component: generalAssignee: Humberto Ibarra <humberto.ibarra.lopez>
Status: RESOLVED FIXED QA Contact:
Severity: minor    
Priority: Medium CC: richard.purdie
Version: 2.1   
Target Milestone: 2.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Humberto Ibarra 2016-03-23 16:07:48 UTC
Bug 8930 automated the coverage sub-process initialization process, but it did not clean warning and error messages properly.

build $ oe-selftest --run-tests-by module bblay* --coverage
Error processing line 1 of /home/humberto/.local/lib/python2.7/site-packages/coverage.pth:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site.py", line 158, in addpackage
      exec line
    File "<string>", line 1, in <module>
  ImportError: No module named coverage

Remainder of file ignored
2016-03-23 18:02:04,115 - selftest - INFO - Running bitbake -e to get BBPATH
DEBUG: tc id missing for test_sanity_unsafe_binary_references
DEBUG: tc id missing for test_read_only_image
DEBUG: tc id missing for test_devtool_add_git_local
DEBUG: tc id missing for test_devtool_upgrade_git
DEBUG: tc id missing for test_recipetool_create_cmake
DEBUG: tc id missing for test_qemu
2016-03-23 18:02:04,958 - selftest - INFO - Checking that everything is in order before running the tests
2016-03-23 18:02:05,802 - selftest - INFO - Running bitbake -p
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_add_remove
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_flatten
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showappends
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showlayers
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showoverlayed
2016-03-23 18:02:08,114 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes
2016-03-23 18:02:08,114 - selftest - INFO - Adding: "include selftest.inc" in local.conf
2016-03-23 18:02:08,115 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2016-03-23 18:02:08,115 - selftest - WARNING - python coverage is not installed
Make sure your coverage takes into account sub-process
More info on https://pypi.python.org/pypi/coverage
2016-03-23 18:02:08 - test_bitbakelayers_add_remove (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:13 - test_bitbakelayers_flatten (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:16 - test_bitbakelayers_showappends (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:19 - test_bitbakelayers_showcrossdepends (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:23 - test_bitbakelayers_showlayers (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:23 - test_bitbakelayers_showoverlayed (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-23 18:02:24 - test_bitbakelayers_showrecipes (oeqa.selftest.bblayers.BitbakeLayers) ... ok

----------------------------------------------------------------------
Ran 7 tests in 25.207s

OK
2016-03-23 18:02:33,322 - selftest - INFO - Finished
2016-03-23 18:02:33,322 - selftest - INFO - Removing the include from local.conf
2016-03-23 18:02:33,322 - selftest - INFO - Removing the include from bblayers.conf


There are 2 issues here:


1) The exception at the start of the command occurs because oe-selftest is creating a temporal coverage file for coverage subprocessing and coverage is not even installed. The file shouldn't have been created.

2) The warning message:

2016-03-23 18:02:08,115 - selftest - WARNING - python coverage is not installed
Make sure your coverage takes into account sub-process
More info on https://pypi.python.org/pypi/coverage

should be changed. The "Make sure your coverage takes into account sub-process" does no longer apply since this is now done automatically by the script.
Comment 1 Richard Purdie 2016-03-24 15:13:28 UTC
The thing which worries me more is that the system doesn't stop after the failure or set an error exit code?
Comment 2 Humberto Ibarra 2016-03-28 15:48:21 UTC
The script doesn't stop after the coverage module isn't found because the script is not trying to find the coverage module. A different file is trying to do the import.

The coverage module is imported by a temporary file saved under the user site directory. Python tries to import it every time it runs.

The problem is that oe-selftest creates this file when given the --coverage option, even if coverage is not installed.