| Summary: | oe-selftest: coverage data file is given a truncated name | ||
|---|---|---|---|
| Product: | [QA/Testing] Build Testing | Reporter: | Humberto Ibarra <humberto.ibarra.lopez> |
| Component: | general | Assignee: | Humberto Ibarra <humberto.ibarra.lopez> |
| Status: | RESOLVED FIXED | QA Contact: | Francisco Pedraza <francisco.j.pedraza.gonzalez> |
| Severity: | normal | ||
| Priority: | Medium+ | CC: | bogdanx.a.voiculescu, francisco.j.pedraza.gonzalez, leonardo.sandoval.gonzalez |
| Version: | 2.1 | ||
| Target Milestone: | 2.1 M4 | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
Found a new issue that should be fixed with the patch for this bug.
When using oe-selftest --run-tests the name is a concatenation of all tests given; this can cause large filenames and coverage not being stored because of this. The following example is unable to get coverage because 5 tests were given:
~/poky/build $ oe-selftest --run-tests bblayers.BitbakeLayers.test_bitbakelayers_add_remove bblayers.BitbakeLayers.test_bitbakelayers_flatten bblayers.BitbakeLayers.test_bitbakelayers_showlayers bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends bblayers.BitbakeLayers.test_bitbakelayers_showrecipes --coverage
2016-03-14 12:01:41,755 - selftest - INFO - Running bitbake -e to get BBPATH
2016-03-14 12:01:42,703 - selftest - INFO - Checking that everything is in order before running the tests
2016-03-14 12:01:43,648 - selftest - INFO - Running bitbake -p
2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_add_remove
2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_flatten
2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showlayers
2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends
2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes
2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include selftest.inc" in local.conf
2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2016-03-14 12:01:46,220 - selftest - INFO - Coverage is enabled
2016-03-14 12:01:47 - test_bitbakelayers_add_remove (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-14 12:02:25 - test_bitbakelayers_flatten (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-14 12:02:34 - test_bitbakelayers_showlayers (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-14 12:02:39 - test_bitbakelayers_showcrossdepends (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-14 12:03:23 - test_bitbakelayers_showrecipes (oeqa.selftest.bblayers.BitbakeLayers) ... ok
2016-03-14 12:04:13,540 - selftest - INFO - Coverage configuration file (/home/humberto/poky/build/.coveragerc)
2016-03-14 12:04:13,540 - selftest - INFO - ===========================
2016-03-14 12:04:13,540 - selftest - INFO -
[run]
data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes
branch = True
source =
/home/humberto/poky/meta
/home/humberto/poky/meta-poky
/home/humberto/poky/meta-yocto-bsp
/home/humberto/poky/meta-selftest
/home/humberto/poky/scripts
/home/humberto/poky/bitbake
2016-03-14 12:04:13,541 - selftest - INFO - Coverage Report
2016-03-14 12:04:13,541 - selftest - INFO - ===============
2016-03-14 12:04:13,543 - selftest - WARNING - No data to report.
----------------------------------------------------------------------
Ran 5 tests in 147.369s
OK
2016-03-14 12:04:13,543 - selftest - INFO - Finished
2016-03-14 12:04:13,543 - selftest - INFO - Removing the include from local.conf
2016-03-14 12:04:13,544 - selftest - INFO - Removing the include from bblayers.conf
As seen above the file name is huge:
data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes
The file name for the coverage data should change to something unique, like a timestamp. Doing this would solve this issue and the first one referring to the truncated name.
Moving from minor to normal since this problem had greater consequences than the original bug.
(In reply to comment #2) > Sent patch > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > html Hi Humberto, Could you please add more information and details for steps to reproduce it?, I need to retest this one. Regards, (In reply to comment #3) > (In reply to comment #2) > > Sent patch > > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > > html > > Hi Humberto, > Could you please add more information and details for steps to reproduce > it?, I need to retest this one. > Regards, Paco, I believe I can help on this one based on the patch sent by Humberto. 1) Run any oe-selftest command executing a specific test suite, just including the --coverage argument to enable coverage 2) once finished, check the following $ grep data_file .coveragerc EXPECTED OUTPUT: 1) grep's output should have a unique value, corresponding to a filename 2) filename should be present and populated |
When using coverage in oe-selftest with --run-all-tests option, a .coverage.all_tests data file is generated; when using --run-tests a .coverage.<test_name> file is created. However, when using --run-tests-by the file name is truncated to only .coverage. Here is the coverage config generated: [run] data_file = /home/humberto/poky/build/.coverage. branch = True source = /home/humberto/poky/meta /home/humberto/poky/meta-poky /home/humberto/poky/meta-yocto-bsp /home/humberto/poky/meta-selftest /home/humberto/poky/scripts /home/humberto/poky/bitbake Here the file name listed: ~/poky/build $ ls -a . .. bitbake.lock cache conf .coverage. .coveragerc sstate-cache tmp This name should have a meaningful name such as .coverage.<info_of_tests_ran> sort of name.