When using coverage in oe-selftest with --run-all-tests option, a .coverage.all_tests data file is generated; when using --run-tests a .coverage.<test_name> file is created. However, when using --run-tests-by the file name is truncated to only .coverage. Here is the coverage config generated: [run] data_file = /home/humberto/poky/build/.coverage. branch = True source = /home/humberto/poky/meta /home/humberto/poky/meta-poky /home/humberto/poky/meta-yocto-bsp /home/humberto/poky/meta-selftest /home/humberto/poky/scripts /home/humberto/poky/bitbake Here the file name listed: ~/poky/build $ ls -a . .. bitbake.lock cache conf .coverage. .coveragerc sstate-cache tmp This name should have a meaningful name such as .coverage.<info_of_tests_ran> sort of name.
Found a new issue that should be fixed with the patch for this bug. When using oe-selftest --run-tests the name is a concatenation of all tests given; this can cause large filenames and coverage not being stored because of this. The following example is unable to get coverage because 5 tests were given: ~/poky/build $ oe-selftest --run-tests bblayers.BitbakeLayers.test_bitbakelayers_add_remove bblayers.BitbakeLayers.test_bitbakelayers_flatten bblayers.BitbakeLayers.test_bitbakelayers_showlayers bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends bblayers.BitbakeLayers.test_bitbakelayers_showrecipes --coverage 2016-03-14 12:01:41,755 - selftest - INFO - Running bitbake -e to get BBPATH 2016-03-14 12:01:42,703 - selftest - INFO - Checking that everything is in order before running the tests 2016-03-14 12:01:43,648 - selftest - INFO - Running bitbake -p 2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_add_remove 2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_flatten 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showlayers 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes 2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include selftest.inc" in local.conf 2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2016-03-14 12:01:46,220 - selftest - INFO - Coverage is enabled 2016-03-14 12:01:47 - test_bitbakelayers_add_remove (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:25 - test_bitbakelayers_flatten (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:34 - test_bitbakelayers_showlayers (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:39 - test_bitbakelayers_showcrossdepends (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:03:23 - test_bitbakelayers_showrecipes (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:04:13,540 - selftest - INFO - Coverage configuration file (/home/humberto/poky/build/.coveragerc) 2016-03-14 12:04:13,540 - selftest - INFO - =========================== 2016-03-14 12:04:13,540 - selftest - INFO - [run] data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes branch = True source = /home/humberto/poky/meta /home/humberto/poky/meta-poky /home/humberto/poky/meta-yocto-bsp /home/humberto/poky/meta-selftest /home/humberto/poky/scripts /home/humberto/poky/bitbake 2016-03-14 12:04:13,541 - selftest - INFO - Coverage Report 2016-03-14 12:04:13,541 - selftest - INFO - =============== 2016-03-14 12:04:13,543 - selftest - WARNING - No data to report. ---------------------------------------------------------------------- Ran 5 tests in 147.369s OK 2016-03-14 12:04:13,543 - selftest - INFO - Finished 2016-03-14 12:04:13,543 - selftest - INFO - Removing the include from local.conf 2016-03-14 12:04:13,544 - selftest - INFO - Removing the include from bblayers.conf As seen above the file name is huge: data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes The file name for the coverage data should change to something unique, like a timestamp. Doing this would solve this issue and the first one referring to the truncated name. Moving from minor to normal since this problem had greater consequences than the original bug.
Sent patch http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440.html
(In reply to comment #2) > Sent patch > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > html Hi Humberto, Could you please add more information and details for steps to reproduce it?, I need to retest this one. Regards,
(In reply to comment #3) > (In reply to comment #2) > > Sent patch > > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > > html > > Hi Humberto, > Could you please add more information and details for steps to reproduce > it?, I need to retest this one. > Regards, Paco, I believe I can help on this one based on the patch sent by Humberto. 1) Run any oe-selftest command executing a specific test suite, just including the --coverage argument to enable coverage 2) once finished, check the following $ grep data_file .coveragerc EXPECTED OUTPUT: 1) grep's output should have a unique value, corresponding to a filename 2) filename should be present and populated