9253 2016-03-14 17:28:04 +0000 oe-selftest: coverage data file is given a truncated name 2016-05-05 11:33:58 +0000 1 1 10 QA/Testing Build Testing general 2.1 All Multiple RESOLVED FIXED Medium+ normal 2.1 M4 1 humberto.ibarra.lopez humberto.ibarra.lopez bogdanx.a.voiculescu francisco.j.pedraza.gonzalez leonardo.sandoval.gonzalez francisco.j.pedraza.gonzalez --- --- No (bug/feature does not impact docs) oldest_to_newest 59948 0 humberto.ibarra.lopez 2016-03-14 17:28:04 +0000 When using coverage in oe-selftest with --run-all-tests option, a .coverage.all_tests data file is generated; when using --run-tests a .coverage.<test_name> file is created. However, when using --run-tests-by the file name is truncated to only .coverage. Here is the coverage config generated: [run] data_file = /home/humberto/poky/build/.coverage. branch = True source = /home/humberto/poky/meta /home/humberto/poky/meta-poky /home/humberto/poky/meta-yocto-bsp /home/humberto/poky/meta-selftest /home/humberto/poky/scripts /home/humberto/poky/bitbake Here the file name listed: ~/poky/build $ ls -a . .. bitbake.lock cache conf .coverage. .coveragerc sstate-cache tmp This name should have a meaningful name such as .coverage.<info_of_tests_ran> sort of name. 59951 1 humberto.ibarra.lopez 2016-03-14 18:40:21 +0000 Found a new issue that should be fixed with the patch for this bug. When using oe-selftest --run-tests the name is a concatenation of all tests given; this can cause large filenames and coverage not being stored because of this. The following example is unable to get coverage because 5 tests were given: ~/poky/build $ oe-selftest --run-tests bblayers.BitbakeLayers.test_bitbakelayers_add_remove bblayers.BitbakeLayers.test_bitbakelayers_flatten bblayers.BitbakeLayers.test_bitbakelayers_showlayers bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends bblayers.BitbakeLayers.test_bitbakelayers_showrecipes --coverage 2016-03-14 12:01:41,755 - selftest - INFO - Running bitbake -e to get BBPATH 2016-03-14 12:01:42,703 - selftest - INFO - Checking that everything is in order before running the tests 2016-03-14 12:01:43,648 - selftest - INFO - Running bitbake -p 2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_add_remove 2016-03-14 12:01:46,173 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_flatten 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showlayers 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends 2016-03-14 12:01:46,174 - selftest - INFO - Loading tests from: oeqa.selftest.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes 2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include selftest.inc" in local.conf 2016-03-14 12:01:46,174 - selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2016-03-14 12:01:46,220 - selftest - INFO - Coverage is enabled 2016-03-14 12:01:47 - test_bitbakelayers_add_remove (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:25 - test_bitbakelayers_flatten (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:34 - test_bitbakelayers_showlayers (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:02:39 - test_bitbakelayers_showcrossdepends (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:03:23 - test_bitbakelayers_showrecipes (oeqa.selftest.bblayers.BitbakeLayers) ... ok 2016-03-14 12:04:13,540 - selftest - INFO - Coverage configuration file (/home/humberto/poky/build/.coveragerc) 2016-03-14 12:04:13,540 - selftest - INFO - =========================== 2016-03-14 12:04:13,540 - selftest - INFO - [run] data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes branch = True source = /home/humberto/poky/meta /home/humberto/poky/meta-poky /home/humberto/poky/meta-yocto-bsp /home/humberto/poky/meta-selftest /home/humberto/poky/scripts /home/humberto/poky/bitbake 2016-03-14 12:04:13,541 - selftest - INFO - Coverage Report 2016-03-14 12:04:13,541 - selftest - INFO - =============== 2016-03-14 12:04:13,543 - selftest - WARNING - No data to report. ---------------------------------------------------------------------- Ran 5 tests in 147.369s OK 2016-03-14 12:04:13,543 - selftest - INFO - Finished 2016-03-14 12:04:13,543 - selftest - INFO - Removing the include from local.conf 2016-03-14 12:04:13,544 - selftest - INFO - Removing the include from bblayers.conf As seen above the file name is huge: data_file = /home/humberto/poky/build/.coverage.bblayers.BitbakeLayers.test_bitbakelayers_add_remove.bblayers.BitbakeLayers.test_bitbakelayers_flatten.bblayers.BitbakeLayers.test_bitbakelayers_showlayers.bblayers.BitbakeLayers.test_bitbakelayers_showcrossdepends.bblayers.BitbakeLayers.test_bitbakelayers_showrecipes The file name for the coverage data should change to something unique, like a timestamp. Doing this would solve this issue and the first one referring to the truncated name. Moving from minor to normal since this problem had greater consequences than the original bug. 60350 2 humberto.ibarra.lopez 2016-03-23 16:26:50 +0000 Sent patch http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440.html 61359 3 francisco.j.pedraza.gonzalez 2016-04-18 21:27:11 +0000 (In reply to comment #2) > Sent patch > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > html Hi Humberto, Could you please add more information and details for steps to reproduce it?, I need to retest this one. Regards, 61362 4 leonardo.sandoval.gonzalez 2016-04-18 21:54:05 +0000 (In reply to comment #3) > (In reply to comment #2) > > Sent patch > > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119440. > > html > > Hi Humberto, > Could you please add more information and details for steps to reproduce > it?, I need to retest this one. > Regards, Paco, I believe I can help on this one based on the patch sent by Humberto. 1) Run any oe-selftest command executing a specific test suite, just including the --coverage argument to enable coverage 2) once finished, check the following $ grep data_file .coveragerc EXPECTED OUTPUT: 1) grep's output should have a unique value, corresponding to a filename 2) filename should be present and populated