Bug 12389

Summary: oe-selftest devtool tests fails when repository comes from a tarball
Product: [QA/Testing] Runtime Testing Reporter: Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Component: generalAssignee: Thomas Roos <thomas>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: bluelightning, randy.macleod, rebecca.swee.fun.chang, richard.purdie, thomas
Version: 2.4   
Target Milestone: 5.99   
Hardware: x86   
OS: Multiple   
Whiteboard: NEWCOMER
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Leonardo Sandoval Gonzalez 2017-11-29 21:04:07 UTC 
some of devtool selftests (those defined on meta/lib/oeqa/selftest/cases/devtool.py) fail when poky repository comes from a tarball (not git). According to the traceback/log, devtools tests expects the source code to be (git)tracked but investigation is needed.

steps to execute:

$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-2.4/poky-rocko-18.0.0.tar.bz2
$ tar xvf poky-rocko-18.0.0.tar.bz2
$ cd poky-rocko-18.0.0
$ source oe-init-build-env
$ oe-selftest -r devtool
..
..

2017-11-29 13:25:42,694 - oe-selftest - INFO - ======================================================================
2017-11-29 13:25:42,694 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_local_files_subdir (devtool.DevtoolTests)
2017-11-29 13:25:42,694 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-11-29 13:25:42,694 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/selftest/cases/devtool.py", line 1099, in test_devtool_update_recipe_local_files_subdir
    self._check_repo_status(os.path.dirname(recipefile), expected_status)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/selftest/cases/devtool.py", line 158, in _check_repo_status
    cwd=repo_dir)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'git status . --porcelain' returned non-zero exit status 128:
fatal: Not a git repository (or any of the parent directories): .git

2017-11-29 13:25:42,694 - oe-selftest - INFO - ======================================================================
2017-11-29 13:25:42,694 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_local_patch_gz (devtool.DevtoolTests)
2017-11-29 13:25:42,694 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-11-29 13:25:42,694 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/selftest/cases/devtool.py", line 1068, in test_devtool_update_recipe_local_patch_gz
    self._check_repo_status(os.path.dirname(recipefile), expected_status)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/selftest/cases/devtool.py", line 158, in _check_repo_status
    cwd=repo_dir)
  File "/home/lsandov1/tmp/poky-rocko-18.0.0/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'git status . --porcelain' returned non-zero exit status 128:
fatal: Not a git repository (or any of the parent directories): .git


2017-11-29 13:25:42,694 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-11-29 13:25:42,695 - oe-selftest - INFO - Ran 38 tests in 7190.118s
2017-11-29 13:25:42,695 - oe-selftest - INFO - FAILED
2017-11-29 13:25:42,695 - oe-selftest - INFO -  (failures=16, skipped=1)
2017-11-29 13:25:42,695 - oe-selftest - INFO - RESULTS:
2017-11-29 13:25:42,695 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_create_workspace - Testcase 1158: PASSED
2017-11-29 13:25:42,695 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add - Testcase 1159: PASSED
2017-11-29 13:25:42,695 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add_fetch - Testcase 1160: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add_fetch_git - Testcase 1161: FAILED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add_fetch_simple - Testcase 1391: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add_git_local - Testcase 1423: FAILED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add_library - Testcase 1162: FAILED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_build_image - Testcase 1366: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_buildclean - Testcase 1620: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_deploy_target - Testcase 1272: SKIPPED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_extract - Testcase 1163: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_extract_virtual - Testcase 1379: PASSED
2017-11-29 13:25:42,696 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_finish_modify_origlayer - Testcase 1621: FAILED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_finish_modify_otherlayer - Testcase 1622: FAILED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_finish_upgrade_origlayer - Testcase 1623: FAILED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_finish_upgrade_otherlayer - Testcase 1624: FAILED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_layer_plugins - Testcase 1352: PASSED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_load_plugin - Testcase 1625: PASSED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify - Testcase 1164: PASSED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify_git - Testcase 1165: PASSED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify_invalid - Testcase 1166: PASSED
2017-11-29 13:25:42,697 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify_localfiles - Testcase 1167: PASSED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify_native - Testcase 1365: PASSED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_modify_virtual - Testcase 1378: PASSED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_rename - Testcase 1626: PASSED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_reset_all - Testcase 1168: PASSED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe - Testcase 1169: FAILED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_append - Testcase 1170: FAILED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_append_git - Testcase 1171: FAILED
2017-11-29 13:25:42,698 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_git - Testcase 1172: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_local_files - Testcase 1370: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_local_files_2 - Testcase 1371: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_local_files_3 - Testcase 1627: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_local_files_subdir - Testcase 1628: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_update_recipe_local_patch_gz - Testcase 1629: FAILED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_upgrade - Testcase 1367: PASSED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_upgrade_git - Testcase 1433: PASSED
2017-11-29 13:25:42,699 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_virtual_kernel_modify - Testcase 1577: PASSED
2017-11-29 13:25:42,699 - oe-selftest - INFO - SUMMARY:
2017-11-29 13:25:42,699 - oe-selftest - INFO - oe-selftest () - Ran 38 tests in 7190.123s
2017-11-29 13:25:42,700 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (skipped=1)
Comment 1 Leonardo Sandoval Gonzalez 2017-11-29 21:05:43 UTC 
In general, the whole suite must be exercised (oe-selftest -a) based on release tarball and see if other areas are affected.
Comment 2 Richard Purdie 2019-06-20 11:21:28 UTC 
We should probably just skip the tests in this case if the tree isn't a git repo (or just error out upfront and refuse to run the tests at all if too many are affected).
Comment 3 Thomas Roos 2023-04-21 11:33:33 UTC 
What about making it a git before starting tests? -> git init?
Comment 4 Thomas Roos 2023-05-12 13:53:45 UTC 
Is there any possibility to skip a whole class of tests?
At test level skipping would be possible by something like this:

        try:
            runCmd("git rev-parse --is-inside-work-tree")
        except:
            self.skipTest("Devtool tests require folder to be a git repo")
Comment 5 Thomas Roos 2023-05-12 13:56:55 UTC 
In general those are the numbers for oe-selftest -a
2023-04-22 01:42:10,561 - oe-selftest - INFO - oe-selftest () - Ran 530 tests in 49170.163s
2023-04-22 01:42:10,561 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=472, skipped=10, failures=19, errors=33)

-> so we could skip to whole devtool class and then also skip others.
Comment 7 Richard Purdie 2023-05-25 09:37:23 UTC 
Thanks for sorting the patch out for this!
Comment 8 Randy MacLeod 2023-10-24 14:24:29 UTC 
Bulk move from 4.99 or 0.00 to 5.99