Bug 12085 - oeqa framework: cleanup the output when dependency test case is not found
Summary: oeqa framework: cleanup the output when dependency test case is not found
Status: RESOLVED OBSOLETE
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 4.99
Assignee: yeoh ee peng
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-09-14 19:17 UTC by Jose Perez C
Modified: 2020-04-01 06:46 UTC (History)
0 users

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jose Perez C 2017-09-14 19:17:07 UTC 
When using decorator @OETestDepends([<test-case-not-foud>]) the output is not really helpful a lot of information is displayed including all possible test cases available, one posible solution is to add format to the output to show the list of test cases or just show the error message and store all other information in a log. 


=== Current Output Example ===
Traceback (most recent call last):
  File "/home/jgperezc/poky-contrib/scripts/oe-selftest", line 70, in <module>
    ret = main()
  File "/home/jgperezc/poky-contrib/scripts/oe-selftest", line 57, in main
    results = args.func(logger, args)
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/selftest/context.py", line 253, in run
    rc = self._internal_run(logger, args)
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/selftest/context.py", line 205, in _internal_run
    self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/core/context.py", line 59, in loadTests
    self.suites = self.loader.discover()
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/core/loader.py", line 315, in discover
    cases = clss.discover(self.tc._registry)
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/core/decorator/depends.py", line 94, in discover
    _validate_test_case_depends(registry['cases'], registry['depends'])
  File "/home/jgperezc/poky-contrib/meta/lib/oeqa/core/decorator/depends.py", line 39, in _validate_test_case_depends
    ", cases available %s." % (case, dep, str(cases.keys())))
oeqa.core.exception.OEQADependency: TestCase devtool.DevtoolTests.test_devtool_export depends on devtool.DevtoolTests.test_devtool_import and isn't available, cases available KeysView(OrderedDict([('devtool.DevtoolTests.test_create_workspace', <devtool.DevtoolTests testMethod=test_create_workspace>), ('devtool.DevtoolTests.test_devtool_add', <devtool.DevtoolTests testMethod=test_devtool_add>), ('devtool.DevtoolTests.test_devtool_add_fetch', <devtool.DevtoolTests testMethod=test_devtool_add_fetch>), ('devtool.DevtoolTests.test_devtool_add_fetch_git', <devtool.DevtoolTests testMethod=test_devtool_add_fetch_git>), ('devtool.DevtoolTests.test_devtool_add_fetch_simple', <devtool.DevtoolTests testMethod=test_devtool_add_fetch_simple>), ('devtool.DevtoolTests.test_devtool_add_git_local', <devtool.DevtoolTests testMethod=test_devtool_add_git_local>), ('devtool.DevtoolTests.test_devtool_add_library', <devtool.DevtoolTests testMethod=test_devtool_add_library>), ('devtool.DevtoolTests.test_devtool_build_image', <devtool.DevtoolTests testMethod=test_devtool_build_image>), ('devtool.DevtoolTests.test_devtool_buildclean', <devtool.DevtoolTests testMethod=test_devtool_buildclean>), ('devtool.DevtoolTests.test_devtool_deploy_target', <devtool.DevtoolTests testMethod=test_devtool_deploy_target>), ('devtool.DevtoolTests.test_devtool_export', <devtool.DevtoolTests testMethod=test_devtool_export>), ('devtool.DevtoolTests.test_devtool_export_exclude_all', <devtool.DevtoolTests testMethod=test_devtool_export_exclude_all>), ('devtool.DevtoolTests.test_devtool_export_exclude_specific_file', <devtool.DevtoolTests testMethod=test_devtool_export_exclude_specific_file>), ('devtool.DevtoolTests.test_devtool_export_include', <devtool.DevtoolTests testMethod=test_devtool_export_include>), ('devtool.DevtoolTests.test_devtool_export_overwrite_file', <devtool.DevtoolTests testMethod=test_devtool_export_overwrite_file>), ('devtool.DevtoolTests.test_devtool_extract', <devtool.DevtoolTests testMethod=test_devtool_extract>), ('devtool.DevtoolTests.test_devtool_extract_virtual', <devtool.DevtoolTests testMethod=test_devtool_extract_virtual>), ('devtool.DevtoolTests.test_devtool_finish_modify_origlayer', <devtool.DevtoolTests testMethod=test_devtool_finish_modify_origlayer>), ('devtool.DevtoolTests.test_devtool_finish_modify_otherlayer', <devtool.DevtoolTests testMethod=test_devtool_finish_modify_otherlayer>), ('devtool.DevtoolTests.test_devtool_finish_upgrade_origlayer', <devtool.DevtoolTests testMethod=test_devtool_finish_upgrade_origlayer>), ('devtool.DevtoolTests.test_devtool_finish_upgrade_otherlayer', <devtool.DevtoolTests testMethod=test_devtool_finish_upgrade_otherlayer>), ('devtool.DevtoolTests.test_devtool_layer_plugins', <devtool.DevtoolTests testMethod=test_devtool_layer_plugins>), ('devtool.DevtoolTests.test_devtool_load_plugin', <devtool.DevtoolTests testMethod=test_devtool_load_plugin>), ('devtool.DevtoolTests.test_devtool_modify', <devtool.DevtoolTests testMethod=test_devtool_modify>), ('devtool.DevtoolTests.test_devtool_modify_git', <devtool.DevtoolTests testMethod=test_devtool_modify_git>), ('devtool.DevtoolTests.test_devtool_modify_invalid', <devtool.DevtoolTests testMethod=test_devtool_modify_invalid>), ('devtool.DevtoolTests.test_devtool_modify_localfiles', <devtool.DevtoolTests testMethod=test_devtool_modify_localfiles>), ('devtool.DevtoolTests.test_devtool_modify_native', <devtool.DevtoolTests testMethod=test_devtool_modify_native>), ('devtool.DevtoolTests.test_devtool_modify_virtual', <devtool.DevtoolTests testMethod=test_devtool_modify_virtual>), ('devtool.DevtoolTests.test_devtool_rename', <devtool.DevtoolTests testMethod=test_devtool_rename>), ('devtool.DevtoolTests.test_devtool_reset_all', <devtool.DevtoolTests testMethod=test_devtool_reset_all>), ('devtool.DevtoolTests.test_devtool_update_recipe', <devtool.DevtoolTests testMethod=test_devtool_update_recipe>), ('devtool.DevtoolTests.test_devtool_update_recipe_append', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_append>), ('devtool.DevtoolTests.test_devtool_update_recipe_append_git', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_append_git>), ('devtool.DevtoolTests.test_devtool_update_recipe_git', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_git>), ('devtool.DevtoolTests.test_devtool_update_recipe_local_files', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_local_files>), ('devtool.DevtoolTests.test_devtool_update_recipe_local_files_2', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_local_files_2>), ('devtool.DevtoolTests.test_devtool_update_recipe_local_files_3', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_local_files_3>), ('devtool.DevtoolTests.test_devtool_update_recipe_local_files_subdir', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_local_files_subdir>), ('devtool.DevtoolTests.test_devtool_update_recipe_local_patch_gz', <devtool.DevtoolTests testMethod=test_devtool_update_recipe_local_patch_gz>), ('devtool.DevtoolTests.test_devtool_upgrade', <devtool.DevtoolTests testMethod=test_devtool_upgrade>), ('devtool.DevtoolTests.test_devtool_upgrade_git', <devtool.DevtoolTests testMethod=test_devtool_upgrade_git>), ('devtool.DevtoolTests.test_devtool_virtual_kernel_modify', <devtool.DevtoolTests testMethod=test_devtool_virtual_kernel_modify>)])).
Comment 1 yeoh ee peng 2020-04-01 06:46:04 UTC 
The codebase had changed that it was no longer output large amount of possible test cases available.

Example: if test dependency not available, it will output concise error message below. 
Exception: oeqa.core.exception.OEQADependency: TestCase cyclictest.CyclicTest.test_cyclic depends on libipt.LibiptTest.test_libipt_can_generate_trace_packetXX and isn't available, cases available odict_keys(['ping.PingTest.test_ping', 'ssh.SSHTest.test_ssh', 'cyclictest.CyclicTest.test_cyclic']).

ERROR: Logfile of failure stored in: /data/eyeoh7/tmp/poky_qa/build-image-sato-sdk-meta-intel/tmp/work/intel_corei7_64-poky-linux/core-image-rt/1.0-r0/temp/log.do_testimage.49189
ERROR: Task (/data/eyeoh7/repo/meta-intel/recipes-rt/images/core-image-rt.bb:do_testimage) failed with exit code '1'