Bug 11357 - oe-selftest: devtool tests don't preserve command output for showing on failure
Summary: oe-selftest: devtool tests don't preserve command output for showing on failure
Status: RESOLVED FIXED
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 4.99
Assignee: yeoh ee peng
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-04-13 12:41 UTC by Paul Eggleton
Modified: 2020-04-09 04:07 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Eggleton 2017-04-13 12:41:07 UTC
If a test fails, e.g. devtool.DevtoolTests.test_devtool_modify because the expected source file wasn't extracted, then we never see the actual devtool command output which might have been useful:

======================================================================
FAIL [33.107s]: test_devtool_modify
(oeqa.selftest.devtool.DevtoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/build1/poky/meta/lib/oeqa/utils/decorators.py", line
109, in wrapped_f
    return func(*args, **kwargs)
  File "/media/build1/poky/meta/lib/oeqa/selftest/devtool.py", line
446, in test_devtool_modify
    self.assertTrue(os.path.exists(os.path.join(tempdir, 'Makefile')),
'Extracted source could not be found')
AssertionError: False is not true : Extracted source could not be found
======================================================================

We should change the devtool tests to be printing out the output from the last run command when the assertion fails.

(I've filed bug 11356 for the "false is not true" nonsense, so ignore that for now).
Comment 1 yeoh ee peng 2020-04-09 04:03:07 UTC
DevtoolTests had been enhanced, it was no longer using assertTrue and the last run command output had been used in the assertion fails. 

result = runCmd('devtool modify mdadm -x %s' % tempdir)
        self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
        self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
        matches = glob.glob(os.path.join(self.workspacedir, 'appends', 'mdadm_*.bbappend'))
        self.assertTrue(matches, 'bbappend not created %s' % result.output)
Comment 2 yeoh ee peng 2020-04-09 04:07:51 UTC
DevtoolTests.test_devtool_modify had been enhanced by community, it was no longer using assertTrue and the last run command output had been used in the assertion fails. 

result = runCmd('devtool modify mdadm -x %s' % tempdir)
        self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
        self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
        matches = glob.glob(os.path.join(self.workspacedir, 'appends', 'mdadm_*.bbappend'))
        self.assertTrue(matches, 'bbappend not created %s' % result.output)