| Summary: | oe-selftest testtools.testresult.real._StringException: lost connection during failure | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Armin Kuster <akuster> |
| Component: | core | Assignee: | Richard Purdie <richard.purdie> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | High | CC: | meta.mr.watcher, meta.watcher, richard.purdie, tvgamblin |
| Version: | 3.1 | ||
| Target Milestone: | 3.1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| URL: | https://errors.yoctoproject.org/Errors/Details/282873/ | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
Armin Kuster
2019-11-24 04:39:16 UTC
Targeting this bug specifically at the "lost connection" error. Its not test specific, you can do:
diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index 78189a4e6d..61138f608a 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -128,6 +128,8 @@ class ReproducibleTests(OESelftestTestCase):
shutil.copyfile(source, dest)
def test_reproducible_builds(self):
+ self.fail("The following %s packages are missing or different")
+
capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes]
pokybuild@debian8-ty-1:~/yocto-worker/oe-selftest-debian/build/build$ oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
2019-11-27 21:31:14,859 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it
2019-11-27 21:31:20,750 - oe-selftest - INFO - Adding layer libraries:
2019-11-27 21:31:20,751 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib
2019-11-27 21:31:20,751 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta-yocto-bsp/lib
2019-11-27 21:31:20,751 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta-selftest/lib
2019-11-27 21:31:20,754 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable
2019-11-27 21:31:22,758 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/pokybuild/yocto-worker/oe-selftest-debian/build/build/conf/local.conf
2019-11-27 21:31:22,759 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2019-11-27 21:31:22,760 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests)
2019-11-27 21:31:24,898 - oe-selftest - INFO - ... FAIL
2019-11-27 21:31:24,898 - oe-selftest - INFO - Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 131, in test_reproducible_builds
self.fail("The following %s packages are missing or different")
AssertionError: The following %s packages are missing or different
2019-11-27 21:31:24,899 - oe-selftest - INFO - ======================================================================
2019-11-27 21:31:24,900 - oe-selftest - INFO - FAIL: test_reproducible_builds (reproducible.ReproducibleTests)
2019-11-27 21:31:24,900 - oe-selftest - INFO - ----------------------------------------------------------------------
2019-11-27 21:31:24,900 - oe-selftest - INFO - Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 131, in test_reproducible_builds
self.fail("The following %s packages are missing or different")
AssertionError: The following %s packages are missing or different
2019-11-27 21:31:24,900 - oe-selftest - INFO - ----------------------------------------------------------------------
2019-11-27 21:31:24,900 - oe-selftest - INFO - Ran 1 test in 2.142s
2019-11-27 21:31:24,901 - oe-selftest - INFO - FAILED
2019-11-27 21:31:24,901 - oe-selftest - INFO - (failures=1)
2019-11-27 21:31:30,581 - oe-selftest - INFO - RESULTS:
2019-11-27 21:31:30,581 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (2.14s)
2019-11-27 21:31:30,590 - oe-selftest - INFO - SUMMARY:
2019-11-27 21:31:30,590 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2.144s
2019-11-27 21:31:30,591 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0)
pokybuild@debian8-ty-1:~/yocto-worker/oe-selftest-debian/build/build$ oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds -j 2
2019-11-27 21:31:49,404 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it
2019-11-27 21:31:55,365 - oe-selftest - INFO - Adding layer libraries:
2019-11-27 21:31:55,366 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib
2019-11-27 21:31:55,366 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta-yocto-bsp/lib
2019-11-27 21:31:55,367 - oe-selftest - INFO - /home/pokybuild/yocto-worker/oe-selftest-debian/build/meta-selftest/lib
2019-11-27 21:31:55,369 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable
2019-11-27 21:32:00,153 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-9396/conf/local.conf
2019-11-27 21:32:00,154 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2019-11-27 21:32:03,351 - oe-selftest - INFO - reproducible.ReproducibleTests.test_reproducible_builds (subunit.RemotedTestCase)
2019-11-27 21:32:03,352 - oe-selftest - INFO - ... ERROR
2019-11-27 21:32:03,353 - oe-selftest - INFO - 0: 1/1 1/1 (3.20s) (reproducible.ReproducibleTests.test_reproducible_builds)
2019-11-27 21:32:03,353 - oe-selftest - INFO - testtools.testresult.real._StringException: lost connection during failure report of test 'reproducible.ReproducibleTests.test_reproducible_builds'
2019-11-27 21:32:03,353 - oe-selftest - INFO - ======================================================================
2019-11-27 21:32:03,354 - oe-selftest - INFO - ERROR: reproducible.ReproducibleTests.test_reproducible_builds (subunit.RemotedTestCase)
2019-11-27 21:32:03,354 - oe-selftest - INFO - ----------------------------------------------------------------------
2019-11-27 21:32:03,354 - oe-selftest - INFO - testtools.testresult.real._StringException: lost connection during failure report of test 'reproducible.ReproducibleTests.test_reproducible_builds'
2019-11-27 21:32:03,354 - oe-selftest - INFO - ----------------------------------------------------------------------
2019-11-27 21:32:03,355 - oe-selftest - INFO - Ran 1 test in 5.496s
2019-11-27 21:32:03,355 - oe-selftest - INFO - FAILED
2019-11-27 21:32:03,355 - oe-selftest - INFO - (errors=1)
2019-11-27 21:32:09,174 - oe-selftest - INFO - RESULTS:
2019-11-27 21:32:09,175 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: ERROR (3.20s)
2019-11-27 21:32:09,184 - oe-selftest - INFO - SUMMARY:
2019-11-27 21:32:09,185 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 5.901s
2019-11-27 21:32:09,185 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=0, errors=1)
i.e. it only occurs with -j X and it only seems to happen on some hosts, here we see it on dbian8. There are probably other hosts it occurs on too like centos. It only occurs when a failure in a test occurs.
Managed to get a traceback which should help a lot:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/core/utils/concurrencytest.py", line 144, in _addResult
foo = method(test, *args, **kwargs)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/subunit/test_results.py", line 143, in addFailure
return self.super.addFailure(test, err, details=details)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/subunit/test_results.py", line 69, in addFailure
return self.decorated.addFailure(test, err, details=details)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/testtools/testresult/real.py", line 1404, in addFailure
return self.decorated.addFailure(test, err)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/subunit/__init__.py", line 677, in addFailure
self._addOutcome("failure", test, error=error, details=details)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/subunit/__init__.py", line 708, in _addOutcome
tb_content = TracebackContent(error, test)
File "/opt/poky/3.0/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/site-packages/testtools/content.py", line 204, in __init__
stack_lines = list(traceback.TracebackException(exctype, value, tb,
AttributeError: 'NoneType' object has no attribute 'TracebackException'
That worker uses our buildtools tarball which does not include python-traceback2 which subunit uses. We can just use traceback so we should patch testtools to do that. Patch on the mailing list. |