2019-10-19 10:28:12,096 - oe-selftest - INFO - ... ERROR 2019-10-19 10:28:12,352 - oe-selftest - INFO - 2: 29/35 185/366 (178.56s) (distrodata.Distrodata.test_maintainers) 2019-10-19 10:28:12,352 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 61, in test_maintainers tinfoil.prepare(config_only=False) File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/tinfoil.py", line 387, in prepare extrafeatures) File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/main.py", line 438, in setup_bitbake server = bb.server.process.BitBakeServer(lock, sockname, configuration, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/server/process.py", line 447, in __init__ raise SystemExit(1) SystemExit: 1
I talked a bit with Paul about this, it appears that something is going wrong with tinfoil's early exception handling when it can't setup/connect to a server. We likely need to fix the logging which would then let us get to the real underlying failure.
Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/538 2019-12-07 23:57:13,529 - oe-selftest - INFO - 2: 29/40 145/384 (138.08s) (distrodata.Distrodata.test_maintainers) 2019-12-07 23:57:13,529 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 61, in test_maintainers tinfoil.prepare(config_only=False) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/tinfoil.py", line 387, in prepare extrafeatures) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/main.py", line 438, in setup_bitbake server = bb.server.process.BitBakeServer(lock, sockname, configuration, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/server/process.py", line 447, in __init__ raise SystemExit(1) SystemExit: 1 2019-12-07 23:57:13,530 - oe-selftest - INFO - runqemu.RunqemuTests.test_boot_deploy (subunit.RemotedTestCase) 2019-12-07 23:57:13,549 - oe-selftest - INFO - ... ERROR 2019-12-07 23:57:13,549 - oe-selftest - INFO - 0: 22/33 146/384 (1983.48s) (runqemu.RunqemuTests.test_boot_deploy) 2019-12-07 23:57:13,549 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 93, in test_boot_deploy with runqemu(self.recipe, ssh=False, launch_cmd=cmd) as qemu: File "/opt/poky/3.0.1/sysroots/x86_64-pokysdk-linux/usr/lib/python3.7/contextlib.py", line 112, in __enter__ return next(self.gen) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/utils/commands.py", line 314, in runqemu tinfoil.prepare(config_only=False, quiet=True) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/tinfoil.py", line 387, in prepare extrafeatures) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/main.py", line 438, in setup_bitbake server = bb.server.process.BitBakeServer(lock, sockname, configuration, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/server/process.py", line 447, in __init__ raise SystemExit(1) SystemExit: 1
I believe I spotted a pattern. This appears to only occur when a test is failing. I also wonder if there is a link to tests which "fork" since maintainers and reproducibility generate a lot of other processes. Next steps may be to stress test with failing tests.
Kai, we'd like this done in the next week or two. Okay?
I think the sys.exit(1) is related from the other bug and likely causing this. *** This bug has been marked as a duplicate of bug 13664 ***
Sadly https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/723 proves this was not caused by the other bug, reopening.
Looks like this issue happens in different tests: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/751
Pattern in the bug seems to always be on selftest-centos...
Kai is having trouble reproducing this bug. Try testing under high load. If anyone has another suggetion, please let us know. It's one of the most commonly occurring non-reproducible bugs that the autobuilder sees.
Not seen in the autobuilder for a few days but it will almost certainly come back. Add debugging and send a patch that can be tested in master-next, please.
Still no sign of this bug in the autobuilder but we'd like to get it fixed ASAP.
I think/hope this may be related to http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=ab8ebfd35a7bbfec3dad612c824b29ca93dbe196
We've not seen this issue, resolving with the above change, will reopen if it recurs.