https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/956 2020-05-19 13:46:43,487 - oe-selftest - INFO - 9: 2/19 142/406 (863.67s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_and_license) 2020-05-19 13:47:40,171 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_default (subunit.RemotedTestCase) 2020-05-19 13:47:40,171 - oe-selftest - INFO - ... ok 2020-05-19 13:47:42,344 - oe-selftest - INFO - 9: 3/19 143/406 (56.68s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_default) NOTE: Starting bitbake server... NOTE: Bitbake server didn't start within 5 seconds, waiting for 90 2020-05-19 13:51:02,062 - oe-selftest - INFO - incompatible_lic.IncompatibleLicensePerImageTests.test_bash_or_license (subunit.RemotedTestCase) 2020-05-19 13:51:02,062 - oe-selftest - INFO - ... ok 2020-05-19 13:51:02,062 - oe-selftest - INFO - 9: 4/19 144/406 (200.90s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_or_license) ERROR: Unable to start bitbake server (False) ERROR: Server log for this session (/home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-5872/bitbake-cookerdaemon.log): --- Starting bitbake server pid 48545 at 2020-05-19 13:49:24.295548 --- NOTE: Reconnecting to bitbake server... 2020-05-19 13:51:17,722 - oe-selftest - INFO - tinfoil.TinfoilTests.test_datastore_operations (subunit.RemotedTestCase) 2020-05-19 13:51:27,732 - oe-selftest - INFO - ... ERROR 2020-05-19 13:51:27,899 - oe-selftest - INFO - 10: 49/58 145/406 (278.26s) (tinfoil.TinfoilTests.test_datastore_operations) 2020-05-19 13:51:27,899 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/tinfoil.py", line 145, in test_datastore_operations tinfoil.prepare(config_only=True) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/tinfoil.py", line 391, in prepare self.server_connection, ui_module = setup_bitbake(config_params, File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/main.py", line 438, in setup_bitbake server = bb.server.process.BitBakeServer(lock, sockname, configuration, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/server/process.py", line 447, in __init__ raise SystemExit(1) SystemExit: 1 NOTE: Retrying server connection (#1)... NOTE: Bitbake server didn't start within 5 seconds, waiting for 90 ERROR: Unable to start bitbake server (False) ERROR: Server log for this session (/home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-5872/bitbake-cookerdaemon.log): --- Starting bitbake server pid 48545 at 2020-05-19 13:49:24.295548 --- NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection (#1)... NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection (#2)... NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection (#2)... NOTE: Starting bitbake server... 2020-05-19 13:52:33,890 - oe-selftest - INFO - tinfoil.TinfoilTests.test_expand (subunit.RemotedTestCase) 2020-05-19 13:52:33,890 - oe-selftest - INFO - ... ok 2020-05-19 13:52:33,890 - oe-selftest - INFO - 10: 50/58 146/406 (76.26s) (tinfoil.TinfoilTests.test_expand) NOTE: Starting bitbake server... 2020-05-19 13:52:36,057 - oe-selftest - INFO - tinfoil.TinfoilTests.test_getvar (subunit.RemotedTestCase) NOTE: Reconnecting to bitbake server... 2020-05-19 13:52:36,673 - oe-selftest - INFO - ... ok NOTE: Previous bitbake instance shutting down?, waiting to retry... 2020-05-19 13:52:36,673 - oe-selftest - INFO - 10: 51/58 147/406 (2.17s) (tinfoil.TinfoilTests.test_getvar) NOTE: Retrying server connection (#1)... NOTE: Reconnecting to bitbake server... NOTE: Previous bitbake instance shutting down?, waiting to retry... NOTE: Retrying server connection (#1)... NOTE: Starting bitbake server... 2020-05-19 13:52:39,002 - oe-selftest - INFO - tinfoil.TinfoilTests.test_getvar_bb_origenv (subunit.RemotedTestCase) 2020-05-19 13:52:39,003 - oe-selftest - INFO - ... ok 2020-05-19 13:52:39,003 - oe-selftest - INFO - 10: 52/58 148/406 (2.95s) (tinfoil.TinfoilTests.test_getvar_bb_origenv) 2020-05-19 13:53:45,296 - oe-selftest - INFO - tinfoil.TinfoilTests.test_list_recipes (subunit.RemotedTestCase) 2020-05-19 13:53:45,297 - oe-selftest - INFO - ... ok 2020-05-19 13:53:45,297 - oe-selftest - INFO - 10: 53/58 149/406 (66.29s) (tinfoil.TinfoilTests.test_list_recipes) 2020-05-19 13:53:52,226 - oe-selftest - INFO - tinfoil.TinfoilTests.test_parse_recipe (subunit.RemotedTestCase) 2020-05-19 13:53:52,226 - oe-selftest - INFO - ... ok 2020-05-19 13:53:52,227 - oe-selftest - INFO - 10: 54/58 150/406 (6.93s) (tinfoil.TinfoilTests.test_parse_recipe) 2020-05-19 13:53:54,744 - oe-selftest - INFO - tinfoil.TinfoilTests.test_parse_recipe_copy_expand (subunit.RemotedTestCase) NOTE: Reconnecting to bitbake server... 2020-05-19 13:53:55,379 - oe-selftest - INFO - ... ok 2020-05-19 13:53:55,379 - oe-selftest - INFO - 10: 55/58 151/406 (2.52s) (tinfoil.TinfoilTests.test_parse_recipe_copy_expand) NOTE: Retrying server connection (#1)... NOTE: Reconnecting to bitbake server...
May be related to #13604
We saw this here: https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1055 but I think I now understand how we might be able to reproduce it. I think this only fails when an oe-selftest fails. In this build on the autobuilder you will note this test failed in other builds: https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1059 https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1055 so i think if we have a failing test which we execute in a loop, we'd probably see this SystemExit issue occasionally. That suggests its something to do with races in the exit path when failures occur. All this is a guess but probably worth exploring.
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1077/steps/8/logs/step2d
May be related to 13978 according to Richard Purdie. Any news Kia?
I think/hope this may be related to http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=ab8ebfd35a7bbfec3dad612c824b29ca93dbe196
We've not seen this issue, resolving with the above change, will reopen if it recurs.