Running oe-selftest with "--machine=all" can fail. Usually for me it fails on the second machine. For simplicity let's just consider the case of running a single test.

E.g.: oe-selftest -r devtool.DevtoolAddTests.test_devtool_add_binary --machine all -v

setup_builddir overwrites BUILDDIR env variable with the new suffixed build dir, build-st. But it doesn't revert it after the test has been run. So each time it is called (for each machine), it re-suffixes "-st" onto the build dir. You'll see build-st, build-st-st, build-st-st-st, etc. directories get created.

The reason for the actual failure is in the code that sets up the meta-selftest layer. The first time around, copytree is performed on: 

poky/meta-selftest => poky/build-st/meta-selftest 

And the git repo is initialized. The second time around, BUILDDIR points to 'build-st', so copytree is run as:

poky/build-st/meta-selftest => poky/build-st/build-st/meta-selftest

The git repo setup is attempted again. The 'git init' succeeds, but the 'git commit' can fail with a '1' return code if no changes are present. Usually, there won't be any changes because we are operating on a git repo that was already setup. But sometimes, new .pyc files can be present which get caught in the 'git add -u'. These will make the 'git commit' succeed.

In the run below, I instrumented selftest/context.py to add some additional 'info' messages showing how copytree is called. You'll notice this got through 3 machines before failing.


chris@chris-virtual-machine:~/repos/poky$ oe-selftest -r devtool.DevtoolAddTests.test_devtool_add_binary --machine all -v
2023-10-19 21:20:38,702 - oe-selftest - INFO - Custom machine mode enabled. MACHINE set to all
2023-10-19 21:20:38,726 - oe-selftest - INFO - Changing cwd to /home/chris/repos/poky/build
2023-10-19 21:20:38,727 - oe-selftest - INFO - Adding layer libraries:
2023-10-19 21:20:38,727 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-poky/lib
2023-10-19 21:20:38,727 - oe-selftest - INFO - 	/home/chris/repos/poky/meta/lib
2023-10-19 21:20:38,727 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-yocto-bsp/lib
2023-10-19 21:20:38,727 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-selftest/lib
2023-10-19 21:20:38,727 - oe-selftest - INFO - Checking base configuration is valid/parsable
NOTE: Starting bitbake server...
2023-10-19 21:20:39,264 - oe-selftest - INFO - Run tests with custom MACHINE set to: qemuppc
2023-10-19 21:20:39,275 - oe-selftest - INFO - copytree: /home/chris/repos/poky/meta-selftest => /home/chris/repos/poky/build-st/meta-selftest
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
2023-10-19 21:20:39,801 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/chris/repos/poky/build-st/conf/local.conf
2023-10-19 21:20:39,801 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2023-10-19 21:20:41,237 - oe-selftest - INFO - test_devtool_add_binary (devtool.DevtoolAddTests.test_devtool_add_binary)
2023-10-19 21:20:41,240 - oe-selftest - DEBUG - Writing to: /home/chris/repos/poky/build-st/conf/machine.inc
MACHINE ??= "qemuppc"


2023-10-19 21:20:41,240 - oe-selftest - DEBUG - Appending to: /home/chris/repos/poky/build-st/conf/selftest.inc
SSTATE_DIR = "/home/chris/repos/poky/build-st/sstate_devtool"
SSTATE_MIRRORS += "file://.* file:////home/chris/repos/poky/build-st/sstate-cache/PATH"
BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"


2023-10-19 21:20:41,241 - oe-selftest - DEBUG - Adding path '/tmp/devtoolqaf9s9rbjc' to be cleaned up when test is over
2023-10-19 21:20:41,249 - oe-selftest - DEBUG - Adding path '/home/chris/repos/poky/build-st/workspace' to be cleaned up when test is over
2023-10-19 21:20:41,249 - oe-selftest - DEBUG - Adding command 'bitbake -c cleansstate tst-bin' to tearDown for this test.
2023-10-19 21:20:41,249 - oe-selftest - DEBUG - Adding command 'bitbake-layers remove-layer */workspace' to tearDown for this test.
2023-10-19 21:36:13,824 - oe-selftest - INFO -  ... ok
2023-10-19 21:36:13,824 - oe-selftest - DEBUG - Deleting devtool sstate cache on /home/chris/repos/poky/build-st/sstate_devtool
2023-10-19 21:36:13,956 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-10-19 21:36:13,956 - oe-selftest - INFO - Ran 1 test in 934.691s
2023-10-19 21:36:13,956 - oe-selftest - INFO - OK
2023-10-19 21:36:17,547 - oe-selftest - INFO - RESULTS:
2023-10-19 21:36:17,547 - oe-selftest - INFO - RESULTS - devtool.DevtoolAddTests.test_devtool_add_binary: PASSED (932.59s)
2023-10-19 21:36:17,550 - oe-selftest - INFO - SUMMARY:
2023-10-19 21:36:17,550 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 934.692s
2023-10-19 21:36:17,550 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
2023-10-19 21:36:17,555 - oe-selftest - INFO - Changing cwd to /home/chris/repos/poky/build-st
2023-10-19 21:36:17,556 - oe-selftest - INFO - Adding layer libraries:
2023-10-19 21:36:17,556 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-poky/lib
2023-10-19 21:36:17,556 - oe-selftest - INFO - 	/home/chris/repos/poky/meta/lib
2023-10-19 21:36:17,556 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-yocto-bsp/lib
2023-10-19 21:36:17,556 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-selftest/lib
2023-10-19 21:36:17,557 - oe-selftest - INFO - Checking base configuration is valid/parsable
NOTE: Starting bitbake server...
2023-10-19 21:36:18,227 - oe-selftest - INFO - Run tests with custom MACHINE set to: qemuarmv5
2023-10-19 21:36:18,244 - oe-selftest - INFO - copytree: /home/chris/repos/poky/build-st/meta-selftest => /home/chris/repos/poky/build-st-st/meta-selftest
2023-10-19 21:36:20,260 - oe-selftest - INFO - test_devtool_add_binary (devtool.DevtoolAddTests.test_devtool_add_binary)
2023-10-19 21:36:20,263 - oe-selftest - DEBUG - Writing to: /home/chris/repos/poky/build-st-st/conf/machine.inc
MACHINE ??= "qemuarmv5"


2023-10-19 21:36:20,263 - oe-selftest - DEBUG - Appending to: /home/chris/repos/poky/build-st-st/conf/selftest.inc
SSTATE_DIR = "/home/chris/repos/poky/build-st-st/sstate_devtool"
SSTATE_MIRRORS += "file://.* file:////home/chris/repos/poky/build-st-st/sstate-cache/PATH"
BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"


2023-10-19 21:36:20,264 - oe-selftest - DEBUG - Adding path '/tmp/devtoolqa2jkz7kgw' to be cleaned up when test is over
2023-10-19 21:36:20,274 - oe-selftest - DEBUG - Adding path '/home/chris/repos/poky/build-st-st/workspace' to be cleaned up when test is over
2023-10-19 21:36:20,274 - oe-selftest - DEBUG - Adding command 'bitbake -c cleansstate tst-bin' to tearDown for this test.
2023-10-19 21:36:20,274 - oe-selftest - DEBUG - Adding command 'bitbake-layers remove-layer */workspace' to tearDown for this test.
2023-10-19 21:51:45,995 - oe-selftest - INFO -  ... ok
2023-10-19 21:51:45,996 - oe-selftest - DEBUG - Deleting devtool sstate cache on /home/chris/repos/poky/build-st-st/sstate_devtool
2023-10-19 21:51:46,124 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-10-19 21:51:46,124 - oe-selftest - INFO - Ran 1 test in 927.896s
2023-10-19 21:51:46,124 - oe-selftest - INFO - OK
2023-10-19 21:51:49,572 - oe-selftest - INFO - RESULTS:
2023-10-19 21:51:49,572 - oe-selftest - INFO - RESULTS - devtool.DevtoolAddTests.test_devtool_add_binary: PASSED (925.74s)
2023-10-19 21:51:49,575 - oe-selftest - INFO - SUMMARY:
2023-10-19 21:51:49,575 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 927.897s
2023-10-19 21:51:49,575 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
2023-10-19 21:51:49,579 - oe-selftest - INFO - Changing cwd to /home/chris/repos/poky/build-st-st
2023-10-19 21:51:49,579 - oe-selftest - INFO - Adding layer libraries:
2023-10-19 21:51:49,579 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-poky/lib
2023-10-19 21:51:49,579 - oe-selftest - INFO - 	/home/chris/repos/poky/meta/lib
2023-10-19 21:51:49,579 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-yocto-bsp/lib
2023-10-19 21:51:49,579 - oe-selftest - INFO - 	/home/chris/repos/poky/meta-selftest/lib
2023-10-19 21:51:49,580 - oe-selftest - INFO - Checking base configuration is valid/parsable
NOTE: Starting bitbake server...
2023-10-19 21:51:50,132 - oe-selftest - INFO - Run tests with custom MACHINE set to: qemux86
2023-10-19 21:51:50,149 - oe-selftest - INFO - copytree: /home/chris/repos/poky/build-st-st/meta-selftest => /home/chris/repos/poky/build-st-st-st/meta-selftest
Traceback (most recent call last):
  File "/home/chris/repos/poky/scripts/oe-selftest", line 65, in <module>
    ret = main()
          ^^^^^^
  File "/home/chris/repos/poky/scripts/oe-selftest", line 49, in main
    results = args.func(logger, args)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/selftest/context.py", line 434, in run
    results.append(self._internal_run(logger, args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/selftest/context.py", line 411, in _internal_run
    rc = self.tc.runTests(**self.tc_kwargs['run'])
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/selftest/context.py", line 177, in runTests
    return super(OESelftestTestContext, self).runTests(processes, skips)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/core/context.py", line 91, in runTests
    result = self.runner.run(self.prepareSuite(self.suites, processes))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/.rye/py/cpython@3.12.0/install/lib/python3.12/unittest/runner.py", line 240, in run
    test(result)
  File "/home/chris/.rye/py/cpython@3.12.0/install/lib/python3.12/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/selftest/context.py", line 47, in run
    (builddir, newbuilddir) = self.setupfunc("-st", None, self.suite)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/repos/poky/meta/lib/oeqa/selftest/context.py", line 113, in setup_builddir
    subprocess.check_output("git init && git add * && git commit -a -m 'initial'", cwd=newselftestdir, shell=True)
  File "/home/chris/.rye/py/cpython@3.12.0/install/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chris/.rye/py/cpython@3.12.0/install/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'git init && git add * && git commit -a -m 'initial'' returned non-zero exit status 1.