Automake in oe-core is patched with http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/automake/automake/buildtest.patch where we basically split "make check" into: make buildtest-TESTS make runtest-TESTS We do this in order to build the tests separately (cross-compile them) but run them later on the target as part of the "ptest" package. There are two problems with the existing patch: 1. The patch only creates the targets buildtest-TESTS, runtest-TESTS for "serial-test-harness" (https://www.gnu.org/software/automake/manual/html_node/Serial-Test-Harness.html) . This harness is deprecated, and utils default to "parallel-tests". So in order to be able to use buildtest-TESTS target we currently need to patch configure.ac to explicitly select "serial-tests", something like: AM_INIT_AUTOMAKE(... [serial-tests] ...) 2. The created targets (builtest-TESTS, runtest-TESTS) are not recursive, in spite of what the patch suggests. This may have worked at some time, but it does not seem to work now. We need to patch configure.ac with an additional patch, something like: AM_EXTRA_RECURSIVE_TARGETS([buildtest-TESTS]) otherwise not all tests may end up being cross-compiled.
Bulk move from 4.99 or 0.00 to 5.99
Closing this, the design of the test harness in automake isn't broken and isn't the problem. Our horrible ways of building and extracting the tests is the problem and is often broken, but this is all worked around in each recipe. We need to improve how we build and install tests but that's sort of a larger problem.