Bug 10887

Looking at the manual there is no specific section about how the project is actually tested. Some pieces are documented but there is no summary. I therefore propose:

"""
Yotco Project Testing and QA strategy

One of the key benefits the Yocto Project brings is that as well as having build tools, we add in testing strategies around this so that as well as being able to build customised Linux, you're also able to test and validate it. This QA/testing infrastructure is in fact woven into the project to the point its core developers take some of it for granted. Roughly speaking there are the following pieces:

bitbake-selftest - A standalone command which runs unittesting on key pieces of bitbake and the bitbake fetchers.

sanity.bbclass - This class is automatically included by default and checks the users environment for missing tools (e.g. gcc) or common misconfigurations (e.g. MACHINE set incorrectly)

insane.bbclass - This class checks the generated output from builds for sanity. For example, if building for an ARM target, were ARM binaries generated? If PPC ones were, there is a problem!

testimage.bbclass - Performs runtime testing of images after they're built. Usually used with qemu to boot the images and check the combined runtime result boots and functions but can also take the IP address of a machine to test.

ptest - These are packages which package up the tests of a given piece of software, allowing them to be run within a target image.

oe-selftest - Used to test cases where combinations of bitbake invocations need to be tested so the tests need to be outside of the buildsystem itself. Runs all tests by default or specific tests or test-suites can be selected.

Originally, much of this testing was done manually but significant effort has been made in automating the tests so that more people can use them and we can run them faster and more efficiently.

The project's main autobuilder (autobuilder.yoctoproject.org) publically tests the code in OE-Core, Poky, and BitBake both for the current state of master and also testing submitted patches, usually in batches in ross/mut (master-under-test) or master-next branches. This ensures in a publicly visible way that all of the main supposed architectures and recipes in OE-Core build and function. Various features such as multilib, sub architectures like x32, poky-tiny, musl, no-x11 and many more are tested as part of this, along with bitbake-selftest and oe-selftest. It does take the autobuilder workers several hours to validate any given code revision. The autobuilder workers are non-homogeneous meaning we get regular testing across a variety of distros, the autobuilder is limited to only testing qemu based setups and not real hardware.

In addition to the autobuilder's tests, the QA team also performs testing on a variety of platforms, including some real hardware ones to ensure that things work as expected there.
"""

and we can then link to the various pieces we do have documentation for. There may also be further documentation needed if this highlights some gaps which can go into separate bugs.