It would be great if we could run all oe-selftest tests barring those specified as an option. Something like: oe-selftest -R distrodata We'd like this because the distrodata selftests are prone to transient failures due to networking issues. We'd like to run these separately on the autobuilder with a lower BB_NUMBER_THREADS, likely even less frequently (automated once per week). If we do so we'd also remove the distrodata.Distrodata.test_checkpkg test from the nightly-oe-selftest builder, but there's no easy way to run all tests except that one. Maintaining a list of tests we do want to run for the nightly-oe-selftest buildset is prone to bitrotting and missing out on valuable testing.
For some reason, I needed to send the patch a second time and this time it arrived to the ML http://lists.openembedded.org/pipermail/openembedded-core/2017-August/140441.html
(In reply to comment #0) > It would be great if we could run all oe-selftest tests barring those > specified as an option. Something like: > > oe-selftest -R distrodata > > We'd like this because the distrodata selftests are prone to transient > failures due to networking issues. We'd like to run these separately on the > autobuilder with a lower BB_NUMBER_THREADS, likely even less frequently > (automated once per week). > > If we do so we'd also remove the distrodata.Distrodata.test_checkpkg test > from the nightly-oe-selftest builder, but there's no easy way to run all > tests except that one. > > Maintaining a list of tests we do want to run for the nightly-oe-selftest > buildset is prone to bitrotting and missing out on valuable testing. The option is good to have it, for a quick fix you can avoid run distrodata tests moving the module to starts with _ (_distrodata), this will avoid to run the test module when request --run-all-tests. Anibal
merged at master: e40eeaa790: Leonardo Sandoval: Fri Aug 11 00:08:31 2017 +0100: context: Include a command line argument to run all except certain tests