Bug 14852

Summary: If TEST_SUITES doesn't contain a valid module, all tests are run
Product: [QA/Testing] Runtime Testing Reporter: Ross Burton <ross.burton>
Component: oeqa/runtimeAssignee: Unassigned <unassigned>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: mikko.rapeli, osama.abdelkader, randy.macleod
Version: unspecified   
Target Milestone: 5.3   
Hardware: x86   
OS: Multiple   
Whiteboard: newcomer
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2022-07-06 10:18:44 UTC 
If TEST_SUITES is empty (""), testimage aborts early.  If TEST_SUITES contains a non-existent module (eg "foobar") then testimage says so.  If TEST_SUITES contains an invalid module name (eg "foo-bar") then the discovery continues and *all* found test suites are executed.

This is very unexpected.
Comment 1 Randy MacLeod 2023-04-21 18:45:43 UTC 
Moved en masse from 4.2 to 4.3 by Randy.
Comment 2 Randy MacLeod 2025-05-01 14:07:12 UTC 
Bulk move of all unassigned 5.2 medium importance bugs to 5.3.
Comment 3 Osama Abdelkader 2025-08-06 19:12:34 UTC 
I tested this on current Poky (5.2.99+snapshot-903710ab5c4ea3c13dd1abf5accdefd27d3a5bdc) and setting TEST_SUITES = "foo-bar" now throws OEQATestNotFound instead of running all tests. The issue seems fixed.