Bug 14852 - If TEST_SUITES doesn't contain a valid module, all tests are run
Summary: If TEST_SUITES doesn't contain a valid module, all tests are run
Status: RESOLVED FIXED
Alias: None
Product: Runtime Testing
Classification: QA/Testing
Component: oeqa/runtime (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 5.3
Assignee: Unassigned
QA Contact:
URL:
Whiteboard: newcomer
Depends on:
Blocks:
 
Reported: 2022-07-06 10:18 UTC by Ross Burton
Modified: 2025-08-07 21:09 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Burton 2022-07-06 10:18:44 UTC 
If TEST_SUITES is empty (""), testimage aborts early.  If TEST_SUITES contains a non-existent module (eg "foobar") then testimage says so.  If TEST_SUITES contains an invalid module name (eg "foo-bar") then the discovery continues and *all* found test suites are executed.

This is very unexpected.
Comment 1 Randy MacLeod 2023-04-21 18:45:43 UTC 
Moved en masse from 4.2 to 4.3 by Randy.
Comment 2 Randy MacLeod 2025-05-01 14:07:12 UTC 
Bulk move of all unassigned 5.2 medium importance bugs to 5.3.
Comment 3 Osama Abdelkader 2025-08-06 19:12:34 UTC 
I tested this on current Poky (5.2.99+snapshot-903710ab5c4ea3c13dd1abf5accdefd27d3a5bdc) and setting TEST_SUITES = "foo-bar" now throws OEQATestNotFound instead of running all tests. The issue seems fixed.