Bug 9921 - oe-selftest naming test guidelines
Summary: oe-selftest naming test guidelines
Status: RESOLVED FIXED
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 5.99
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: Future
Assignee: Mark Morton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-07-12 19:55 UTC by Benjamin Esquivel
Modified: 2020-06-19 13:08 UTC (History)
11 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Benjamin Esquivel 2016-07-12 19:55:36 UTC
Write a set of guides for naming tests for mainly 2 reasons: to facilitate a filtering feature at test listing & execution and to have a more consistent test list.

Currently, filtering would be highly ineffective due to the repetition of the fields in the test name. i.e. "wic.Wic.test_wic_image_type" this allows no differentiation of module/submodule. Another example: recipetool.RecipetoolAppendsrcTests.test_recipetool_appendsrcfile_srcdir_basic

This guide should be written at the QA wiki and should also trigger a modification of the existing tests.

Test naming guide proposal for oe-selftest:

Test name composition: <Module>.<submodule>.<testname> (as currently implemented).

  - Module: the python library where the classes/tests live. Name it after the component you are applying tests to: i.e. bitbake, wic, sstates, etc. 

  - Submodule: a division within the library (usually the class)that tests a specific part of the module. i.e. from bitbake, test memres.

  - test name: the function that implements the test, named after the feature that it tests. several tests can exist for the same feature (negative, boundary, etc) so it can be name<consecutive> like name01, name02.
Comment 1 Jose Perez C 2017-08-30 15:48:44 UTC
Moved to 2.5 due resources constraint
Comment 2 Armin Kuster 2019-12-22 19:42:48 UTC
not sure if this is done but we should doc this.
Comment 3 Randy MacLeod 2020-01-30 16:20:44 UTC
Mark, for the test manual according to Richard.
Comment 4 Mark Morton 2020-06-19 13:08:53 UTC
This is now complete and added to the yocto-docs repository.