Bug 9921

Summary: oe-selftest naming test guidelines
Product: [QA/Testing] Build Testing Reporter: Benjamin Esquivel <benjamin.esquivel>
Component: generalAssignee: Mark Morton <mark.morton>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: akuster, benjamin.esquivel, bluelightning, francisco.j.pedraza.gonzalez, humberto.ibarra.lopez, jose.perez.carranza, joshuagloe, juan.m.cruz.alcaraz, libertad.gonzalez.de.la.cruz, randy.macleod, sgw
Version: 5.99   
Target Milestone: Future   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Benjamin Esquivel 2016-07-12 19:55:36 UTC
Write a set of guides for naming tests for mainly 2 reasons: to facilitate a filtering feature at test listing & execution and to have a more consistent test list.

Currently, filtering would be highly ineffective due to the repetition of the fields in the test name. i.e. "wic.Wic.test_wic_image_type" this allows no differentiation of module/submodule. Another example: recipetool.RecipetoolAppendsrcTests.test_recipetool_appendsrcfile_srcdir_basic

This guide should be written at the QA wiki and should also trigger a modification of the existing tests.

Test naming guide proposal for oe-selftest:

Test name composition: <Module>.<submodule>.<testname> (as currently implemented).

  - Module: the python library where the classes/tests live. Name it after the component you are applying tests to: i.e. bitbake, wic, sstates, etc. 

  - Submodule: a division within the library (usually the class)that tests a specific part of the module. i.e. from bitbake, test memres.

  - test name: the function that implements the test, named after the feature that it tests. several tests can exist for the same feature (negative, boundary, etc) so it can be name<consecutive> like name01, name02.
Comment 1 Jose Perez C 2017-08-30 15:48:44 UTC
Moved to 2.5 due resources constraint
Comment 2 Armin Kuster 2019-12-22 19:42:48 UTC
not sure if this is done but we should doc this.
Comment 3 Randy MacLeod 2020-01-30 16:20:44 UTC
Mark, for the test manual according to Richard.
Comment 4 Mark Morton 2020-06-19 13:08:53 UTC
This is now complete and added to the yocto-docs repository.