| Summary: | Document best practises/style guide for selftest tests and update tests | ||
|---|---|---|---|
| Product: | [QA/Testing] Runtime Testing | Reporter: | Ross Burton <ross.burton> |
| Component: | general | Assignee: | Mark Morton <mark.morton> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium | CC: | akuster, alexandru.c.georgescu, anibal.limon, jose.perez.carranza, joshuagloe, leonardo.sandoval.gonzalez, libertad.gonzalez.de.la.cruz, mariano.lopez, randy.macleod, stephano |
| Version: | 5.99 | ||
| Target Milestone: | 3.1.2 | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Ross Burton
2015-10-20 14:18:46 UTC
Assigning to Aníbal, CC Mariano. These two have rewritten OEQA and are better positioned to document how to write a test. Hopefully the updated tests are already more consistent? Aníbal/Mariano — feel free to ask for review of the docs here. As part of this we should include a list of do/don't items for selftests. One that came up recently[1] is that we really need to be careful to ensure that the tests are idempotent. We share the DL_DIR and SSTATE_CACHE across the entire autobuilder so anything that wipes files from those shared directories should be avoided. 1. http://git.yoctoproject.org/clean/cgit.cgi/poky/commit/?id=58a02e73d0a6b110d96e94033e1fdf634b42ccec Leo and myself are finishing the oe-selftest migration into the new framework, after that, we will write a document describing best practices to write tests for runtime, sdk, esdk and selftest. http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=alimon/oeqa_integration Cheers, Anibal With the new oe-qa framework about to be merged into master during M1, moving this into M2 and expected to be finished early 2.4 M2. Since the main users of the OEQA framework is the QA team, it is the best to someone of them document it. (In reply to comment #5) > Since the main users of the OEQA framework is the QA team, it is the best to > someone of them document it. Some part that need to be added is about migration to the new QA framework, - oe-selftest cases needs to execute setUpClass to create it's own builddir. The BKM/lessons learnt from the refkit team to use v2:
* moved the files to <layer>/lib/oeqa/selftest/cases/
* updated class imports with (we're not using decorators):
sed -i -e '/decorators/d; \
s/oeSelfTest/OESelftestTestCase/g; \
s/selftest\.base/selftest\.case/; \
s/selftest\.sstate/selftest\.cases\.sstate/'
* changed filenames with '-' in them to have '_' instead due to how
python unittest loader sees (or does not) the files.
* added modules using SetUpClass() to call their parent's SetUpClass()
* changed self.log to self.logger
(the sed part is refkit specific but I noticed some parts of the transition is easy to automate/script. it worked with 'find meta-refkit* -wholename '*selftest/cases/*.py'|sed ...' just OK)
As I understand this in this bug the idea is to better describe the Basic architecture information of the oeqa test. On this wiki https://wiki.yoctoproject.org/wiki/Oe-selftest. Describe how a new person can contribute to creating testcases for their new modules. Make documentation proposal on this framework that later we can decide to add it to the mega manual or some other place. We will decide upon the content. The core development of this framework under http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/core Start by reading the README then the context.py http://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html we should still do this I completed the first version of the new test-manual source and added it to the yocto-docs repo. There are a few minor updates, but the document itself is complete. |