| Summary: | Ptest: add binary PASS/FAIL scheme for use during build. | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Armin Kuster <akuster> |
| Component: | core | Assignee: | Armin Kuster <akuster> |
| Status: | CLOSED NOTABUG | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium | CC: | meta.mr.watcher, meta.watcher |
| Version: | 5.99 | ||
| Target Milestone: | 2.7 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
I could not find a bug for addressing this idea. Bug 12654 is for the overall QA process not just ptest. re-purposing this for comparing one ptest run against another to trigger PASS/FAIL in the AB. seems like a dup. . |
To help trigger changes in the AB, maybe the concept of expected pass/fail can be implemented at the recipe level. example: If we know a recipe that has ptest support has passed in the past, we can set a flag "PTEST_expected = "PASS" and if that changes during a AB ptest step, we fail the build so swat and friends can act upon it. if we know a recipe has failures maybe "PTEST_expected_{machine} = "FAIL" or something crazy the includes a # 4 out of 22 test should fail. just a thought...