To help trigger changes in the AB, maybe the concept of expected pass/fail can be implemented at the recipe level. example: If we know a recipe that has ptest support has passed in the past, we can set a flag "PTEST_expected = "PASS" and if that changes during a AB ptest step, we fail the build so swat and friends can act upon it. if we know a recipe has failures maybe "PTEST_expected_{machine} = "FAIL" or something crazy the includes a # 4 out of 22 test should fail. just a thought...
I could not find a bug for addressing this idea. Bug 12654 is for the overall QA process not just ptest.
re-purposing this for comparing one ptest run against another to trigger PASS/FAIL in the AB.
seems like a dup.
.