Test exclusion based in coverage is an analysis of what (code )areas are covering the tests and be able to list what tests are entirely repetitive coverage-wise. This new ability helps the QA team to know which is the shortest list of tests that covers the maximal amount of code.
Assigning to Humberto, he is leading this area on the development area.
Lets consider the following: the value 1 indicates that the executed lines between testi and testj are the same, and value value 0 is the exactly the opposite, so a formula like this may help (too pythonic but the idea is there) res = [executed lines on testi] and [executed lines on testj] where if both tests executed the same lines, then res = [] and if both tests executed completely different lines res = [executed lines on testi] thus value = (len(res) - len([executed lines on testi])) ------------------------------------------ len([executed lines on testi]) we can construct a matrix with the values, just populating the upper right part as seem below: ---------------------------------------------- test1 test2 test3 test4 ......... testn test1 1 0.9 0 0.1 .......... 0.5 test2 1 test3 1 test4 1 . . . . testn 1 ---------------------------------------------- The problem is to identify those pair of tests with higher numbers, i.e test1 and test2 on the sample matrix have a high value, basically exercising the same code.
Beto, as discussed today, and to store in other place rather that the blackboard, this would be a better formula for the similarity between two sets A and B(each sets in this case correspond to the executed lines by a specific test): |(A-B)| - |(B-A)| ----------------- |A+B| if A=B 0 + 0 ----- = 0 |2A| if A != B |A| + |B| --------- = 1 |A + B| so, 0 indicates tests are identical and 1 indicates tests are not related at all The operator || indicates the length of the set.
This is low priority for 2.3 and there is no window time to work on this area during 2.3.
I think result tool does something like this. We also added fast and slow test groups. I believe this in some form has been addressed.
A nice idea but not resource feasible.