8919 2016-01-13 20:58:41 +0000 Code coverage: add ability to create coverage-based test exclusion reports 2020-02-13 16:15:06 +0000 1 1 7 Build System, Metadata & Runtime OE-Core Scripts and Tools 5.99 All Multiple RESOLVED WONTFIX OB REVIEW Medium enhancement Future 1 benjamin.esquivel unassigned humberto.ibarra.lopez joshuagloe leonardo.sandoval.gonzalez mhalstead randy.macleod ross.burton --- --- Don't know oldest_to_newest 57798 0 benjamin.esquivel 2016-01-13 20:58:41 +0000 Test exclusion based in coverage is an analysis of what (code )areas are covering the tests and be able to list what tests are entirely repetitive coverage-wise. This new ability helps the QA team to know which is the shortest list of tests that covers the maximal amount of code. 58205 1 leonardo.sandoval.gonzalez 2016-01-25 18:10:11 +0000 Assigning to Humberto, he is leading this area on the development area. 59231 2 leonardo.sandoval.gonzalez 2016-02-18 21:48:40 +0000 Lets consider the following: the value 1 indicates that the executed lines between testi and testj are the same, and value value 0 is the exactly the opposite, so a formula like this may help (too pythonic but the idea is there) res = [executed lines on testi] and [executed lines on testj] where if both tests executed the same lines, then res = [] and if both tests executed completely different lines res = [executed lines on testi] thus value = (len(res) - len([executed lines on testi])) ------------------------------------------ len([executed lines on testi]) we can construct a matrix with the values, just populating the upper right part as seem below: ---------------------------------------------- test1 test2 test3 test4 ......... testn test1 1 0.9 0 0.1 .......... 0.5 test2 1 test3 1 test4 1 . . . . testn 1 ---------------------------------------------- The problem is to identify those pair of tests with higher numbers, i.e test1 and test2 on the sample matrix have a high value, basically exercising the same code. 59350 3 leonardo.sandoval.gonzalez 2016-02-22 22:52:49 +0000 Beto, as discussed today, and to store in other place rather that the blackboard, this would be a better formula for the similarity between two sets A and B(each sets in this case correspond to the executed lines by a specific test): |(A-B)| - |(B-A)| ----------------- |A+B| if A=B 0 + 0 ----- = 0 |2A| if A != B |A| + |B| --------- = 1 |A + B| so, 0 indicates tests are identical and 1 indicates tests are not related at all The operator || indicates the length of the set. 68867 4 leonardo.sandoval.gonzalez 2016-12-06 17:22:39 +0000 This is low priority for 2.3 and there is no window time to work on this area during 2.3. 85910 5 akuster 2019-12-22 19:24:21 +0000 I think result tool does something like this. We also added fast and slow test groups. I believe this in some form has been addressed. 86428 6 randy.macleod 2020-02-13 16:15:06 +0000 A nice idea but not resource feasible.