10050 2016-07-29 07:01:59 +0000 oeqa runner generate wrong test results in log file when running dynamic test cases 2020-04-09 15:28:27 +0000 1 1 10 QA/Testing Runtime Testing general 2.4 x86 Multiple RESOLVED OBSOLETE Medium enhancement Future 1 daweix.wu unassigned jose.perez.carranza joshuagloe patrick.ohly randy.macleod sgw --- --- Don't know oldest_to_newest 64633 0 3333 daweix.wu 2016-07-29 07:01:59 +0000 Created attachment 3333 Test case Details: The results is correct printed on screen but not correct in log file Step: Use yocto oeqa runner to run the attach test cases *Expected Results:* See pass/fail/skip/error 4 results in log file *Actual Results:* All passed in log file 64634 1 daweix.wu 2016-07-29 07:08:26 +0000 From Zhang,Junhui: there is a workaround in line 19 , if comment it out , this issue is not reproduced. You can see more explanation there. 64651 2 benjamin.esquivel 2016-07-29 15:00:00 +0000 Mariano, can you replicate this test? 64682 3 mariano.lopez 2016-07-29 21:17:04 +0000 In the test cases attached it creates dynamically the class and the for every new class it creates a TextTestResult, so this can lead to the behaviour observed. OE doesn't use/require dynamic generated test. 64777 4 daweix.wu 2016-08-02 08:07:20 +0000 For us , we need this function to generate logs for this case: One oe runner static *Total* case includes many real cases , so to the runner, it just only generate one result for those many real cases in the log file, which doesn't reflect the real results. 76290 5 jair.de.jesus.gonzalez.plascencia 2017-08-29 21:53:19 +0000 Moving to 2.5, due to limited resources. 76295 6 jair.de.jesus.gonzalez.plascencia 2017-08-29 22:03:10 +0000 Returning version to 2.4. 78583 7 joshuagloe 2017-12-07 15:12:54 +0000 Much of the oeqa code has been rewritten since this issue was filed, need to test whether the issue is still present. 86970 8 randy.macleod 2020-04-09 15:28:27 +0000 If this problem shows up again, reopen. 3333 2016-07-29 07:01:59 +0000 2017-08-30 07:33:58 +0000 Test case case_interface.py text/plain 2584 daweix.wu IiIiClRoaXMgbW9kdWxlIHByb3ZpZGUgc29tZSBBUEkgZm9yIGhlbHAgcnVuIHNvbWUgbm90IHB5 LXVuaXQgYmFzZWQgdGVzdGNhc2VzLgoiIiIKaW1wb3J0IHN5cwppbXBvcnQgdW5pdHRlc3QKZnJv bSBvZXFhLm9ldGVzdCBpbXBvcnQgb2VSdW50aW1lVGVzdAoKY2xhc3MgRXJyb3JFeGNlcHRpb24o RXhjZXB0aW9uKToKICAgIHBhc3MKCmRlZiBnZW5UZXN0T2JqKGNsYXNzbmFtZSwgdGVzdG5hbWUs IGZ1bmM9bGFtYmRhIHNlbGY6Tm9uZSk6CiAgICBjbHMgPSB0eXBlKGNsYXNzbmFtZSwgKG9lUnVu dGltZVRlc3QsKSwge3Rlc3RuYW1lOiBmdW5jfSkKICAgIHJldCA9IGNscyh0ZXN0bmFtZSkKICAg ICMgYSB3YWxrIGFyb3VuZCBtZXRob2QgaGVyZTogYmVjYXVzZSBzb21lIGJ1Z3MgaW4gCiAgICAj IG9lcWEudXRpbHMuZGVjb3JhdG9ycy5Mb2dSZXN1bHRzLmdldF9jbGFzc190aGF0X2RlZmluZWRf bWV0aG9kCiAgICAjIEJlY2F1c2UgdGhpcyBjYXNlIGludGVyZmFjZSB3aWxsIGdlbmVyYXRlIHRl c3QgY2FzZXMgZHlhbWljbGx5LAogICAgIyBTbyB3ZSBjYW4ndCBydW4gaW50byBlbHNlIGJyYW5j aCBpbiB0aGF0IGZ1bmN0aW9ucy4KICAgICMgVG8gcHJldmVudCB0aGF0LCBJdCBuZWVkIGNoYW5n IHRoaXMgYXR0ciBvZiBjbHMgdG8gbWV0aG9kCiAgICAjc2V0YXR0cihjbHMsIHRlc3RuYW1lLCBn ZXRhdHRyKHJldCwgdGVzdG5hbWUpKQogICAgcmV0dXJuIHJldAoKY2xhc3MgVGVzdENhc2VJbnRl cmZhY2Uob2VSdW50aW1lVGVzdCk6CiAgICAiIiIKICAgIHVzYWdlOgogICAgZm9yIGV4YW1wbGU6 CiAgICBjbGFzcyBNeVVwc3RyZWFtVGVzdENhc2UoVGVzdENhc2VJbnRlcmZhY2UpOgogICAgICAg ICMgdGhpcyBjYXNlKHRlc3RVcHN0cmVhbSkgd2Fzbid0IHNob3dlZCBpbiByZXN1bHQKICAgICAg ICBkZWYgdGVzdFVwc3RyZWFtKHNlbGYpOgogICAgICAgICAgICAjcnVuIGFuZCBwYXJzZSBzb21l IGNhc2VzCiAgICAgICAgICAgICN1c2Ugc2VsZi5hZGRTdWNjZXNzIHRvIGFkZCBzdWNjZXNzIGNh c2UKICAgICAgICAgICAgIy4uLgogICAgIiIiCiAgICBkZWYgcnVuKHNlbGYsIHJlc3VsdCwgKmFy Z3MsICoqa3dhcmdzKToKICAgICAgICBzZWxmLnJlc3VsdCA9IHJlc3VsdAogICAgICAgIG9yaSA9 IHVuaXR0ZXN0LlRlc3RDYXNlLnJ1bgogICAgICAgIF9yZXN1bHQgPSB1bml0dGVzdC5UZXh0VGVz dFJlc3VsdCh1bml0dGVzdC5ydW5uZXIuX1dyaXRlbG5EZWNvcmF0b3Ioc3lzLnN0ZGVyciksIFRy dWUsIDEpCiAgICAgICAgb3JpKHNlbGYsIF9yZXN1bHQsICphcmdzLCAqKmt3YXJncykKICAgICAg ICBfcmVzdWx0LnN0b3BUZXN0UnVuKCkKICAgICAgICBfcmVzdWx0LnByaW50RXJyb3JzKCkKCiAg ICBkZWYgYWRkQ2FzZShzZWxmLCBmdW5jLCBjYXNlbmFtZSwgY2xhc3NuYW1lPU5vbmUsIHN0ZG91 dD0iIiwgc3RkZXJyPSIiKToKICAgICAgICBjYXNlbmFtZSA9IGNhc2VuYW1lLnN0cmlwKCkKICAg ICAgICBjbGFzc25hbWUgPSBjbGFzc25hbWUgaWYgY2xhc3NuYW1lIGVsc2UgY2FzZW5hbWUKICAg ICAgICBkZWYgdGVzdEZha2Uoc2VsZik6CiAgICAgICAgICAgIHN5cy5zdGRvdXQud3JpdGUoc3Rk b3V0KQogICAgICAgICAgICBzeXMuc3RkZXJyLndyaXRlKHN0ZGVycikKICAgICAgICAgICAgZnVu YygpCiAgICAgICAgdGVzdEZha2UuX19uYW1lX18gPSBjYXNlbmFtZQogICAgICAgIG9iaiA9IGdl blRlc3RPYmooY2xhc3NuYW1lLCBjYXNlbmFtZSwgdGVzdEZha2UpCiAgICAgICAgb2JqLnJ1bihz ZWxmLnJlc3VsdCkKCiAgICBkZWYgYWRkU3VjY2VzcyhzZWxmLCAqYXJncywgKiprd2FyZ3MpOgog ICAgICAgIGRlZiBzdWNjZXNzKCk6CiAgICAgICAgICAgIHBhc3MKICAgICAgICByZXR1cm4gc2Vs Zi5hZGRDYXNlKHN1Y2Nlc3MsICphcmdzLCAqKmt3YXJncykKCiAgICBkZWYgYWRkRXJyb3Ioc2Vs ZiwgKmFyZ3MsICoqa3dhcmdzKToKICAgICAgICBkZWYgZXJyb3IoKToKICAgICAgICAgICAgcmFp c2UgRXJyb3JFeGNlcHRpb24oKQogICAgICAgIHJldHVybiBzZWxmLmFkZENhc2UoZXJyb3IsICph cmdzLCAqKmt3YXJncykKCiAgICBkZWYgYWRkRmFpbHVyZShzZWxmLCAqYXJncywgKiprd2FyZ3Mp OgogICAgICAgIGRlZiBmYWlsdXJlKCk6CiAgICAgICAgICAgIGFzc2VydCBGYWxzZQogICAgICAg IHJldHVybiBzZWxmLmFkZENhc2UoZmFpbHVyZSwgKmFyZ3MsICoqa3dhcmdzKQoKICAgIGRlZiBh ZGRTa2lwKHNlbGYsICphcmdzLCAqKmt3YXJncyk6CiAgICAgICAgZGVmIHNraXAoKToKICAgICAg ICAgICAgc2VsZi5za2lwVGVzdCgiIikKICAgICAgICByZXR1cm4gc2VsZi5hZGRDYXNlKHNraXAs ICphcmdzLCAqKmt3YXJncykKCiAgICBkZWYgdGVzdEludGVyZmFjZShzZWxmKToKICAgICAgICBz ZWxmLmFkZFN1Y2Nlc3MoInRlc3R4eCIsIlRlc3RYWCIsICJ0ZXN0eHgiLCAiIikKICAgICAgICBz ZWxmLmFkZEVycm9yKCJ0ZXN0eXkiLCAiVGVzdFlZIiwgIiIsICJ0ZXN0eXkiKQogICAgICAgIHNl bGYuYWRkRmFpbHVyZSgidGVzdHp6IiwgIlRlc3RaWiIsICIiLCAiIikKICAgICAgICBzZWxmLmFk ZFNraXAoInRlc3Rza2lwIikKCg==