13248 2019-03-28 15:47:46 +0000 New QA process issues 2019-06-27 14:49:17 +0000 1 1 10 QA/Testing Test Plans/Suite test-plans/suite unspecified x86 Multiple RESOLVED FIXED High normal 2.8 M2 1 richard.purdie apoorv.sangal akuster bluelightning ee.peng.yeoh randy.macleod ross.burton sangeeta.jain tracey.m.erway --- --- Yes (doc changes required) oldest_to_newest 83364 0 richard.purdie 2019-03-28 15:47:46 +0000 Now that we're running through the new QA process there are a few problems we're running into. Stephen and myself have no visibility into who is running tests, what progress is being made or when we can expect results (or which results to block on). At a bare minimum I'd propose we need to have "ack" replies to the "QA start" email from anyone who is planning to run tests saying what tests they plan to run and when they'll send the results back. This would mean we know what to wait for. This needs to be documented publicly. I've not seen the new QA process being talked about publicly with the community yet either? This doesn't help with progress updates, not sure what we can do there. We're also seeing problems with the way ptest issues are being reported. Examples are bugs https://bugzilla.yoctoproject.org/show_bug.cgi?id=13238 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13239 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13240 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13241 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13242 . 13242 is puzzling as all the tests are passing (which is good) The test results reported here differ quite substantially to those from other builds, e.g. the report in: https://autobuilder.yocto.io/pub/non-release/20190326-8/testresults/testresult-report.txt which has one test failure for python3. Its unclear if the reported bugs are in QA locally run tests or tests run on the public autobuilder. The logs shown should probably include the high level ptest summary and the actual log output from the ptests. We spent quite some time in the triage meeting trying to work out where the problems may be and who should try and do something about them. Its certainly clear that we need to improve the reporting process though to include more information and to report things correctly. Also, we need to identify "timeout" tests. These are show in the main report with a T against them (e.g. strace, python3, lttng-tools). These should be specific bugs making it clear a timeout occurred with the test. Right now I don't think the QA process is flagging those up and it should. 83399 1 sangeeta.jain 2019-04-02 02:53:14 +0000 for ptest bugs, https://bugzilla.yoctoproject.org/show_bug.cgi?id=13238 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13239 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13240 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13241 , https://bugzilla.yoctoproject.org/show_bug.cgi?id=13242 these were reported on the basis on results from ptest run by Intel QA team. But these results were found to be inconsistent and inaccurate in follow up investigations, hence these bugs are marked as "NOT A BUG". Final ptest bugs for 2.7 M3 RC1 are files as follows: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13253 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13254 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13249 83400 2 sangeeta.jain 2019-04-02 02:56:01 +0000 Also, we need to identify "timeout" tests. These are show in the main report with a T against them (e.g. strace, python3, lttng-tools). These should be specific bugs making it clear a timeout occurred with the test. Right now I don't think the QA process is flagging those up and it should. QA team was not flagging "timeout" issues so far. We take it as an AR and start reporting timeout issues. For 2.7 M3 RC1, 3 timeout issues reported: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13255 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13256 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13257 83401 3 sangeeta.jain 2019-04-02 03:03:29 +0000 (In reply to comment #0) > Now that we're running through the new QA process there are a few problems > we're running into. > > Stephen and myself have no visibility into who is running tests, what > progress is being made or when we can expect results (or which results to > block on). This can be resolved once we can upload test results on "yocto-testresults" repo. You can see what tests are completed and results are available. > > At a bare minimum I'd propose we need to have "ack" replies to the "QA > start" email from anyone who is planning to run tests saying what tests they > plan to run and when they'll send the results back. This would mean we know > what to wait for. > Agree on this. We can send you an email for our plan of QA cycle specifying what test we are going to run on which build. 83402 4 4457 ee.peng.yeoh 2019-04-02 03:32:39 +0000 Created attachment 4457 2.7 m3 rc1 report from intel and windriver 83403 5 ee.peng.yeoh 2019-04-02 03:34:15 +0000 Sent the new QA process to yocto mailing list. Here was the link https://lists.yoctoproject.org/pipermail/yocto/2019-April/044661.html For new QA process trial run (2.7 m3 rc1), extra test results from intel and windriver team are compiled and sent to Richard for review. Attached the QA result report for extra test results from intel and windriver. 83576 6 randy.macleod 2019-04-18 15:11:17 +0000 We will create new sub-bug to track unresolved activities then close the defect. 84095 7 ee.peng.yeoh 2019-06-07 03:05:42 +0000 Documented the new QA process: https://wiki.yoctoproject.org/wiki/New_QA_process Attached a link at main QA page to the new QA process page: https://wiki.yoctoproject.org/wiki/QA#New_QA_Process 84096 8 akuster 2019-06-07 03:26:03 +0000 (In reply to comment #7) > Documented the new QA process: > https://wiki.yoctoproject.org/wiki/New_QA_process > > Attached a link at main QA page to the new QA process page: > https://wiki.yoctoproject.org/wiki/QA#New_QA_Process thank you 84161 9 apoorv.sangal 2019-06-12 23:36:38 +0000 The resulttool is command line tool available inside poky/scripts, it was developed to manage test result files for both automated tests (eg. oeqa/runtime, oeqa/selftest, oeqa/sdk, etc) and manual tests (eg. oeqa/manual). It was developed as an alternative tool to replace test result storage and reporting using Testopia and Wiki Page. https://wiki.yoctoproject.org/wiki/Resulttool 84328 10 richard.purdie 2019-06-27 14:49:17 +0000 QA wiki page has now been updated with the new process. 4457 2019-04-02 03:32:39 +0000 2019-04-02 03:32:39 +0000 2.7 m3 rc1 report from intel and windriver 2.7_m3_rc1_intel_wr_merged_report application/octet-stream 8304 ee.peng.yeoh PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KVGVz dCBSZXN1bHQgU3RhdHVzIFN1bW1hcnkgKENvdW50cy9QZXJjZW50YWdlcyBzb3J0ZWQgYnkgdGVz dHNlcmllcywgSUQpCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09Ci0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tClRlc3QgU2VyaWVzICAgICB8IElEICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgUGFzc2VkICAgIHwgRmFpbGVkICAgfCBTa2lw cGVkIAotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LQpiZWFnbGVib25lICAgICAgfCBtYW51YWxfYnNwcy1od18yMDE5MDMyNTA4NTE0MSAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICB8IDIyICg0OCUpICB8IDAgKDAlKSAgIHwgMjQgKDUyJSkK YmVhZ2xlYm9uZSAgICAgIHwgbWFudWFsX2NvbXBsaWFuY2UtdGVzdF8yMDE5MDMyNjEzNTIxNyAg ICAgICAgICAgICAgICAgICAgICAgfCA0ICg2NyUpICAgfCAwICgwJSkgICB8IDIgKDMzJSkgCmJl YWdsZWJvbmUgICAgICB8IHJ1bnRpbWVfY29yZS1pbWFnZS1zYXRvLXNka19iZWFnbGVib25lLXlv Y3RvXzIwMTkwMzIyMDIzMDE4IHwgNDYgKDc3JSkgIHwgMiAoMyUpICAgfCAxMiAoMjAlKQpiaXRi YWtlICAgICAgICAgfCBtYW51YWxfb2UtY29yZV8yMDE5MDMyMTA4NTEyMCAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB8IDcgKDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKYnVpbGQt YXBwbGlhbmNlIHwgbWFudWFsX2J1aWxkLWFwcGxpYW5jZV8yMDE5MDMyODEzMjAyMCAgICAgICAg ICAgICAgICAgICAgICAgfCAxICgzMyUpICAgfCAxICgzMyUpICB8IDEgKDMzJSkgCmNyb3BzICAg ICAgICAgICB8IHNka2V4dF9jb3JlLWltYWdlLXNhdG8tc2RrX3g4Nl82NF9xZW11eDg2XzIwMTkw MzI1MDQ1NDAxICAgIHwgMTcgKDgxJSkgIHwgMCAoMCUpICAgfCA0ICgxOSUpIAplY2xpcHNlLXBs dWdpbiAgfCBtYW51YWxfZWNsaXBzZS1wbHVnaW5fMjAxOTAzMjYxMzI5NTcgICAgICAgICAgICAg ICAgICAgICAgICB8IDYgKDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKZWNsaXBzZS1wbHVn aW4gIHwgbWFudWFsX2VjbGlwc2UtcGx1Z2luXzIwMTkwMzI2MTcwNzU2ICAgICAgICAgICAgICAg ICAgICAgICAgfCA2ICgxMDAlKSAgfCAwICgwJSkgICB8IDAgKDAlKSAgCmVkZ2Vyb3V0ZXIgICAg ICB8IG1hbnVhbF9ic3BzLWh3XzIwMTkwMzI1MTAxNzQ4ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHwgMTEgKDI0JSkgIHwgMCAoMCUpICAgfCAzNSAoNzYlKQplZGdlcm91dGVyICAgICAg fCBtYW51YWxfY29tcGxpYW5jZS10ZXN0XzIwMTkwMzI2MTM1OTM4ICAgICAgICAgICAgICAgICAg ICAgICB8IDIgKDMzJSkgICB8IDAgKDAlKSAgIHwgNCAoNjclKSAKZWRnZXJvdXRlciAgICAgIHwg cnVudGltZV9jb3JlLWltYWdlLXNhdG8tc2RrX2VkZ2Vyb3V0ZXJfMjAxOTAzMjEwMDI2MDQgICAg ICAgfCA0NSAoNzUlKSAgfCAzICg1JSkgICB8IDEyICgyMCUpCmtlcm5lbC1kZXYgICAgICB8IG1h bnVhbF9rZXJuZWwtZGV2XzIwMTkwMzIxMTMyOTU2ICAgICAgICAgICAgICAgICAgICAgICAgICAg IHwgOSAoMTAwJSkgIHwgMCAoMCUpICAgfCAwICgwJSkgIAptaW5ub3dib2FyZCAgICAgfCBtYW51 YWxfYnNwcy1od18yMDE5MDMyMjE3MjkxNSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8 IDQ2ICgxMDAlKSB8IDAgKDAlKSAgIHwgMCAoMCUpICAKbWlubm93Ym9hcmQgICAgIHwgcnVudGlt ZV9jb3JlLWltYWdlLWxzYi1zZGtfZ2VuZXJpY3g4Ni02NF8yMDE5MDMyMzA2MDkxOCAgICAgfCA0 NyAoNzglKSAgfCAwICgwJSkgICB8IDEzICgyMiUpCm1pbm5vd2JvYXJkICAgICB8IHJ1bnRpbWVf Y29yZS1pbWFnZS1sc2Itc2RrX2dlbmVyaWN4ODZfMjAxOTAzMjMwOTE2NDAgICAgICAgIHwgNDcg KDc4JSkgIHwgMCAoMCUpICAgfCAxMyAoMjIlKQptaW5ub3dib2FyZCAgICAgfCBydW50aW1lX2Nv cmUtaW1hZ2Utc2F0by1zZGtfZ2VuZXJpY3g4Ni02NF8yMDE5MDMyMjA3MjgzMiAgICB8IDQ4ICg4 MCUpICB8IDAgKDAlKSAgIHwgMTIgKDIwJSkKbWlubm93Ym9hcmQgICAgIHwgcnVudGltZV9jb3Jl LWltYWdlLXNhdG8tc2RrX2dlbmVyaWN4ODZfMjAxOTAzMjMwNzI1NTEgICAgICAgfCA0OCAoODAl KSAgfCAwICgwJSkgICB8IDEyICgyMCUpCm1wYzgzMTVlLXJkYiAgICB8IG1hbnVhbF9ic3BzLWh3 XzIwMTkwMzI1MDkzOTAzICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgMTUgKDMzJSkg IHwgMCAoMCUpICAgfCAzMSAoNjclKQptcGM4MzE1ZS1yZGIgICAgfCBtYW51YWxfY29tcGxpYW5j ZS10ZXN0XzIwMTkwMzI2MTM1ODExICAgICAgICAgICAgICAgICAgICAgICB8IDIgKDMzJSkgICB8 IDAgKDAlKSAgIHwgNCAoNjclKSAKbXBjODMxNWUtcmRiICAgIHwgcnVudGltZV9jb3JlLWltYWdl LXNhdG8tc2RrX21wYzgzMTVlLXJkYl8yMDE5MDMyMTAxMTI1NyAgICAgfCA0NyAoNzglKSAgfCAx ICgyJSkgICB8IDEyICgyMCUpCm51YzYgICAgICAgICAgICB8IG1hbnVhbF9jb21wbGlhbmNlLXRl c3RfMjAxOTAzMjcxNTEyMjkgICAgICAgICAgICAgICAgICAgICAgIHwgNCAoNjclKSAgIHwgMCAo MCUpICAgfCAyICgzMyUpIApudWM2ICAgICAgICAgICAgfCBydW50aW1lX2NvcmUtaW1hZ2UtbHNi LXNka19nZW5lcmljeDg2LTY0XzIwMTkwMzI1MDY0NDI1ICAgICB8IDQ3ICg3OCUpICB8IDAgKDAl KSAgIHwgMTMgKDIyJSkKbnVjNiAgICAgICAgICAgIHwgcnVudGltZV9jb3JlLWltYWdlLXNhdG8t c2RrX2dlbmVyaWN4ODYtNjRfMjAxOTAzMjUwMjE2NDcgICAgfCA0OCAoODAlKSAgfCAwICgwJSkg ICB8IDEyICgyMCUpCm51YzcgICAgICAgICAgICB8IG1hbnVhbF9ic3BzLWh3XzIwMTkwMzIxMTQz MzQxICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgNDYgKDEwMCUpIHwgMCAoMCUpICAg fCAwICgwJSkgIApudWM3ICAgICAgICAgICAgfCBydW50aW1lX2NvcmUtaW1hZ2UtbHNiLXNka19n ZW5lcmljeDg2LTY0XzIwMTkwMzIyMDMxODA2ICAgICB8IDQ3ICg3OCUpICB8IDAgKDAlKSAgIHwg MTMgKDIyJSkKbnVjNyAgICAgICAgICAgIHwgcnVudGltZV9jb3JlLWltYWdlLXNhdG8tc2RrX2dl bmVyaWN4ODYtNjRfMjAxOTAzMjIwNTEwMjkgICAgfCA0OCAoODAlKSAgfCAwICgwJSkgICB8IDEy ICgyMCUpCnFlbXUtc2h1dGRvd24gICB8IG9lc2VsZnRlc3RfdWJ1bnR1LTE2LjA0X3FlbXVhcm02 NF8yMDE5MDMyMjE0MTAzNSAgICAgICAgICAgIHwgMiAoMTAwJSkgIHwgMCAoMCUpICAgfCAwICgw JSkgIApxZW11LXNodXRkb3duICAgfCBvZXNlbGZ0ZXN0X3VidW50dS0xNi4wNF9xZW11YXJtXzIw MTkwMzIxMTM0MDQ4ICAgICAgICAgICAgICB8IDIgKDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUp ICAKcWVtdS1zaHV0ZG93biAgIHwgb2VzZWxmdGVzdF91YnVudHUtMTYuMDRfcWVtdW1pcHM2NF8y MDE5MDMyMTE2MjM1MSAgICAgICAgICAgfCAwICgwJSkgICAgfCAyICgxMDAlKSB8IDAgKDAlKSAg CnFlbXUtc2h1dGRvd24gICB8IG9lc2VsZnRlc3RfdWJ1bnR1LTE2LjA0X3FlbXVtaXBzXzIwMTkw MzIyMTAzNDMzICAgICAgICAgICAgIHwgMCAoMCUpICAgIHwgMiAoMTAwJSkgfCAwICgwJSkgIApx ZW11LXNodXRkb3duICAgfCBvZXNlbGZ0ZXN0X3VidW50dS0xNi4wNF9xZW11cHBjXzIwMTkwMzIy MTYwMDA0ICAgICAgICAgICAgICB8IDIgKDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKcWVt dS1zaHV0ZG93biAgIHwgb2VzZWxmdGVzdF91YnVudHUtMTYuMDRfcWVtdXg4Ni02NF8yMDE5MDMy MTE3NDYzNyAgICAgICAgICAgfCAyICgxMDAlKSAgfCAwICgwJSkgICB8IDAgKDAlKSAgCnFlbXUt c2h1dGRvd24gICB8IG9lc2VsZnRlc3RfdWJ1bnR1LTE2LjA0X3FlbXV4ODZfMjAxOTAzMjExNzA1 MzUgICAgICAgICAgICAgIHwgMiAoMTAwJSkgIHwgMCAoMCUpICAgfCAwICgwJSkgIApxZW11YXJt NjQgICAgICAgfCBtYW51YWxfYnNwcy1xZW11XzIwMTkwMzI2MTY1NjEwICAgICAgICAgICAgICAg ICAgICAgICAgICAgICB8IDUgKDYyJSkgICB8IDAgKDAlKSAgIHwgMyAoMzglKSAKcWVtdWFybSAg ICAgICAgIHwgbWFudWFsX2JzcHMtcWVtdV8yMDE5MDMyNjE2NTQxOSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgfCA1ICg2MiUpICAgfCAwICgwJSkgICB8IDMgKDM4JSkgCnFlbXVtaXBzNjQg ICAgICB8IG1hbnVhbF9ic3BzLXFlbXVfMjAxOTAzMjYxNzAyMDIgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIHwgNSAoNjIlKSAgIHwgMCAoMCUpICAgfCAzICgzOCUpIApxZW11bWlwcyAgICAg ICAgfCBtYW51YWxfYnNwcy1xZW11XzIwMTkwMzI2MTY1ODA5ICAgICAgICAgICAgICAgICAgICAg ICAgICAgICB8IDUgKDYyJSkgICB8IDAgKDAlKSAgIHwgMyAoMzglKSAKcWVtdXBwYyAgICAgICAg IHwgbWFudWFsX2JzcHMtcWVtdV8yMDE5MDMyNjE2NTk0MiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgfCA1ICg2MiUpICAgfCAwICgwJSkgICB8IDMgKDM4JSkgCnFlbXV4ODYtNjQgICAgICB8 IG1hbnVhbF9ic3BzLXFlbXVfMjAxOTAzMjYxNjUxMjYgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHwgNSAoNjIlKSAgIHwgMCAoMCUpICAgfCAzICgzOCUpIApxZW11eDg2ICAgICAgICAgfCBt YW51YWxfYnNwcy1xZW11XzIwMTkwMzI2MTY0NzEzICAgICAgICAgICAgICAgICAgICAgICAgICAg ICB8IDggKDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKc2RrLWV4dHJhcyAgICAgIHwgbWFu dWFsX3Nka18yMDE5MDMyMTEyNTIyMyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg fCAxICgxMDAlKSAgfCAwICgwJSkgICB8IDAgKDAlKSAgCnNkay1leHRyYXMgICAgICB8IG1hbnVh bF9zZGtfMjAxOTAzMjgxMDMzNDEgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwg MSAoMTAwJSkgIHwgMCAoMCUpICAgfCAwICgwJSkgIApzZGstZXh0cmFzICAgICAgfCBvZXNlbGZ0 ZXN0X3VidW50dS0xNi4wNF9xZW11YXJtXzIwMTkwMzIyMTIzMzIyICAgICAgICAgICAgICB8IDMg KDEwMCUpICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKc2RrLWV4dHJhcyAgICAgIHwgb2VzZWxmdGVz dF91YnVudHUtMTYuMDRfcWVtdW1pcHNfMjAxOTAzMjIxMzA5NTEgICAgICAgICAgICAgfCAzICgx MDAlKSAgfCAwICgwJSkgICB8IDAgKDAlKSAgCnNkay1leHRyYXMgICAgICB8IG9lc2VsZnRlc3Rf dWJ1bnR1LTE2LjA0X3FlbXVwcGNfMjAxOTAzMjIxMzQ5MjAgICAgICAgICAgICAgIHwgMyAoMTAw JSkgIHwgMCAoMCUpICAgfCAwICgwJSkgIApzZGstZXh0cmFzICAgICAgfCBvZXNlbGZ0ZXN0X3Vi dW50dS0xNi4wNF9xZW11eDg2XzIwMTkwMzIyMTAyMDAyICAgICAgICAgICAgICB8IDMgKDEwMCUp ICB8IDAgKDAlKSAgIHwgMCAoMCUpICAKdG9hc3RlciAgICAgICAgIHwgbWFudWFsX3RvYXN0ZXIt bWFuYWdlZC1tb2RlXzIwMTkwMzI4MTE1NTIzICAgICAgICAgICAgICAgICAgfCA2MiAoOTUlKSAg fCAzICg1JSkgICB8IDAgKDAlKSAgCnRvYXN0ZXIgICAgICAgICB8IG1hbnVhbF90b2FzdGVyLXVu bWFuYWdlZC1tb2RlXzIwMTkwMzI4MTE1MDE1ICAgICAgICAgICAgICAgIHwgMjggKDEwMCUpIHwg MCAoMCUpICAgfCAwICgwJSkgIAotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQoKVGhlcmUgd2FzIG5vIHB0ZXN0IGRhdGEKCj09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09CkZhaWxlZCB0ZXN0IGNhc2Vz IChzb3J0ZWQgYnkgdGVzdHNlcmllcywgSUQpCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCnRlc3RzZXJpZXMgfCByZXN1bHRfaWQgOiBiZWFnbGVib25l IHwgcnVudGltZV9jb3JlLWltYWdlLXNhdG8tc2RrX2JlYWdsZWJvbmUteW9jdG9fMjAxOTAzMjIw MjMwMTgKICAgIHN0YXAuU3RhcFRlc3QudGVzdF9zdGFwCiAgICBwYXJzZWxvZ3MuUGFyc2VMb2dz VGVzdC50ZXN0X3BhcnNlbG9ncwoJIyMgY29tbWVudFtRQV06IFdSIHRlYW0gaXMgaW52ZXN0aWdh dGluZyBtb3JlIG9uIHRoaXMgaXNzdWUKdGVzdHNlcmllcyB8IHJlc3VsdF9pZCA6IGJ1aWxkLWFw cGxpYW5jZSB8IG1hbnVhbF9idWlsZC1hcHBsaWFuY2VfMjAxOTAzMjgxMzIwMjAKICAgIGJ1aWxk LWFwcGxpYW5jZS5idWlsZC1hcHBsaWFuY2UuQ3JlYXRlX2NvcmUtaW1hZ2Utc2F0by1zZGtfdXNp bmdfYnVpbGRfYXBwbGlhbmNlCnRlc3RzZXJpZXMgfCByZXN1bHRfaWQgOiBlZGdlcm91dGVyIHwg cnVudGltZV9jb3JlLWltYWdlLXNhdG8tc2RrX2VkZ2Vyb3V0ZXJfMjAxOTAzMjEwMDI2MDQKICAg IHhvcmcuWG9yZ1Rlc3QudGVzdF94b3JnX3J1bm5pbmcKCSMjIGNvbW1lbnRbUUFdOiBUaGlzIGlz IGFuIGV4cGVjdGVkIGZhaWx1cmUgZHVlIHRvIGh3IGxpbWl0YXRpb24uCglwYXJzZWxvZ3MuUGFy c2VMb2dzVGVzdC50ZXN0X3BhcnNlbG9ncwogICAgYnVpbGRjcGlvLkJ1aWxkQ3Bpb1Rlc3QudGVz dF9jcGlvCgkjIyBjb21tZW50W1FBXTogV1IgdGVhbSBpcyBpbnZlc3RpZ2F0aW5nIG1vcmUgb24g dGhlc2UgaXNzdWVzCnRlc3RzZXJpZXMgfCByZXN1bHRfaWQgOiBtcGM4MzE1ZS1yZGIgfCBydW50 aW1lX2NvcmUtaW1hZ2Utc2F0by1zZGtfbXBjODMxNWUtcmRiXzIwMTkwMzIxMDExMjU3CiAgICB4 b3JnLlhvcmdUZXN0LnRlc3RfeG9yZ19ydW5uaW5nCgkjIyBjb21tZW50W1FBXTogVGhpcyBpcyBh biBleHBlY3RlZCBmYWlsdXJlIGR1ZSB0byBodyBsaW1pdGF0aW9uLgp0ZXN0c2VyaWVzIHwgcmVz dWx0X2lkIDogcWVtdS1zaHV0ZG93biB8IG9lc2VsZnRlc3RfdWJ1bnR1LTE2LjA0X3FlbXVtaXBz NjRfMjAxOTAzMjExNjIzNTEKICAgIHJ1bnFlbXUuUWVtdVRlc3QudGVzdF9xZW11X2Nhbl9ib290 X25mc19hbmRfc2h1dGRvd24KICAgIHJ1bnFlbXUuUWVtdVRlc3QudGVzdF9xZW11X2Nhbl9zaHV0 ZG93bgp0ZXN0c2VyaWVzIHwgcmVzdWx0X2lkIDogcWVtdS1zaHV0ZG93biB8IG9lc2VsZnRlc3Rf dWJ1bnR1LTE2LjA0X3FlbXVtaXBzXzIwMTkwMzIyMTAzNDMzCiAgICBydW5xZW11LlFlbXVUZXN0 LnRlc3RfcWVtdV9jYW5fYm9vdF9uZnNfYW5kX3NodXRkb3duCiAgICBydW5xZW11LlFlbXVUZXN0 LnRlc3RfcWVtdV9jYW5fc2h1dGRvd24KdGVzdHNlcmllcyB8IHJlc3VsdF9pZCA6IHRvYXN0ZXIg fCBtYW51YWxfdG9hc3Rlci1tYW5hZ2VkLW1vZGVfMjAxOTAzMjgxMTU1MjMKICAgIHRvYXN0ZXIt bWFuYWdlZC1tb2RlLnRvYXN0ZXItbWFuYWdlZC5BZGRpbmdfcGFja2FnZXNfd2l0aG91dF9kZXBl bmRlbmNpZXNfZnJvbV9jdXN0b21faW1hZ2VzCiAgICB0b2FzdGVyLW1hbmFnZWQtbW9kZS50b2Fz dGVyLW1hbmFnZWQuUmVtb3ZpbmdfcGFja2FnZXNfd2l0aG91dF9hbmRfd2l0aF9kZXBlbmRlbmNp ZXNfX2Zyb21fY3VzdG9tX2ltYWdlcwogICAgdG9hc3Rlci1tYW5hZ2VkLW1vZGUudG9hc3Rlci1t YW5hZ2VkLkFkZGluZ19wYWNrYWdlc193aXRoX2RlcGVuZGVuY2llcwotLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoK