15262 2023-10-27 11:51:41 +0000 Improve patchtest testresult messages 2024-02-29 18:09:58 +0000 1 1 6 Yocto Project Subprojects Patchwork/Patchtest Patchtest 5.0 x86 Multiple RESOLVED FIXED Medium+ normal 5.0 M3 1 tgamblin simone.p.weiss randy.macleod simone.p.weiss --- --- Yes (doc changes required) oldest_to_newest 97019 0 tgamblin 2023-10-27 11:51:41 +0000 Patchtest provides a simple set of PASS/FAIL/SKIP results with names of tests that were run, and (if failed or skipped) a short explanation of why and what to do. This is useful but could still be more user-friendly, especially when it comes to the automated service providing feedback on the mailing list. A non-exhaustive list of possible improvements: - Better test indicators (not just a reformatted version of the test names), including what the tests are looking for - Better suggestions for failing patches - Links to the contributor guide or other docs - (If/when the test suite is expanded) Direct feedback on how variables, syntax, etc. are being misused 98216 1 tgamblin 2024-02-15 19:58:41 +0000 Something that recently came up was a desire to remove all of the unittest-specific text that gets printed with the results, e.g.: ---------------------------------------------------------------------- Ran 21 tests in 2.715s OK In theory, this should be catchable by redirecting the stderr output from unittest into a variable that is otherwise ignored, although that's kind of hacky. A better way would be to implement a custom test framework for patchtest, but that's obviously a lot more complicated. I've been wondering about pytest and whether that might be better too, but haven't had time to really look and see what the changes would involve. There might be another way that I've missed... 98219 2 simone.p.weiss 2024-02-15 20:21:39 +0000 pytest should ideally also be more performant, which could help with overhead added when extending patchtests like the one mentioned here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15400#c1