The current test already has a check that should send a failure message when the LIC_FILES_CHKSUM is modified and the explanation as to why is not present on the commit message. For some reason it is not detecting the patches lacking the explanation and not failing properly. The usage of Tinfoil may solve this issue.
This issue may be resolved with current master-next branches which will land to master shortly.
(In reply to comment #1) > This issue may be resolved with current master-next branches which will land > to master shortly. This issue is not present anymore considering the following: a) Both framework and test suite are on master-next b) Latest change on base.py: find_pn() returns PN instead of FN (PATH) c) All metadata test cases use parse_recipe() with PN as input instead of FN b and c are not in master-next, but certainly helped solving runtime errors and might be merged soon.
Changing status "IN PROGRESS REVIEW".
Latest changes on master solve this issue. http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe/log/