15389 2024-02-08 08:51:41 +0000 Patchtest reports a failure but says "OK" at the end 2024-02-19 18:50:13 +0000 1 1 6 Yocto Project Subprojects Patchwork/Patchtest Patchtest 5.0 x86 Multiple RESOLVED FIXED Medium+ normal 5.0 M3 1 michael.opdenacker simone.p.weiss michael.opdenacker randy.macleod simone.p.weiss --- --- No (bug/feature does not impact docs) oldest_to_newest 98153 0 5009 michael.opdenacker 2024-02-08 08:51:41 +0000 Created attachment 5009 Patch without a Signed-off-by line I'm testing this on Poky master... Given the attached patch, patchtest reports a failure but says "OK" at the end: Testing patch v2-0002-alsa-tools-upgrade-1.2.5-1.2.11.patch Loading cache: 100% | | ETA: --:--:-- Loaded 0 entries from dependency cache. Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:13 Parsing of 912 .bb files complete (0 cached, 912 parsed). 1849 targets, 47 skipped, 0 masked, 0 errors. SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files) SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint) ---------------------------------------------------------------------- Ran 2 tests in 14.104s OK PASS: test author valid (test_mbox.TestMbox.test_author_valid) SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format) PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence) PASS: test mbox format (test_mbox.TestMbox.test_mbox_format) PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade) SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head) PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format) PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length) FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence) PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list) Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:00 Loaded 1849 entries from dependency cache. PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore) PASS: test lic files chksum modified not mentioned (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned) SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence) SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence) PASS: test max line length (test_metadata.TestMetadata.test_max_line_length) SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files) SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence) SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format) SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence) SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format) SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint) ---------------------------------------------------------------------- Ran 21 tests in 1.099s OK Because of this, the "FAIL" line got unnoticed, and I sent the patch in good faith. 98214 1 michael.opdenacker 2024-02-15 18:42:28 +0000 That's better, now, thanks Simone! ---------------------------------------------------------------------- patchtest: At least one patchtest caused a failure or an error - please check ---------------------------------------------------------------------- However, I think this would have been harder to miss: ---------------------------------------------------------------------- FAIL: At least one patchtest caused a failure or an error - please check ---------------------------------------------------------------------- I'm reopening the bug, but I don't mind if you disagree and close it again. It's already a good improvement, but I'm unsure people reading the output too quickly won't miss the error. Thanks again 98218 2 simone.p.weiss 2024-02-15 20:17:54 +0000 Hi Michael, While true that adding FAIL makes it even more obvious, the problem is that this would be picked up by patchtest-send-results will pick it up and put it in the list of failed testcases. I would hence keep it as is. 98222 3 michael.opdenacker 2024-02-16 07:46:34 +0000 Oops, I understand. "ERROR" maybe if it doesn't interfere? Anyway, I was just trying to help. Don't hesitate to close the bug again. 98224 4 simone.p.weiss 2024-02-16 16:22:35 +0000 Sure, this is absolutely possible. I can check and send a patch later for sth like WARN, or NOK. 5009 2024-02-08 08:51:41 +0000 2024-02-08 08:51:41 +0000 Patch without a Signed-off-by line v2-0002-alsa-tools-upgrade-1.2.5-1.2.11.patch text/plain 1491 michael.opdenacker RnJvbSA2MTNiNjBmODI0NmVhNGUzMTkwMmMxNzJlMGEzZDJjNmU3MmEyNWMxIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoYWVsIE9wZGVuYWNrZXIgPG1pY2hhZWwub3BkZW5hY2tl ckBib290bGluLmNvbT4KRGF0ZTogV2VkLCA3IEZlYiAyMDI0IDIyOjQwOjQ4ICswMTAwClN1Ympl Y3Q6IFtQQVRDSCB2MiAyLzRdIGFsc2EtdG9vbHM6IHVwZ3JhZGUgMS4yLjUgLT4gMS4yLjExCgot IFVwc3RyZWFtIHJlbGVhc2UgdXBkYXRlCiAgU2VlIGh0dHBzOi8vd3d3LmFsc2EtcHJvamVjdC5v cmcvd2lraS9EZXRhaWxlZF9jaGFuZ2VzX3YxLjIuMTBfdjEuMi4xMQoKLS0tCgpDaGFuZ2VzIGlu IFYyOgotIEZpeCBjb21taXQgdGl0bGUsIGNvcnJlY3RpbmcgdXBkYXRlZCB2ZXJzaW9uIGZvciBh bHNhLXRvb2xzCi0tLQogLi4uL2Fsc2Eve2Fsc2EtdG9vbHNfMS4yLjUuYmIgPT4gYWxzYS10b29s c18xLjIuMTEuYmJ9ICAgICAgICAgIHwgMiArLQogMSBmaWxlIGNoYW5nZWQsIDEgaW5zZXJ0aW9u KCspLCAxIGRlbGV0aW9uKC0pCiByZW5hbWUgbWV0YS9yZWNpcGVzLW11bHRpbWVkaWEvYWxzYS97 YWxzYS10b29sc18xLjIuNS5iYiA9PiBhbHNhLXRvb2xzXzEuMi4xMS5iYn0gKDk3JSkKCmRpZmYg LS1naXQgYS9tZXRhL3JlY2lwZXMtbXVsdGltZWRpYS9hbHNhL2Fsc2EtdG9vbHNfMS4yLjUuYmIg Yi9tZXRhL3JlY2lwZXMtbXVsdGltZWRpYS9hbHNhL2Fsc2EtdG9vbHNfMS4yLjExLmJiCnNpbWls YXJpdHkgaW5kZXggOTclCnJlbmFtZSBmcm9tIG1ldGEvcmVjaXBlcy1tdWx0aW1lZGlhL2Fsc2Ev YWxzYS10b29sc18xLjIuNS5iYgpyZW5hbWUgdG8gbWV0YS9yZWNpcGVzLW11bHRpbWVkaWEvYWxz YS9hbHNhLXRvb2xzXzEuMi4xMS5iYgppbmRleCA1NmFjYzg0NTU5Li41Mzg2ODA0MWMwIDEwMDY0 NAotLS0gYS9tZXRhL3JlY2lwZXMtbXVsdGltZWRpYS9hbHNhL2Fsc2EtdG9vbHNfMS4yLjUuYmIK KysrIGIvbWV0YS9yZWNpcGVzLW11bHRpbWVkaWEvYWxzYS9hbHNhLXRvb2xzXzEuMi4xMS5iYgpA QCAtMTQsNyArMTQsNyBAQCBMSUNfRklMRVNfQ0hLU1VNID0gImZpbGU6Ly9oZHNwbG9hZGVyL0NP UFlJTkc7bWQ1PTU5NTMwYmRmMzM2NTliMjllNzNkNGFkYjlmOWY2NQogCiBTUkNfVVJJID0gImh0 dHBzOi8vd3d3LmFsc2EtcHJvamVjdC5vcmcvZmlsZXMvcHViL3Rvb2xzLyR7QlB9LnRhci5iejIi CiAKLVNSQ19VUklbc2hhMjU2c3VtXSA9ICIzNWE3MTAyN2EwMWY0ZDdkZTQ3MjJlMjIzNTIwZTk0 MGRlNjhiM2M1NzBiNmM2NzE2OTE1NjdhZTI4Zjk4OTNlIgorU1JDX1VSSVtzaGEyNTZzdW1dID0g IjA5MTVjOTYzNGE1MDJmZDM2NTVjYTljNTc0ZDI1OWJjOWU3OTk4M2Q5MWQ0NWFlYWNmZjZmM2Mw MGY4YWUzZTkiCiAKIGluaGVyaXQgYXV0b3Rvb2xzLWJyb2tlbnNlcCBwa2djb25maWcKICMgYnJv a2Vuc2VwIGFzIGFzMTBrMSAoYW5kIHByb2JhYmx5IG1vcmUpIGZhaWwgb3V0IG9mIHRyZWUKLS0g CjIuMzQuMQoK