Bug 8394

Summary:	patchwork: ensure that patches received during a PW server outage are processed
Product:	[Yocto Project Subprojects] Patchwork/Patchtest	Reporter:	Jose Lamego <jose.a.lamego>
Component:	Patchwork	Assignee:	Unassigned <unassigned>
Status:	RESOLVED FIXED	QA Contact:	Apoorv <apoorv.sangal>
Severity:	enhancement
Priority:	Medium	CC:	benjamin.esquivel, bluelightning, damien.lespiau, jose.perez.carranza, mhalstead, randy.macleod
Version:	unspecified
Target Milestone:	Future
Hardware:	All
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Jose Lamego 2015-09-24 21:25:28 UTC

All patches that are received at the mailing list during a Patchwork server outage (either planned or not) are obviously missing at Patchwork when the service is restored. When the service is available again, it continues processing any new patch email, but currently there is no automated process to be aware of such missing patches, nor to re-parse the emails, so they and all the subsequent related comments are not displayed in Patchwork.

Probably a daily-nightly job to review the mail list content could ensure that any missed patch is processed, or (if the resulting overhead is unacceptable) at least provide a report that could be used for a manual fix.

Comment 1 Damien Lespiau 2015-10-26 16:53:10 UTC

Patches are delivered through SMTP, so if the patchwork server is down, there's some wiggle room as SMTP servers will try to deliver the message again later on.

Another option is to use a gmail account to receive mails from a mailing list and then fetchmail + procmail from the patchwork server to inject the mails into the system from the gmail account.

So, I don't think there's any actionable item here and I'd close this bug.

Comment 2 Jose Lamego 2015-11-04 18:45:25 UTC

In response to comment 1:

The option to wait for SMTP servers to finally deliver any message that didn't make it during a Patchwork server down time does not always work:

An example of a patch message sent during a Patchwork server down time is [1]. This patch is already merged in master [2], but since original message was not received, neither this, the subsequent messages nor the merge are displayed in patchwork.

Regarding the Gmail account option, we did have that in the past, but we confirmed that some emails were blocked by the email account provider due to high traffic considerations (reference in[3]), so we decided to use another one.

If I understand correctly, what you are suggesting is to use this Gmail account + fetchmail/procmail as a backup process for mail injection to ensure retrieval of any patch that was skipped during the "main" process. Am I right?

[1] http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110947.html
[2] commit 4562f3f7713f836e6060a23357ff805c38983390
[3] https://bugzilla.yoctoproject.org/show_bug.cgi?id=7657

Comment 3 Damien Lespiau 2015-11-04 22:01:48 UTC

What I'm saying is: as long as you have reliable mail delivery to the patchwork email address, you can use that mailbox as buffer until the patchwork service comes back up and start processing mails again. It's all on the side of mail configuration and how patches are delivered to parsemail.sh.

I don't think this problem is high priority, but your project may think differently.

Comment 4 Jose Lamego 2015-11-04 22:23:25 UTC

In response to Comment 3:

Agree on the low priority of this issue given the few occurrences for a server outage. So we can prioritize other issues and when appropriate, return to this and evaluate if an alternate email delivery account/provider would be required.

Comment 5 Randy MacLeod 2022-05-12 15:25:00 UTC

WE are working off lore.kernel.org now