Bug 8394 - patchwork: ensure that patches received during a PW server outage are processed
Summary: patchwork: ensure that patches received during a PW server outage are processed
Status: RESOLVED FIXED
Alias: None
Product: Patchwork/Patchtest
Classification: Yocto Project Subprojects
Component: Patchwork (show other bugs)
Version: unspecified
Hardware: All Multiple
: Medium enhancement
Target Milestone: Future
Assignee: Unassigned
QA Contact: Apoorv
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-09-24 21:25 UTC by Jose Lamego
Modified: 2022-05-12 15:25 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jose Lamego 2015-09-24 21:25:28 UTC
All patches that are received at the mailing list during a Patchwork server outage (either planned or not) are obviously missing at Patchwork when the service is restored. When the service is available again, it continues processing any new patch email, but currently there is no automated process to be aware of such missing patches, nor to re-parse the emails, so they and all the subsequent related comments are not displayed in Patchwork.

Probably a daily-nightly job to review the mail list content could ensure that any missed patch is processed, or (if the resulting overhead is unacceptable) at least provide a report that could be used for a manual fix.
Comment 1 Damien Lespiau 2015-10-26 16:53:10 UTC
Patches are delivered through SMTP, so if the patchwork server is down, there's some wiggle room as SMTP servers will try to deliver the message again later on.

Another option is to use a gmail account to receive mails from a mailing list and then fetchmail + procmail from the patchwork server to inject the mails into the system from the gmail account.

So, I don't think there's any actionable item here and I'd close this bug.
Comment 2 Jose Lamego 2015-11-04 18:45:25 UTC
In response to comment 1:

The option to wait for SMTP servers to finally deliver any message that didn't make it during a Patchwork server down time does not always work:

An example of a patch message sent during a Patchwork server down time is [1]. This patch is already merged in master [2], but since original message was not received, neither this, the subsequent messages nor the merge are displayed in patchwork.

Regarding the Gmail account option, we did have that in the past, but we confirmed that some emails were blocked by the email account provider due to high traffic considerations (reference in[3]), so we decided to use another one.

If I understand correctly, what you are suggesting is to use this Gmail account + fetchmail/procmail as a backup process for mail injection to ensure retrieval of any patch that was skipped during the "main" process. Am I right?

[1] http://lists.openembedded.org/pipermail/openembedded-core/2015-September/110947.html
[2] commit 4562f3f7713f836e6060a23357ff805c38983390
[3] https://bugzilla.yoctoproject.org/show_bug.cgi?id=7657
Comment 3 Damien Lespiau 2015-11-04 22:01:48 UTC
What I'm saying is: as long as you have reliable mail delivery to the patchwork email address, you can use that mailbox as buffer until the patchwork service comes back up and start processing mails again. It's all on the side of mail configuration and how patches are delivered to parsemail.sh.

I don't think this problem is high priority, but your project may think differently.
Comment 4 Jose Lamego 2015-11-04 22:23:25 UTC
In response to Comment 3:

Agree on the low priority of this issue given the few occurrences for a server outage. So we can prioritize other issues and when appropriate, return to this and evaluate if an alternate email delivery account/provider would be required.
Comment 5 Randy MacLeod 2022-05-12 15:25:00 UTC
WE are working off lore.kernel.org now