Sometimes patches that are sent to the mailing list aren't showing up in Patchwork. Some time ago we had problems with patches that contained non-ASCII characters (e.g. in the submitter's name); that may still be a problem, but it seems like there are other triggers for this issue that aren't immediately obvious. Some recent examples: http://lists.openembedded.org/pipermail/openembedded-devel/2015-April/101112.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-April/101057.html It's possible that this is a problem with the OE Patchwork infrastructure rather than Patchwork itself, but regardless of where the issue lies we need to fix it.
More examples: http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101364.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101358.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101361.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101387.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101402.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101404.html
It has become apparent that at least some of the mails not appearing in patchwork are lost because they are being marked as spam in the gmail account that is subscribed to the list in order to have mails delivered to it. Martin still thinks we have other issues so this doesn't necessarily account for everything: <JaMa> bluelightning: it's only one of the reasons for patches not appearing <JaMa> simple renames without any changes aren't recognized as patches as well <JaMa> and I know about couple cases like this which weren't in patchwork and weren't marked as spam (in my gmail folder)
Another case: http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101537.html not marked as spam in my gmail folder
More examples of patches marked as spam (and missing in patchwork) http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101454.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101533.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101544.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101547.html http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101548.html
Martin and Paul, I need to look at the complete header information in the notification emails received from the email list for the mentioned patches missing at patchwork, but I don't have them nor is that info available in the mailing list archive file. (I'm suspecting about the "List-Id" item) Do you guys have by any chance (any of) those emails?
Here are the headers from last one, I've also bounced it to you and will attach it as file (I guess the formatting will break here inline). Delivered-To: martin.jansa@gmail.com Received: by 10.28.133.18 with SMTP id h18csp167678wmd; Mon, 18 May 2015 07:14:08 -0700 (PDT) X-Received: by 10.68.222.198 with SMTP id qo6mr43908952pbc.149.1431958447882; Mon, 18 May 2015 07:14:07 -0700 (PDT) Return-Path: <openembedded-devel-bounces@lists.openembedded.org> Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id q2si16212619pdc.102.2015.05.18.07.14.06; Mon, 18 May 2015 07:14:07 -0700 (PDT) Received-SPF: pass (google.com: domain of openembedded-devel-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; spf=pass (google.com: domain of openembedded-devel-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mail=openembedded-devel-bounces@lists.openembedded.org Received: from mail.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 3385D7425F; Mon, 18 May 2015 14:13:34 +0000 (UTC) X-Original-To: openembedded-devel@lists.openembedded.org Delivered-To: openembedded-devel@lists.openembedded.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mail.openembedded.org (Postfix) with ESMTP id 7AE3874156 for <openembedded-devel@lists.openembedded.org>; Mon, 18 May 2015 14:12:33 +0000 (UTC) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP; 18 May 2015 07:12:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,453,1427785200"; d="scan'208";a="696587209" Received: from linux-8hqz.fi.intel.com (HELO linux-8hqz.site.fi.intel.com) ([10.237.68.142]) by orsmga001.jf.intel.com with ESMTP; 18 May 2015 07:12:32 -0700 From: Alexander Kanavin <alexander.kanavin@linux.intel.com> To: openembedded-devel@lists.openembedded.org Date: Mon, 18 May 2015 17:12:48 +0300 Message-Id: <1431958369-14668-5-git-send-email-alexander.kanavin@linux.intel.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1431958369-14668-1-git-send-email-alexander.kanavin@linux.intel.com> References: <1431958369-14668-1-git-send-email-alexander.kanavin@linux.intel.com> MIME-Version: 1.0 Subject: [oe] [PATCH 4/4] Rename gstreamer directory to gstreamer-0.10 X-BeenThere: openembedded-devel@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: openembedded-devel@lists.openembedded.org List-Id: Using the OpenEmbedded metadata to build Distributions <openembedded-devel.lists.openembedded.org> List-Unsubscribe: <http://lists.openembedded.org/mailman/options/openembedded-devel>, <mailto:openembedded-devel-request@lists.openembedded.org?subject=unsubscribe> List-Archive: <http://lists.openembedded.org/pipermail/openembedded-devel/> List-Post: <mailto:openembedded-devel@lists.openembedded.org> List-Help: <mailto:openembedded-devel-request@lists.openembedded.org?subject=help> List-Subscribe: <http://lists.openembedded.org/mailman/listinfo/openembedded-devel>, <mailto:openembedded-devel-request@lists.openembedded.org?subject=subscribe> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: openembedded-devel-bounces@lists.openembedded.org Errors-To: openembedded-devel-bounces@lists.openembedded.org X-TUID: fyBPNaTMIAY1
Created attachment 2518 [details] missed-e-mail.txt
Created attachment 2524 [details] Previously mentioned mails from my system I've attached a tarball of mboxes for some of the previously mentioned missing patch mails. What we really need is more non-spam examples at this point; at least some of these were ones that were blocked due to the spam issue so I suspect they'll probably go through our test system just fine.
It's confirmed that some emails from the mailing list are being blocked by the patchwork email account provider due to high traffic considerations. Decision has been taken to use a different email account o avoid this blocking. After implementing this change, new tests will be in order to discard other possible root causes for missing patches (if any).
Looping in Michael Halstead who will provide an update when the new email account is associated with the oe-patchwork instance.
What about "rename-only" patches which aren't recognized by patchwork as patches? Latest example: http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101652.html
Martin, I've run the provided example through a Patchwork staging instance we implemented for tests and it passed without errors (I'll attach a screen capture) so I believe we'll have to wait for the email account changes to be implemented before running more tests.
Created attachment 2527 [details] Patchwork staging instance screenshot
Created attachment 2528 [details] Rename-only patch email Just as a second check, the patch Martin highlighted earlier wasn't sent with -M so it did actually have a bunch of changes in it. I'm attaching a recent example of where -M has been used so there are no actual changes (even though in this case the patch was incorrect for the original purpose).
A patch was implemented to avoid skipped patches due to format variations in the commit description string. Michael Halstead is working on using a different email account to avoid notifications to be filtered as spam, which seems to be the only remaining cause for missing patches.
Michael: any update on patchwork e-mail account? It's still happening, another example from today: all 7 patches from http://lists.openembedded.org/pipermail/openembedded-devel/2015-June/102037.html
Michael H. has switched the email account. Expected behavior is that no patches submitted after June 30 should be missing from Patchwork. I will monitor for a couple days before closing this bug. As a side note, cover letters for patch series that are created using "format-patch" are not displayed, while those created with "create-pull-request" are, since this script adds the required commit information. Example: http://patchwork.openembedded.org/patch/96403/
Both email account change and applied patch proved to work as expected: Patches sent to the mailing lists are appropriately displayed in Patchwork, so I'm closing this bug now (see note below). Note.- An unexpected behavior was detected: New patches sent to the meta OE mailing list are appropriately displayed in Patchwork (http://patchwork.openembedded.org/project/oe/list/), but after some time they are wrongly archived and no longer available. This will be tracked as a separate bug in 7970.
Week or two ago, I've seen many newly shown patches sent long time ago, it looks like now it found all the patches originally marked as spam not read by patchwork, so I can confirm that missing patches due to spam were fixed. Separate issue with patches being archived is not a bug and I've closed that ticket now.
verified as per Martin's and Jose comments.
Some random occurrences of patches that are missing at patchwork were detected. One example is patch [1]. This same patch was appropriately parsed and displayed at the patchwork staging instance in [2] that mimics production configuration, so there is no issue with the email itself nor with the email delivery process. A possible cause for this type of error is the lack of serialization when parsing email notifications. This condition is already fixed in patchwork upstream: [3]. Additionally, there is also a mail notification for parsing errors implemented in patchwork upstream: [4] which could warn admins if/when this situation repeats. Both of the mentioned upstream patches add to the list of fixes that could benefit OE if patchwork is updated to a newer version. [1] http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111336.html [2] http://yctvm04.amr.corp.intel.com:8000/patch/7517849/ [3] https://patchwork.ozlabs.org/patch/532930/