Bug 9953 - patchtest-oe: some series cannot be parse correctly and no checks can be done
Summary: patchtest-oe: some series cannot be parse correctly and no checks can be done
Status: RESOLVED FIXED
Alias: None
Product: Patchwork/Patchtest
Classification: Yocto Project Subprojects
Component: Patchtest (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.2 M4
Assignee: Daniela Plascencia
QA Contact:
URL:
Whiteboard:
: 10355 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-07-15 15:58 UTC by Leonardo Sandoval Gonzalez
Modified: 2017-12-06 17:03 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Leonardo Sandoval Gonzalez 2016-07-15 15:58:56 UTC
The suite that check patches uses two (external) python modules to parse the mailbox (containing the patch series): mailbox and unidiff. The latter is used to convert diff into a python object but some of the series [1] raise an exception:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/unittest/suite.py", line 146, in _handleClassSetUp
    setUpClass()
  File "/home/lsandov1/repos/pt-oe/base.py", line 38, in setUpClass
    cls.patchset = PatchSet(f, encoding=u'UTF-8')
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
UnidiffParseError: Hunk diff line expected: diff --git a/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch b/meta/recipes-graphics/directfb/d\
irectfb/fix-client-gfx_state-initialisation.patch

We need to figure out the root cause (series malformed or patch syntax not recognized by the library?).


[1] https://patchwork.openembedded.org/series/11/
Comment 1 Leonardo Sandoval Gonzalez 2016-07-15 16:35:33 UTC
Another series with problems: https://patchwork.openembedded.org/series/193/

Traceback (most recent call last):
  File "/usr/lib64/python2.7/unittest/suite.py", line 146, in _handleClassSetUp
    setUpClass()
  File "/home/lsandov1/repos/pt-oe/base.py", line 38, in setUpClass
    cls.patchset = PatchSet(f, encoding=u'UTF-8')
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
UnidiffParseError: Hunk diff line expected: self.rpm_libdir[1:])
Comment 2 Leonardo Sandoval Gonzalez 2016-08-31 15:02:14 UTC
Due to time constrains, moving it to M4.
Comment 3 Leonardo Sandoval Gonzalez 2016-09-28 21:02:41 UTC
The problems comes from patchwork. Somehow, pw instruments the patch, destroying the diff format.

This is way another way to reproduce it without patchtest:
wget https://patchwork.openembedded.org/patch/125979/raw/ -O pw.mbox 
wget http://git.yoctoproject.org/cgit/cgit.cgi/poky/patch/?id=8c95d2220f1b477e32ed6cf4a28199590d15a411 -O git.patch


>>> import unidiff
>>> unidiff.PatchSet.from_filename('git.patch')
<PatchSet: [<PatchedFile: meta/recipes-graphics/directfb/directfb-examples/configure.in-Fix-string-argument-syntax.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb-examples_1.7.0.bb>, <PatchedFile: meta/recipes-graphics/directfb/directfb.inc>, <PatchedFile: meta/recipes-graphics/directfb/directfb/0001-gfx-direct-Aboid-usng-VLAs-and-printf-formats.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/bashism.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/compar_fn_t.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/configurefix.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/fusion.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/union-sigval.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/use-PTHREAD_MUTEX_RECURSIVE.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb_1.7.7.bb>]>
>>> unidiff.PatchSet.from_filename('pw.patch')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 326, in from_filename
    instance = cls(f)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
unidiff.errors.UnidiffParseError: Hunk diff line expected: diff --git a/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch b/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch


Perhaps long diff lines are being cut by pw, breaking the patch.
Comment 4 Leonardo Sandoval Gonzalez 2016-09-28 21:05:53 UTC
Damien,

we are seeing that long lines on mailing list patches are being truncated by PW. Do you happen to know the reason?

Daniela,
Please continue your debugging and file a bug in pw repository if necessary, so the pw community is aware of this problem.
Comment 5 Daniela Plascencia 2016-10-04 22:29:56 UTC
*** Bug 10355 has been marked as a duplicate of this bug. ***