Bug 9953

Summary: patchtest-oe: some series cannot be parse correctly and no checks can be done
Product: [Yocto Project Subprojects] Patchwork/Patchtest Reporter: Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Component: PatchtestAssignee: Daniela Plascencia <daniela.plascencia>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: damien.lespiau, daniela.plascencia, sgw
Version: 2.3   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Leonardo Sandoval Gonzalez 2016-07-15 15:58:56 UTC
The suite that check patches uses two (external) python modules to parse the mailbox (containing the patch series): mailbox and unidiff. The latter is used to convert diff into a python object but some of the series [1] raise an exception:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/unittest/suite.py", line 146, in _handleClassSetUp
    setUpClass()
  File "/home/lsandov1/repos/pt-oe/base.py", line 38, in setUpClass
    cls.patchset = PatchSet(f, encoding=u'UTF-8')
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
UnidiffParseError: Hunk diff line expected: diff --git a/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch b/meta/recipes-graphics/directfb/d\
irectfb/fix-client-gfx_state-initialisation.patch

We need to figure out the root cause (series malformed or patch syntax not recognized by the library?).


[1] https://patchwork.openembedded.org/series/11/
Comment 1 Leonardo Sandoval Gonzalez 2016-07-15 16:35:33 UTC
Another series with problems: https://patchwork.openembedded.org/series/193/

Traceback (most recent call last):
  File "/usr/lib64/python2.7/unittest/suite.py", line 146, in _handleClassSetUp
    setUpClass()
  File "/home/lsandov1/repos/pt-oe/base.py", line 38, in setUpClass
    cls.patchset = PatchSet(f, encoding=u'UTF-8')
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/lib/python2.7/site-packages/unidiff-0.5.2-py2.7.egg/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
UnidiffParseError: Hunk diff line expected: self.rpm_libdir[1:])
Comment 2 Leonardo Sandoval Gonzalez 2016-08-31 15:02:14 UTC
Due to time constrains, moving it to M4.
Comment 3 Leonardo Sandoval Gonzalez 2016-09-28 21:02:41 UTC
The problems comes from patchwork. Somehow, pw instruments the patch, destroying the diff format.

This is way another way to reproduce it without patchtest:
wget https://patchwork.openembedded.org/patch/125979/raw/ -O pw.mbox 
wget http://git.yoctoproject.org/cgit/cgit.cgi/poky/patch/?id=8c95d2220f1b477e32ed6cf4a28199590d15a411 -O git.patch


>>> import unidiff
>>> unidiff.PatchSet.from_filename('git.patch')
<PatchSet: [<PatchedFile: meta/recipes-graphics/directfb/directfb-examples/configure.in-Fix-string-argument-syntax.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb-examples_1.7.0.bb>, <PatchedFile: meta/recipes-graphics/directfb/directfb.inc>, <PatchedFile: meta/recipes-graphics/directfb/directfb/0001-gfx-direct-Aboid-usng-VLAs-and-printf-formats.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/bashism.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/compar_fn_t.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/configurefix.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/fusion.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/union-sigval.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb/use-PTHREAD_MUTEX_RECURSIVE.patch>, <PatchedFile: meta/recipes-graphics/directfb/directfb_1.7.7.bb>]>
>>> unidiff.PatchSet.from_filename('pw.patch')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 326, in from_filename
    instance = cls(f)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 278, in __init__
    self._parse(data, encoding=encoding)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 320, in _parse
    current_file._parse_hunk(line, diff, encoding)
  File "/usr/local/lib/python2.7/dist-packages/unidiff/patch.py", line 193, in _parse_hunk
    raise UnidiffParseError('Hunk diff line expected: %s' % line)
unidiff.errors.UnidiffParseError: Hunk diff line expected: diff --git a/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch b/meta/recipes-graphics/directfb/directfb/fix-client-gfx_state-initialisation.patch


Perhaps long diff lines are being cut by pw, breaking the patch.
Comment 4 Leonardo Sandoval Gonzalez 2016-09-28 21:05:53 UTC
Damien,

we are seeing that long lines on mailing list patches are being truncated by PW. Do you happen to know the reason?

Daniela,
Please continue your debugging and file a bug in pw repository if necessary, so the pw community is aware of this problem.
Comment 5 Daniela Plascencia 2016-10-04 22:29:56 UTC
*** Bug 10355 has been marked as a duplicate of this bug. ***