Bug 6693

Summary: package_deb doesn't handle non-ASCII control files
Product: [Runtime] Package Management Issues Reporter: Ross Burton <ross.burton>
Component: Debian Packaging - DEBAssignee: Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: anibal.limon, jose.a.lamego, richard.purdie
Version: 1.7   
Target Milestone: 1.9 M3   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2014-09-08 11:28:32 UTC
If eg a DESCRIPTION field contains non-ASCII, package_deb will throw an exception:

*** 0137:                         ctrlfile.write(u'%s\n' % unicode(textwrap.fill(description.strip(), width=74, initial_indent=' ', subsequent_indent=' ')))
     0138:
     0139:                else:
     0140:                     ctrlfile.write(unicode(c % tuple(pullData(fs, localdata))))
     0141:        except KeyError:

Exception: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 45: ordinal not in range(128)
Comment 1 Richard Purdie 2014-09-11 14:38:00 UTC
Rather than package specific backends falling over this kind of issue we should sanity check the various fields in insane.bbclass for these kinds of issues. I know upper case and certain symbols can cause issues in some fields for example.
Comment 2 Leonardo Sandoval Gonzalez 2015-08-13 17:08:33 UTC
A couple of patches sent to the community, so under reviewed now.
Comment 3 Ross Burton 2015-08-13 19:07:09 UTC
Considering I filed this last year I should have been more explicit.

Debian control files are defined to be in UTF-8, so we should just use a UTF-8 encoder when writing the files instead of ASCII.  RPM also appears to be UTF-8.  Not sure about opkg right now, I suspect it just assumes ASCII-compatible and hopes for the best...
Comment 4 Leonardo Sandoval Gonzalez 2015-08-17 15:17:07 UTC
V2 patch sent to the ML. This time, invalid characters are considered those non-UTF8.
Comment 5 Leonardo Sandoval Gonzalez 2015-09-15 15:05:03 UTC
Patch merged on master:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1019d8c802d426e0692f3b5939fa34fb46ff002d