Bug 6693 - package_deb doesn't handle non-ASCII control files
Summary: package_deb doesn't handle non-ASCII control files
Status: RESOLVED FIXED
Alias: None
Product: Package Management Issues
Classification: Runtime
Component: Debian Packaging - DEB (show other bugs)
Version: 1.7
Hardware: All Multiple
: Medium normal
Target Milestone: 1.9 M3
Assignee: Leonardo Sandoval Gonzalez
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-09-08 11:28 UTC by Ross Burton
Modified: 2015-09-15 15:05 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Burton 2014-09-08 11:28:32 UTC 
If eg a DESCRIPTION field contains non-ASCII, package_deb will throw an exception:

*** 0137:                         ctrlfile.write(u'%s\n' % unicode(textwrap.fill(description.strip(), width=74, initial_indent=' ', subsequent_indent=' ')))
     0138:
     0139:                else:
     0140:                     ctrlfile.write(unicode(c % tuple(pullData(fs, localdata))))
     0141:        except KeyError:

Exception: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 45: ordinal not in range(128)
Comment 1 Richard Purdie 2014-09-11 14:38:00 UTC 
Rather than package specific backends falling over this kind of issue we should sanity check the various fields in insane.bbclass for these kinds of issues. I know upper case and certain symbols can cause issues in some fields for example.
Comment 2 Leonardo Sandoval Gonzalez 2015-08-13 17:08:33 UTC 
A couple of patches sent to the community, so under reviewed now.
Comment 3 Ross Burton 2015-08-13 19:07:09 UTC 
Considering I filed this last year I should have been more explicit.

Debian control files are defined to be in UTF-8, so we should just use a UTF-8 encoder when writing the files instead of ASCII.  RPM also appears to be UTF-8.  Not sure about opkg right now, I suspect it just assumes ASCII-compatible and hopes for the best...
Comment 4 Leonardo Sandoval Gonzalez 2015-08-17 15:17:07 UTC 
V2 patch sent to the ML. This time, invalid characters are considered those non-UTF8.
Comment 5 Leonardo Sandoval Gonzalez 2015-09-15 15:05:03 UTC 
Patch merged on master:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=1019d8c802d426e0692f3b5939fa34fb46ff002d