Bug 12723 - mysql requires unicode and char length filtering
Summary: mysql requires unicode and char length filtering
Status: IN PROGRESS IMPLEMENTATION
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 6.0
Assignee: David Reyna
QA Contact: Toaster default assignee
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2018-05-01 10:04 UTC by David Reyna
Modified: 2025-11-06 15:40 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
"lsupdates.py", patched to support MySQL (12.65 KB, application/octet-stream)
2018-05-01 10:12 UTC, David Reyna
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Reyna 2018-05-01 10:04:46 UTC
The MySQL database is more sensitive to content exceptions that Sqlite. Toaster needs additional filtering on problematic data to be correct for both database and not fail for MySql.

Observed issues causing fails in MySql:
  (a) Unicode characters are appearing in layer descriptions ('\\xC2\\x96', "meta-crosswalk")
  (b) Some layer URL's exceed 200 characters (204 characters, "meta-nexell")
  (c) Some recipe licenses exceed 200 characters (1451 characters, "linux-firmware")

NOTE: the MySQL docs imply a max character length of 256, but 204 characters fail in reality.

The solution is:
  (a) Filter unicode, e.g. li['description'].encode('ascii','replace')
  (b) Add proactive filter for char fields, e.g. generally enforce char limits, truncate then append a " ..." for extra long fields
  (c) Promote some Char fields to un-limited Text fields
Comment 1 David Reyna 2018-05-01 10:12:18 UTC
Created attachment 4294 [details]
"lsupdates.py", patched to support MySQL

This patched version of "bitbake/lib/toaster/orm/management/commands/lsupdates.py" allows integration with MySql.
Comment 2 Randy MacLeod 2025-04-10 13:53:17 UTC
David, 
Can Mingli help with this bug?
She has worked on MariaDB and is available to work on YP bugs now so I thought of her.