Bug 7140 - Performance issue when using MySQL with innodb on Toaster
Summary: Performance issue when using MySQL with innodb on Toaster
Status: RESOLVED FIXED
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.8 M3
Assignee: Alexandru Damian
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-01-14 14:11 UTC by Magnus D
Modified: 2015-02-25 14:03 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Magnus D 2015-01-14 14:11:46 UTC
This problem applies to Toaster using commit 876370419a50264a28feb4278f8c465b0019e709 from the master branch.

MySQL 5.5.40 on Ubuntu uses InnoDB as the default engine. When running Toaster it takes about 20 minutes to save the results to the database. A brief analysis shows that every single query is wrapped in explicit set autocommit commands. Example from the MySQL log:

Query     set autocommit=0
Query     INSERT INTO `orm_package_file` (`package_id`, `path`, `size`) VALUES (7, '/etc/init.d/modutils.sh', 1223)
Query     commit
Query     set autocommit=1

For a normal build there are more than 50000 such explicit commands, resulting in poor performance. The same wrapping of queries appears when using MyISAM, but overall performance is better.

A possible solution would be to disable the autocommit in Django.MySQL, and add explicit commits.
Comment 1 Alexandru Damian 2015-02-18 15:12:46 UTC
Pushed for review on 

poky-contrib: adamian/20150217_bug_7004

Basically moving all build data in a single transaction, data insertion time using InnoDB over TCP/IP dropped from 5 minutes to 3 seconds.
Comment 2 Alexandru Damian 2015-02-25 14:03:33 UTC
Merged to master.