Bug 7140

Summary: Performance issue when using MySQL with innodb on Toaster
Product: [Build System, Metadata & Runtime] Toaster Reporter: Magnus D <magnustoaster>
Component: toasterAssignee: Alexandru Damian <alexandru.damian>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: alexandru.damian, belen.barros.pena, cristiana.voicu
Version: unspecified   
Target Milestone: 1.8 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Magnus D 2015-01-14 14:11:46 UTC 
This problem applies to Toaster using commit 876370419a50264a28feb4278f8c465b0019e709 from the master branch.

MySQL 5.5.40 on Ubuntu uses InnoDB as the default engine. When running Toaster it takes about 20 minutes to save the results to the database. A brief analysis shows that every single query is wrapped in explicit set autocommit commands. Example from the MySQL log:

Query     set autocommit=0
Query     INSERT INTO `orm_package_file` (`package_id`, `path`, `size`) VALUES (7, '/etc/init.d/modutils.sh', 1223)
Query     commit
Query     set autocommit=1

For a normal build there are more than 50000 such explicit commands, resulting in poor performance. The same wrapping of queries appears when using MyISAM, but overall performance is better.

A possible solution would be to disable the autocommit in Django.MySQL, and add explicit commits.
Comment 1 Alexandru Damian 2015-02-18 15:12:46 UTC 
Pushed for review on 

poky-contrib: adamian/20150217_bug_7004

Basically moving all build data in a single transaction, data insertion time using InnoDB over TCP/IP dropped from 5 minutes to 3 seconds.
Comment 2 Alexandru Damian 2015-02-25 14:03:33 UTC 
Merged to master.