9651 2016-05-17 22:22:13 +0000 Error report web: Optimize searching times 2026-04-23 15:01:04 +0000 1 1 6 Yocto Project Subprojects Error Reporting tool Error Reporting Tool unspecified x86 Multiple NEW Medium enhancement Future 1 anibal.limon tim.orling belen.barros.pena benjamin.esquivel changqing.li juan.m.cruz.alcaraz randy.macleod richard.purdie tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 62272 0 anibal.limon 2016-05-17 22:22:13 +0000 When you made a search in error report web it takes too much ~1 minute depending on the search. This is because the tool is searching over the log without an index [1], currently MySQL supports Full Text Search but Django ORM didn't expose directly. The work is to create an full text search index for the log field and modify error report web code to use a custom MySQL query when MySQL is available. [1] http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/tree/Post/views.py#n202 62325 1 richard.purdie 2016-05-19 14:52:28 +0000 We did tweak some things like the link used from the autobuilder so its no longer a full text search but just searching for specific revisions. Does that help your usecase or not? We're struggling to figure out the priority of this without knowing what usecases its impacting. If there is a significant impact we could make it a Medium+. We could also assign to someone on the GDC team, depending on how Michael and GDC want to handle this. 62326 2 anibal.limon 2016-05-19 14:57:04 +0000 (In reply to comment #1) > We did tweak some things like the link used from the autobuilder so its no > longer a full text search but just searching for specific revisions. Does > that help your usecase or not? We're struggling to figure out the priority > of this without knowing what usecases its impacting. If there is a > significant impact we could make it a Medium+. We could also assign to > someone on the GDC team, depending on how Michael and GDC want to handle > this. We can workaround the search over log using only the link from auto builder for access the failures also using specific revisions but this not solve the current searching problem if the search functionality isn't needed may be is better to remove it instead of leave as is and only using specific revisions and AB link. 62350 3 3182 anibal.limon 2016-05-19 20:11:58 +0000 Created attachment 3182 mysql slow query log When did a search of smart, in the current db. 62351 4 anibal.limon 2016-05-19 20:36:11 +0000 (In reply to comment #3) > Created attachment 3182 [details] > mysql slow query log > > When did a search of smart, in the current db. As summary the query took 440s, # Query_time: 440.112315 Lock_time: 0.000082 Rows_sent: 1 Rows_examined: 79488 62354 5 michael.g.wood 2016-05-19 21:57:49 +0000 Fine to take this bug, pretty easy to implement. Note that we already have a bug #6541 to improve the search on the UI side which may help where people haven't discovered out how to do the correct query. 66940 6 michael.g.wood 2016-10-05 14:53:42 +0000 We would need to special case when the backend database selection is mysql and create a full text index on the log field. We would also need to check which backend database is in use to be able to switch between the querying method. https://docs.djangoproject.com/en/1.8/ref/models/querysets/#search 92901 7 randy.macleod 2022-03-31 15:08:21 +0000 Still slow... 105227 8 tim.orling 2026-04-23 15:01:04 +0000 Still valid. 3182 2016-05-19 20:11:58 +0000 2016-05-19 20:11:58 +0000 mysql slow query log mariadb-slow.log text/plain 1470 anibal.limon L3Vzci9zYmluL215c3FsZCwgVmVyc2lvbjogMTAuMS4xNC1NYXJpYURCLTF+amVzc2llIChtYXJp YWRiLm9yZyBiaW5hcnkgZGlzdHJpYnV0aW9uKS4gc3RhcnRlZCB3aXRoOgpUY3AgcG9ydDogMzMw NiAgVW5peCBzb2NrZXQ6IC92YXIvcnVuL215c3FsZC9teXNxbGQuc29jawpUaW1lICAgICAgICAg ICAgICAgICBJZCBDb21tYW5kICAgIEFyZ3VtZW50CiMgVGltZTogMTYwNTE5IDE1OjA4OjQ5CiMg VXNlckBIb3N0OiByb290W3Jvb3RdIEAgbG9jYWxob3N0IFsxMjcuMC4wLjFdCiMgVGhyZWFkX2lk OiA2ICBTY2hlbWE6IGVycm9ycyAgUUNfaGl0OiBObwojIFF1ZXJ5X3RpbWU6IDQ0MC4xMTIzMTUg IExvY2tfdGltZTogMC4wMDAwODIgIFJvd3Nfc2VudDogMSAgUm93c19leGFtaW5lZDogNzk0ODgK IyBSb3dzX2FmZmVjdGVkOiAwCiMgRnVsbF9zY2FuOiBZZXMgIEZ1bGxfam9pbjogTm8gIFRtcF90 YWJsZTogTm8gIFRtcF90YWJsZV9vbl9kaXNrOiBObwojIEZpbGVzb3J0OiBObyAgRmlsZXNvcnRf b25fZGlzazogTm8gIE1lcmdlX3Bhc3NlczogMCAgUHJpb3JpdHlfcXVldWU6IE5vCiMKIyBleHBs YWluOiBpZAlzZWxlY3RfdHlwZQl0YWJsZQl0eXBlCXBvc3NpYmxlX2tleXMJa2V5CWtleV9sZW4J cmVmCXJvd3MJcl9yb3dzCWZpbHRlcmVkCXJfZmlsdGVyZWQJRXh0cmEKIyBleHBsYWluOiAxCVNJ TVBMRQlQb3N0X2J1aWxkCUFMTAlQUklNQVJZCU5VTEwJTlVMTAlOVUxMCTE1NzI5CTE1NzUxLjAw CTEwMC4wMAkxMDAuMDAJCiMgZXhwbGFpbjogMQlTSU1QTEUJUG9zdF9idWlsZGZhaWx1cmUJcmVm CVBvc3RfYnVpbGRmYWlsdXJlX2QyYjc5ZDFkCVBvc3RfYnVpbGRmYWlsdXJlX2QyYjc5ZDFkCTQJ ZXJyb3JzLlBvc3RfYnVpbGQuaWQJMQk0LjA1CTEwMC4wMAk1LjY1CVVzaW5nIHdoZXJlCiMKdXNl IGVycm9yczsKU0VUIHRpbWVzdGFtcD0xNDYzNjg4NTI5OwpTRUxFQ1QgQ09VTlQoKikgRlJPTSBg UG9zdF9idWlsZGZhaWx1cmVgIElOTkVSIEpPSU4gYFBvc3RfYnVpbGRgIE9OICggYFBvc3RfYnVp bGRmYWlsdXJlYC5gQlVJTERfaWRgID0gYFBvc3RfYnVpbGRgLmBpZGAgKSBXSEVSRSAoYFBvc3Rf YnVpbGRgLmBOQU1FYCBMSUtFICclc21hcnQlJyBPUiBgUG9zdF9idWlsZGAuYERJU1RST2AgTElL RSAnJXNtYXJ0JScgT1IgYFBvc3RfYnVpbGRgLmBNQUNISU5FYCBMSUtFICclc21hcnQlJyBPUiBg UG9zdF9idWlsZGZhaWx1cmVgLmBSRUNJUEVgIExJS0UgJyVzbWFydCUnIE9SIGBQb3N0X2J1aWxk YC5gTkFUSVZFTFNCU1RSSU5HYCBMSUtFICclc21hcnQlJyBPUiBgUG9zdF9idWlsZGAuYENPTU1J VGAgTElLRSAnJXNtYXJ0JScgT1IgYFBvc3RfYnVpbGRgLmBCUkFOQ0hgIExJS0UgJyVzbWFydCUn IE9SIGBQb3N0X2J1aWxkZmFpbHVyZWAuYFRBU0tgIExJS0UgJyVzbWFydCUnIE9SIGBQb3N0X2J1 aWxkZmFpbHVyZWAuYEVSUk9SX0RFVEFJTFNgIExJS0UgJyVzbWFydCUnKTsK