Bug 6137 - excessive load time for All Recipes page
Summary: excessive load time for All Recipes page
Status: VERIFIED FIXED
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: 1.6.1
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.7 M3
Assignee: David Reyna
QA Contact: Mihail Stanciu
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-04-09 04:59 UTC by David Reyna
Modified: 2015-08-07 14:27 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Reyna 2014-04-09 04:59:35 UTC
The All Recipes page has redundant foreign key table counts, each of which are very expensive and causes the page to load slowly.
Comment 1 David Reyna 2014-04-09 05:01:08 UTC
The simple solution from Dave Lerner is to use the "with" keyword to only calculate these values once per page, for example:

    {% with deps=recipe.r_dependencies_recipe.all %}
    {% with count=deps.count %}
    ...
Comment 2 David Reyna 2014-05-08 18:52:38 UTC
Refactoring:

- it loads the entirety of the Recipe Dependency table into memory; this doesn't scale with the number of builds
- it has whitespace problems
Comment 3 Alexandru Damian 2014-06-24 13:07:00 UTC
I have loading at 681ms on my machine. Can we set up target times ?
Comment 4 Alexandru Damian 2014-07-31 14:11:22 UTC
Need to understand if this is still a problem. Targeting M3.
Comment 5 Belen Barros Pena 2014-08-18 15:41:44 UTC
(In reply to comment #3)
> I have loading at 681ms on my machine. Can we set up target times ?

We can probably work out a target time by looking at the loading times in all other pages from Toaster 1.6. I'll ask QA to see if they can help with this.
Comment 6 Belen Barros Pena 2014-08-22 12:28:49 UTC
Mihail has run some load time tests for us (see below). David, Alex, is this enough to decide what to do with this bug?


**** 

Here’s the page load times I got after doing a bit of testing with wget:
 
recipes: ~1780 ms
tasks: ~1540ms
packages: ~1440 ms
time: ~1560ms
cpu: ~1550ms
disk I/O: ~1600ms
image info: ~4000ms
configuration: ~5000ms
 
These tests were done on a Fedora machine, 8 GB RAM, i7-2600 CPU @ 3.40GHz quad-core, tested on a core-image-sato image and most importantly, DB used was sqlite. I suspect this is why the times I got were significantly larger than AlexD’s example time.

Note that the first load test for all these pages were ignored, as they were significantly larger than the rest of the results (usually 2x to 3x larger) due to increased disk access time, I suspect.
 
Cheers,
Mihail
Comment 7 Belen Barros Pena 2014-09-08 09:25:22 UTC
After checking with David Reyna, he is happy to mark this bug as resolved - fixed based on this patch

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=bf9ff3de5f24ffabbf7093f0664730f22d31f191

and the load times gather by QA.
Comment 8 Mihail Stanciu 2014-09-08 10:51:40 UTC
Verified.