11717 2017-06-24 05:21:04 +0000 large package set causes 'too many SQL variables' error 2017-07-12 15:28:10 +0000 1 1 7 Build System, Metadata & Runtime Toaster toaster 2.4 x86 Multiple RESOLVED FIXED Medium+ normal 2.4 M2 1 david.reyna david.reyna jessica.zhang stanciux.mihail libertad.gonzalez.de.la.cruz --- --- No (bug/feature does not impact docs) oldest_to_newest 74534 0 david.reyna 2017-06-24 05:21:04 +0000 If you build a project with a large package set, you will get a crash in "views.py" when the dashboard attempts to fetch the package set to calculate the package count and size. This is a sqlite limitation, and it fails with as few as 1220 packages. To reproduce, build the "intelx86-64" machine using the intel-x86 layer, and then try to bring up the build's dashboard page. 74535 1 david.reyna 2017-06-24 05:22:17 +0000 The solution is to break the query into parts. diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py index 2efb0fd5..39014379 100755 --- a/lib/toaster/toastergui/views.py +++ b/lib/toaster/toastergui/views.py @@ -483,10 +483,15 @@ def builddashboard( request, build_id ): npkg = 0 pkgsz = 0 package = None - for package in Package.objects.filter(id__in = [x.package_id for x in t.target_installed_package_set.all()]): - pkgsz = pkgsz + package.size - if package.installed_name: - npkg = npkg + 1 + # Chunk the query to avoid "too many SQL variables" error + package_set = t.target_installed_package_set.all() + package_set_len = len(package_set) + for ps_start in range(0,package_set_len,500): + ps_stop = min(ps_start+500,package_set_len) + for package in Package.objects.filter(id__in = [x.package_id for x in package_set[ps_start:ps_stop]]): + pkgsz = pkgsz + package.size + if package.installed_name: + npkg = npkg + 1 elem['npkg'] = npkg elem['pkgsz'] = pkgsz 74960 2 david.reyna 2017-07-12 15:28:10 +0000 commit eea5cb317137d71ad0cab954d2c9ad2069580b6f