Bug 10524

Summary: Command line build monitoring broken when using custom builddir and config file
Product: [Build System, Metadata & Runtime] Toaster Reporter: Joshua Lock <joshuagloe>
Component: toasterAssignee: David Reyna <david.reyna>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: major    
Priority: Medium CC: brian.avery, david.reyna, jessica.zhang, libertad.gonzalez.de.la.cruz, stanciux.mihail
Version: 2.2   
Target Milestone: 2.3 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
Some log output including several instances of the tracebacks none

Description Joshua Lock 2016-10-28 14:04:57 UTC
Created attachment 3512 [details]
Some log output including several instances of the tracebacks

I'm using Toaster in a custom build directory which is not a child of the poky repository, i.e:

. oe-init-build-env /srv/builds/foo

To support this I have a custom config file, settings_joshuagl.py (in POKYDIR/bitbake/lib/toaster/toastermain/) and I start Toaster with:

DJANGO_SETTINGS_MODULE=toastermain.settings_joshuagl TOASTER_DIR=/srv/builds/toaster source toaster start

I can drive Toaster via the web UI and start builds there, but when I try and start command line builds my terminal is flooded with tracebacks:

NOTE: ToasterUI waiting for events                                                                       ERROR: unable to open database file                                                                      Traceback (most recent call last):                                                                         File "/home/joshuagl/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection                                                                                            self.connect()                                                                                         File "/home/joshuagl/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 119, in connect                                                                                                      self.connection = self.get_new_connection(conn_params)
  File "/home/joshuagl/.local/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", line 204, in get_new_connection                                                                                        conn = Database.connect(**conn_params)
sqlite3.OperationalError: unable to open database file                                                                                                                                                            The above exception was the direct cause of the following exception:
                                                                                                         Traceback (most recent call last):                                                                         File "/home/joshuagl/Projects/poky/bitbake/lib/bb/ui/toasterui.py", line 283, in main
    buildinfohelper.store_log_event(event)                                                                 File "/home/joshuagl/Projects/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1569, in store_log_event    self._ensure_build()
  File "/home/joshuagl/Projects/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 931, in _ensure_build       self.orm_wrapper.get_or_create_build_object(self.brbe)                                                 File "/home/joshuagl/Projects/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 165, in get_or_create_build_object                                                                                                   prj = Project.objects.get_or_create_default_project()                                                  File "/home/joshuagl/Projects/poky/bitbake/lib/toaster/orm/models.py", line 164, in get_or_create_default_project                                                                                                   if len(projects) > 1:                                                                                  File "/home/joshuagl/.local/lib/python3.5/site-packages/django/db/models/query.py", line 144, in __len__
    self._fetch_all()
<snip>

There's a bunch of similar "unable to open database file" followed by "above exception was the direct cause of the following exception".

I can't replicate this error when using Toaster in a build directory which *is* a child of poky and without the custom configuration file.
Comment 1 Joshua Lock 2016-10-28 14:06:35 UTC
$ cat ~/Projects/poky/bitbake/lib/toaster/toastermain/settings_joshuagl.py
from toastermain.settings import *

DEBUG = True
TEMPLATE_DEBUG = DEBUG

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': '/srv/builds/toaster/toaster-db.sqlite',
        'TEST': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': '/srv/builds/toaster/toaster-db.sqlite',
        }
    }
}
Comment 2 brian avery 2016-11-02 15:02:13 UTC
duplicate.

*** This bug has been marked as a duplicate of bug 9992 ***
Comment 3 brian avery 2016-11-02 15:09:31 UTC
not really a duplicate.
Comment 4 Joshua Lock 2016-12-06 15:23:59 UTC
I don't have any cycles for this atm so resetting to New and default assignee so that it can be re-triaged.
Comment 5 David Reyna 2017-01-23 04:10:18 UTC
I am moving this to "Medium". While this is an important use case and will be resolved, is it not a core use case so I do not want to block on this on this milestone.
Comment 6 brian avery 2017-02-04 02:19:44 UTC
I think this is fixed on poky-contrib/bavery/toaster/9992-builddir-not-buildV4

with a couple of caveats.  To do what you outlined as the goal in the beginning you would NOT need to have a settings_joshuagl.py file.

You would just do the following:
/home/cow$ . ./poky/oe-init-build-env /srv/builds/foo
That script changes dir into the new bitbake builddir like so:
/srv/builds/foo$ . toaster start
then you just start toaster without any mods...
---
This means that cli builds happen in /srv/builds/foo.
The first toaster project will happen in 
/srv/builds/build-toaster-2
If you do the first toaster project against master, the master checkout will be in 
/srv/builds/_toaster_clones

If this is the desired behaviour, let's close this when/if the poky-contrib/bavery/toaster/9992-builddir-not-buildV4 changes get merged,
Comment 7 Joshua Lock 2017-02-06 16:40:47 UTC
(In reply to comment #6)
> I think this is fixed on
> poky-contrib/bavery/toaster/9992-builddir-not-buildV4

confirmed with:

[~/Projects/poky] $ . oe-init-build-env /srv/toaster
[/srv/toaster] $ . toaster start
[/srv/toaster] $ bitbake core-image-minimal

Thanks!
Comment 8 David Reyna 2017-03-28 20:42:10 UTC
Resolved as per Joshua.