With the introduction of Django-3, there is a new database commit race condition. At the end of short builds (quilt-native, re-build of a previous build), the behavior is the same as before. However for long builds, the event handler for "bb.event.BuildCompleted" sets the build record's status to 'success', but that change does not make it to the actual database, resulting in a 'finished' build visually stuck at 100%. The solution is to introduce this code: def update_build_stats_and_outcome(self, build, errors, warnings, taskfailures): ... # We force a sync point here to force the outcome status commit, # which resolves a race condition with the build completion takedown transaction.set_autocommit(True) transaction.set_autocommit(False) The problem is set up by the fact that Toaster has SQLite has autocommit normally turned off so that Toaster can wait to catch up with the flood of events and avoid a database lock. But in this case, if the commit queue is too long as per a full build, the build worker threads could be killed before the queue is finish and lose late commits like this one.
Resolved. Commit: c71a9c89e3c23ba1f1eec06c0183ee1d7148041d