Bug 9021

Summary: Toaster merges builds started from its web UI with builds started from CLI
Product: [Build System, Metadata & Runtime] Toaster Reporter: Belen Barros Pena <belen.barros.pena>
Component: toasterAssignee: Elliot Smith <elliot.smith>
Status: VERIFIED FIXED QA Contact: Mihail Stanciu <stanciux.mihail>
Severity: normal    
Priority: Medium+ CC: belen.barros.pena, elliot.smith, jessica.zhang, stanciux.mihail
Version: 2.0   
Target Milestone: 2.1 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
weird-build.png none

Description Belen Barros Pena 2016-02-01 15:49:40 UTC
Created attachment 2956 [details]
weird-build.png

Steps to reproduce:

I used a brand new sqlite database and removed the /tmp directory for good measure. Then I did the following:

1. Create a project in Toaster
2. Build something from that project (e.g. zlib)
3. Then switch to the command line and build something (e.g. dropbear)

Instead of showing 2 builds, one for zlib inside the project you created in step 1, and a second build for dropbear inside the 'Command line builds' project, Toaster will show a single build with targets zlib and dropbear inside the project you created in step 1.

See the screenshot attached to appreciate the weirdness :)
Comment 1 Elliot Smith 2016-02-02 11:13:21 UTC
This happens if you start a command-line build while Toaster is still processing the Toaster-triggered build.

buildinfohelper makes the assumption that only one build can be happening at a time, and holds a reference to a "current" build. All events which occur while the "current" build is open are attached to that build.

However, even though a Toaster-triggered build may look "done", Toaster may still have database records to process and will hold the build object open until that's finished. In the meantime, a second build on the command line could be started, and its events get erroneously attached to the "current" build.

We need to do a better job of detecting when the build is really done, and ensure that we attach events to the correct build as they occur. This could involve quite a lot of work, as we may have to have multiple build objects open at the same time in buildinfohelper, and a way to decide which of those builds an incoming event belongs to.

I hypothesise that if you leave a decent gap between the end of the Toaster-triggered build and the command-line one, this shouldn't happen. If it does still happen under these circumstances, then the problem is even worse, as it means we're not closing builds correctly even when they're completely done and all the database operations for incoming events have been processed.
Comment 2 Elliot Smith 2016-02-02 11:15:22 UTC
Again, I want to wait for the patches to fix bug 8440 to be merged before tackling this, as I may need to do some refactoring in buildinfohelper.
Comment 3 Elliot Smith 2016-02-02 11:23:10 UTC
It looks like it may be worse than I thought initially, as I still get the two builds (Toaster-triggered and cli) merging with each other, even when there's a good gap between them.
Comment 4 Elliot Smith 2016-02-02 11:34:00 UTC
After more testing, I have found that if I run cli builds, they are picked up and logged correctly by Toaster. If I then run Toaster-triggered builds, they are also picked up correctly. But if I then go back to cli builds (after the Toaster-triggered one), they get attached to the previous Toaster-triggered build...
Comment 5 Belen Barros Pena 2016-02-02 11:43:51 UTC
(In reply to comment #4)
> After more testing, I have found that if I run cli builds, they are picked
> up and logged correctly by Toaster. If I then run Toaster-triggered builds,
> they are also picked up correctly. But if I then go back to cli builds
> (after the Toaster-triggered one), they get attached to the previous
> Toaster-triggered build...

Yes, this is exactly the behaviour I see as well.
Comment 6 Elliot Smith 2016-02-02 12:05:26 UTC
It occurred to me that this might have something to do with BRBE being set, or not being set. I think Ed made some changes so that the TOASTER_BRBE variable is set on the bitbake server; this is then looked up in buildinfohelper to decide whether a BuildStarted event represents a new build or an old one, according to whether BRBE is set.

If we are not unsetting TOASTER_BRBE at the end of a Toaster-triggered build, this might explain why command-line builds are attached to the last Toaster-triggered build.
Comment 7 Elliot Smith 2016-02-02 14:50:16 UTC
I have a fix for this, but it depends on the patches for bug 8440:
elliot/toaster/mixed_builds-9021

I'll submit the patch for review once those patches are in toaster-next.
Comment 8 Elliot Smith 2016-02-03 13:04:17 UTC
Removed dependency, as I reworked the branch so it no longer depends on 8440.

Submitted to toaster mailing list for review:
https://lists.yoctoproject.org/pipermail/toaster/2016-February/003775.html
Comment 9 Elliot Smith 2016-02-23 09:36:43 UTC
Merged into master: 1f7f07746629a55e4efd5f505e84eb6c66567115
Comment 10 Mihail Stanciu 2016-02-23 12:47:24 UTC
Verified on 2.1_M2_rc3: 152914f2983c5d69001de1d46ce99547fa1e75fe