Bug 15935 - swatbot: fix buildid clash between different buildbot instances
Summary: swatbot: fix buildid clash between different buildbot instances
Status: RESOLVED FIXED
Alias: None
Product: Infrastructure General
Classification: Infrastructure
Component: infrastructure-general (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.3 M3
Assignee: Michael Halstead
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2025-07-17 11:24 UTC by Mathieu Dubois-Briand
Modified: 2025-09-12 19:03 UTC (History)
1 user (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Dubois-Briand 2025-07-17 11:24:27 UTC
An issue had been found lately on swatbot, where the last reported builds show incoherent data: builds started in 2021 on the old autobuilder (typhoon) and with an end date in 2025.

After some investigations, it appeared the issue is builds are stored in swatbot using the buildbot buildid as an identifier, but this buildid is only unique across a single buildbot controller. As a consequence, if a build running on buildbot has the same buildid as an older build already stored in swatbot, the old data in swatbot will be partly replaced, leading to a corrupt build description.

Swatbot has been put offline as a preventive action, to avoid further corruption of the builds data. Now we must think about how swatbot can be fixed, so data of new builds are no longer merged with old builds.

Two possibilities arise so far:
- Adding a new field to swatbot build and buildcollection tables, allowing to store an identifier of the buildbot controller instance the builds come from. This will require to update all existing entries with the correct buildbot instance.
- Removing all entries coming from typhoon and only keep entries from valkyrie. We will probably still need to implement the additional field with controller identifier, to be safe if we ever introduce another buildbot controller, but this can be done with less urgency.

Once swatbot is fixed, missing builds will have to be imported: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14316
Comment 1 Michael Halstead 2025-07-17 22:52:34 UTC
I've moved swatbot to a new VM and updated to the newest django 4.2 LTS release.
I added 2,000,000,000,000 to all typhoon builds which avoid the collision problem for a few million years. It would be cleaner to modify the software to track cluster names but this works for now.If we ever add another cluster though we will get collisions in approximately 12,000 builds. 

I've re-enabled swatbot submissions on valkyrie.

We are ready to insert the missing data from between 2025-07-10 07:07:48 and now.
Comment 2 Michael Halstead 2025-09-12 19:03:54 UTC
Mathieu has restored the rest of the missing data.