Bug 15569 - Add janitor support for handling failed builds
Summary: Add janitor support for handling failed builds
Status: NEW
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 6.1 M2
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2024-08-05 11:56 UTC by Richard Purdie
Modified: 2026-04-02 15:10 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Purdie 2024-08-05 11:56:57 UTC
I've changed the code in autobuilder2:

https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=d809e29a23d5614d8873c3f5f1fb7293da96cdd0

which increases the space limit from 100G to 200G. I also added
monitoring to /tmp since that caused a lot of build issues in a recent
failure.

This will help but I think we need to improve the janitor to clean up
failed builds. I'm thinking of something like a queuing system so any
build failure gets put into the queue and anything older gets deleted
as soon as the system is under disk pressure?

This would solve the issue that:

* builds the rerun on the same worker destroy failed data immediately
* older builds can build up and clog systems

It would also make it easy to put a policy in place:

* to immediately delete reproducible fails (for exmaple)
* to delete builds when space drops to a threshold

Help with this would be very welcome.

Ideally we'd have a way via a UI to mark failed builds as not wanted
but I'm not sure how to do right now.
Comment 1 Randy MacLeod 2024-08-08 14:33:51 UTC
Likely got to be done in 5.2...
Comment 2 Randy MacLeod 2025-01-30 16:06:38 UTC
Bulk move to 5.2-M3.
Comment 3 Randy MacLeod 2025-01-30 16:17:04 UTC
Bulk move to 5.2-M3 and this time I'll actually specify M3!
Comment 4 Randy MacLeod 2026-04-01 18:33:58 UTC
I doubt this has been done but leaving for bug review team to schedule.