15205 2023-09-04 10:48:33 +0000 Need to cleanup work directories for cancelled builds 2023-09-13 10:27:15 +0000 1 1 5 Infrastructure AutoBuilder autobuilder unspecified x86 Multiple RESOLVED FIXED Medium+ normal 4.3 M4 1 richard.purdie alex.kanavin alex.kanavin infras.ab.watcher Infras.watcher mhalstead pidge --- --- No (bug/feature does not impact docs) oldest_to_newest 96340 0 richard.purdie 2023-09-04 10:48:33 +0000 When builds are cancelled, large work directories are left behind and with the limited storage on our SSDs, this is problematic. we either need: a) to find a way to query buildbot, "is this build directory actively being used?" and then remove any build/build/ directory that isn't active -or- b) find a way to have a cancelled build cleanup hook that runs when a build is cancelled 96390 1 alex.kanavin 2023-09-08 17:24:51 +0000 This is where interrupting a build step is implemented: http://docs.buildbot.net/current/developer/cls-buildsteps.html#buildbot.process.buildstep.BuildStep.interrupt Next is to check how it's implemented, and whether it is extensible or sub-classable etc. 96395 2 alex.kanavin 2023-09-11 11:59:58 +0000 --- a/builders.py +++ b/builders.py @@ -128,11 +128,12 @@ def create_builder_factory(): f.addStep(RunConfigCheckSteps(posttrigger=False)) - # If the build was successful, clean up the build directory + # If the build was successful or cancelled, clean up the build directory f.addStep(steps.ShellCommand( command=[clob, util.Interpolate("%(prop:builddir)s/")], - doStepIf=lambda step: step.build.results == SUCCESS, + doStepIf=lambda step: step.build.results in (SUCCESS, CANCELLED), haltOnFailure=False, + alwaysRun=True, name="Clobber build dir")) return f This needs to be tested, as I'm not completely sure that alwaysRun will run the step also when the previous one was interrupted by the stop button. 96396 3 4975 alex.kanavin 2023-09-11 12:01:00 +0000 Created attachment 4975 proposed patch (untested) 96403 4 alex.kanavin 2023-09-13 10:27:15 +0000 The patch seems to be working: https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7793 https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=adcf9d6143451c6700aa1ca3763155b3bb986e47 4975 2023-09-11 12:01:00 +0000 2023-09-11 12:01:00 +0000 proposed patch (untested) 0001-builders.py-clobber-build-dir-also-when-the-build-wa.patch text/plain 1221 alex.kanavin RnJvbSA5ZTUxNzU4ZTQyYjRmOGIxZWNjZmI4MDkzMmE3NWQ1ODhkNWZmNGE5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBBbGV4YW5kZXIgS2FuYXZpbiA8YWxleC5rYW5hdmluQGdtYWls LmNvbT4KRGF0ZTogTW9uLCAxMSBTZXAgMjAyMyAxMzo1NDoyMCArMDIwMApTdWJqZWN0OiBbeW9j dG8tYXV0b2J1aWxkZXIyXVtQQVRDSF0gYnVpbGRlcnMucHk6IGNsb2JiZXIgYnVpbGQgZGlyIGFs c28gd2hlbgogdGhlIGJ1aWxkIHdhcyBjYW5jZWxsZWQKClRoaXMgZW5hYmxlcyBmYXN0ZXIgY2xl YW51cCBvZiBjYW5jZWxsZWQgYnVpbGRzLCByZWR1Y2luZyBjaGFuY2UKb2YgZGlzayBzcGFjZSBy dW5uaW5nIG91dC4KCltZT0NUTyAjMTUyMDVdCgpTaWduZWQtb2ZmLWJ5OiBBbGV4YW5kZXIgS2Fu YXZpbiA8YWxleC5rYW5hdmluQGdtYWlsLmNvbT4KLS0tCiBidWlsZGVycy5weSB8IDUgKysrLS0K IDEgZmlsZSBjaGFuZ2VkLCAzIGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCgpkaWZmIC0t Z2l0IGEvYnVpbGRlcnMucHkgYi9idWlsZGVycy5weQppbmRleCAzOGIyMDE4Li45ODA0NTliIDEw MDY0NAotLS0gYS9idWlsZGVycy5weQorKysgYi9idWlsZGVycy5weQpAQCAtMTI4LDExICsxMjgs MTIgQEAgZGVmIGNyZWF0ZV9idWlsZGVyX2ZhY3RvcnkoKToKIAogICAgIGYuYWRkU3RlcChSdW5D b25maWdDaGVja1N0ZXBzKHBvc3R0cmlnZ2VyPUZhbHNlKSkKIAotICAgICMgSWYgdGhlIGJ1aWxk IHdhcyBzdWNjZXNzZnVsLCBjbGVhbiB1cCB0aGUgYnVpbGQgZGlyZWN0b3J5CisgICAgIyBJZiB0 aGUgYnVpbGQgd2FzIHN1Y2Nlc3NmdWwgb3IgY2FuY2VsbGVkLCBjbGVhbiB1cCB0aGUgYnVpbGQg ZGlyZWN0b3J5CiAgICAgZi5hZGRTdGVwKHN0ZXBzLlNoZWxsQ29tbWFuZCgKICAgICAgICAgY29t bWFuZD1bY2xvYiwgdXRpbC5JbnRlcnBvbGF0ZSgiJShwcm9wOmJ1aWxkZGlyKXMvIildLAotICAg ICAgICBkb1N0ZXBJZj1sYW1iZGEgc3RlcDogc3RlcC5idWlsZC5yZXN1bHRzID09IFNVQ0NFU1Ms CisgICAgICAgIGRvU3RlcElmPWxhbWJkYSBzdGVwOiBzdGVwLmJ1aWxkLnJlc3VsdHMgaW4gKFNV Q0NFU1MsIENBTkNFTExFRCksCiAgICAgICAgIGhhbHRPbkZhaWx1cmU9RmFsc2UsCisgICAgICAg IGFsd2F5c1J1bj1UcnVlLAogICAgICAgICBuYW1lPSJDbG9iYmVyIGJ1aWxkIGRpciIpKQogCiAg ICAgcmV0dXJuIGYKLS0gCjIuNDEuMAoK