11508 2017-05-12 16:17:15 +0000 Bugs in MultiBranch GitPoller example code and limitation in buildbot library 2018-03-22 08:49:21 +0000 1 1 5 Infrastructure AutoBuilder autobuilder unspecified x86 Multiple RESOLVED OBSOLETE Medium normal 2.5 1 johannes.schrimpf joshuagloe infras.ab.watcher Infras.watcher pidge pokylinux --- --- Yes (doc changes required) oldest_to_newest 73239 0 3763 johannes.schrimpf 2017-05-12 16:17:15 +0000 Created attachment 3763 Patch that includes branch names in GitPoller name to make it possible to use several pollers for same repo with different branches There are several bugs in the code in example of MultiBranch GitPoller in the documentation and a related limitation in the buildbot library that is used: 1) In both schedulers there is a '}' after 3600 in the line for the interval Solution for 1: Remove the '}'s After doing this, autobuilder managed to start but in Changesources in the webinterface only one GitPoller is visible. 2) The second scheduler seems to overwrite the first one. I could not verify with same repository and different branches but I could verify it happens with different repositories) Solution for 2: Add the two schedulers to the same scheduler list scheduler:[{....}, {....}] That leads to the following problem: 3) The two pollers are not allowed to have the same name Solution for 3: Change the names for the two schedulers to be unique Doing these three steps, I was able to add scheduler with multiple different repositories with a similar buildset config, but still did not get two GitPollers to work on the same repository. This is a problem in gitpoller.py. The problem is the following: - Two GitPollers with the same repository but different branches will get the same name which is a problem for twisted. To prevent this, gitpoller.py has to be changed. The applies patch is a quick fix that appends the list of branches as string in the name of the service. It works in my local setup, together with my suggested patch in ID 11507. Autobuilder version: commit 867b093d540fcd51d821adcdd8184668ab23b40e 73812 1 joshuagloe 2017-06-07 15:29:01 +0000 As this bug is in buildbot core, have you considered submitting your fix to the upstream buildbot project? https://github.com/buildbot/buildbot Over the course of the next development cycle or so I'd really like to move to a newer version of buildbot and ideally no longer carry a fork of the code in the yocto-autobuilder repository. 73933 2 johannes.schrimpf 2017-06-09 08:32:25 +0000 I had a brief look at the upstream repo, but since autobuilder uses a fork in an older version, I did not look more into it. Also, the use of gitpoller in autobuilder is quite specific and traceable and I don't know how this change would impact other software using buildbot. I don't have enough insight in the mainline buildbot setup to know how gitpoller is intended to be used outside autobuilder, and I could imagine this specific change could potentially break compatibility with other software. I think moving to a mainline buildbot version would be very good and I guess then this issue should be addressed then in regard to the new version. If still present, it should definitively be reported to the buildbot developers, but I suggest to apply the patch until then to make autobuilder work in this setup. 79927 3 joshuagloe 2018-03-22 08:49:21 +0000 Resolved by the new infrastructure architecture of helper scripts (yocto-autobuilder-helper) with a more recent buildbot with minimal configuration (yocto-autobuilder2). 3763 2017-05-12 16:17:15 +0000 2017-05-12 16:17:15 +0000 Patch that includes branch names in GitPoller name to make it possible to use several pollers for same repo with different branches 0001-lib-python2.7-site-packages-buildbot-0.8.8-py2.7.egg.patch application/octet-stream 1979 johannes.schrimpf RnJvbSA1ZjdlMjU0MzUxYTdhMzMzODJmNGM2MmQ3YTBmYjdjYjRlOWM3NjQ0IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKb2hhbm5lcyBTY2hyaW1wZiA8am9oYW5uZXMuc2NocmltcGZA Ymx1ZXllLm5vPgpEYXRlOiBGcmksIDEyIE1heSAyMDE3IDE4OjAyOjE1ICswMjAwClN1YmplY3Q6 IFtQQVRDSF0gCiBsaWIvcHl0aG9uMi43L3NpdGUtcGFja2FnZXMvYnVpbGRib3QtMC44LjgtcHky LjcuZWdnL2J1aWxkYm90L2NoYW5nZXMvZ2l0cG9sbGVyLnB5OgogQWxsb3cgdHdvIGdpdHBvbGxl cnMgd2l0aCBzYW1lIHJlcG9zaXRvcnkgYW5kIGRpZmZlcmVudCBicmFuY2hlcwoKVHdvIEdpdFBv bGxlcnMgd2l0aCB0aGUgc2FtZSByZXBvc2l0b3J5IGJ1dCBkaWZmZXJlbnQgYnJhbmNoZXMgd2ls bCBnZXQgdGhlIHNhbWUgbmFtZSB3aGljaCBpcyBhIHByb2JsZW0gZm9yIHR3aXN0ZWQuIFRvIHBy ZXZlbnQgdGhpcywgdGhlIGxpc3Qgb2YgYnJhbmNoZXMgaXMgYXBwZW5kZWQgYXMgc3RyaW5nIGlu IHRoZSBuYW1lIG9mIHRoZSBzZXJ2aWNlLgoKU2lnbmVkLW9mZi1ieTogSm9oYW5uZXMgU2Nocmlt cGYgPGpvaGFubmVzLnNjaHJpbXBmQGJsdWV5ZS5ubz4KLS0tCiAuLi4vYnVpbGRib3QtMC44Ljgt cHkyLjcuZWdnL2J1aWxkYm90L2NoYW5nZXMvZ2l0cG9sbGVyLnB5ICAgICB8IDEwICsrKysrLS0t LS0KIDEgZmlsZSBjaGFuZ2VkLCA1IGluc2VydGlvbnMoKyksIDUgZGVsZXRpb25zKC0pCgpkaWZm IC0tZ2l0IGEvbGliL3B5dGhvbjIuNy9zaXRlLXBhY2thZ2VzL2J1aWxkYm90LTAuOC44LXB5Mi43 LmVnZy9idWlsZGJvdC9jaGFuZ2VzL2dpdHBvbGxlci5weSBiL2xpYi9weXRob24yLjcvc2l0ZS1w YWNrYWdlcy9idWlsZGJvdC0wLjguOC1weTIuNy5lZ2cvYnVpbGRib3QvY2hhbmdlcy9naXRwb2xs ZXIucHkKaW5kZXggMTIwMjUzYS4uNmJhYjk1YSAxMDA2NDQKLS0tIGEvbGliL3B5dGhvbjIuNy9z aXRlLXBhY2thZ2VzL2J1aWxkYm90LTAuOC44LXB5Mi43LmVnZy9idWlsZGJvdC9jaGFuZ2VzL2dp dHBvbGxlci5weQorKysgYi9saWIvcHl0aG9uMi43L3NpdGUtcGFja2FnZXMvYnVpbGRib3QtMC44 LjgtcHkyLjcuZWdnL2J1aWxkYm90L2NoYW5nZXMvZ2l0cG9sbGVyLnB5CkBAIC00MiwxMSArNDIs NiBAQCBjbGFzcyBHaXRQb2xsZXIoYmFzZS5Qb2xsaW5nQ2hhbmdlU291cmNlLCBTdGF0ZU1peGlu KToKICAgICAgICAgaWYgcG9sbGludGVydmFsICE9IC0yOgogICAgICAgICAgICAgcG9sbEludGVy dmFsID0gcG9sbGludGVydmFsCiAKLSAgICAgICAgYmFzZS5Qb2xsaW5nQ2hhbmdlU291cmNlLl9f aW5pdF9fKHNlbGYsIG5hbWU9cmVwb3VybCwKLSAgICAgICAgICAgICAgICBwb2xsSW50ZXJ2YWw9 cG9sbEludGVydmFsKQotCi0gICAgICAgIGlmIHByb2plY3QgaXMgTm9uZTogcHJvamVjdCA9ICcn Ci0KICAgICAgICAgaWYgYnJhbmNoIGFuZCBicmFuY2hlczoKICAgICAgICAgICAgIGNvbmZpZy5l cnJvcigiR2l0UG9sbGVyOiBjYW4ndCBzcGVjaWZ5IGJvdGggYnJhbmNoIGFuZCBicmFuY2hlcyIp CiAgICAgICAgIGVsaWYgYnJhbmNoOgpAQCAtNTQsNiArNDksMTEgQEAgY2xhc3MgR2l0UG9sbGVy KGJhc2UuUG9sbGluZ0NoYW5nZVNvdXJjZSwgU3RhdGVNaXhpbik6CiAgICAgICAgIGVsaWYgbm90 IGJyYW5jaGVzOgogICAgICAgICAgICAgYnJhbmNoZXMgPSBbJ21hc3RlciddCiAKKyAgICAgICAg YmFzZS5Qb2xsaW5nQ2hhbmdlU291cmNlLl9faW5pdF9fKHNlbGYsIG5hbWU9cmVwb3VybCtzdHIo YnJhbmNoZXMpLAorICAgICAgICAgICAgICAgIHBvbGxJbnRlcnZhbD1wb2xsSW50ZXJ2YWwpCisK KyAgICAgICAgaWYgcHJvamVjdCBpcyBOb25lOiBwcm9qZWN0ID0gJycKKwogICAgICAgICBzZWxm LnJlcG91cmwgPSByZXBvdXJsCiAgICAgICAgIHNlbGYuYnJhbmNoZXMgPSBicmFuY2hlcwogICAg ICAgICBzZWxmLmVuY29kaW5nID0gZW5jb2RpbmcKLS0gCjIuNy40Cgo=