The proposal is to have a set of instructions to deploy an AB cluster via provisioning tools like Puppet, Ansible, etc. One option is to leave workers fixed in number and distro or it could be parametric over a set of supported distros. This work includes creating reference images for the controller and the workers as well as the instructions for deployment. The goal of this task is to facilitate AB replication.
I manage the internal WindRiver Linux build cluster and a lot of my work overlaps with the AB. I have built up Puppet code for provisioning bare-metal builders that does things like setup ntp, install host packages, install monitoring, etc. I also have Dockerfiles and docker images which are used to build WRLinux. Using Docker in this way has increased our host coverage a lot while reducing the administration overhead by giving configuring all the builders in the same way. I use Mesos and a custom scheduler to automate the builds and share workloads efficiently across the entire cluster of machines. I use Docker to create a local test setup of the production environment for development and testing. It is important to have a system that scales up and down easily. I was hesitant to share this earlier because it was in early stages, but the system has matured and has been solid for over a year now. The cluster does over 2000 builds a day over 5 releases and 1500 configurations. It still has rough edges as infrastructure projects tend to have, but I would be happy to share any and all pieces of my work that you feel would be useful.
Konrad, you seem to be very close to what we're looking with this entry. The success of this task is measured by having short and concise instructions to replicate a simple AB setup. If you are a couple of steps ahead with this and you have time to write instructions for replicating it, would be very much appreciated. We would test them.
Should be much simpler with the new model of using vanilla upstream buildbot with a repo containing configuration and extensions.
Fixed by the bitbake-setup work.
(In reply to Randy MacLeod from comment #4) > Fixed by the bitbake-setup work. This comment was on the wrong bug. This bug is actually closed by this docker-compose configuration: https://git.yoctoproject.org/yocto-autobuilder2/tree/docker/README.md?id=4a7eacd4ff4231989902ff39840cf0b8a95e2958