Bug 13649

Summary: Create autobuilder-worker container
Product: [Yocto Project Subprojects] CROPS Reporter: Tim Orling <tim.orling>
Component: crops-defaultAssignee: Unassigned <unassigned>
Status: NEW --- QA Contact: Toaster default assignee <toaster>
Severity: enhancement    
Priority: Medium CC: konrad.scherer, randy.macleod, tvgamblin
Version: 3.1   
Target Milestone: 5.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Tim Orling 2019-11-22 21:12:24 UTC
With YP LTS initiative, we may get into a time when the AutoBuilder itself can no longer support an older release (e.g. the automated testing infrastructure has changed, etc.).

It might also increase usage of the AutoBuilder if it was easier to setup/try.

This container would be a blend of the poky-container [1,2] and the buildbot-worker [3,4] container. The autobuilder-master container might be able to be simpler, but this worker container will need all the bits to build and run tests. Testing may benefit from running QEMU outside the container using libvirt or other means.

Although it would be tempting to run Docker-in-Docker, this is ill advised [5] and instead the docker socket should be bind mounted:
'docker run -v /var/run/docker.sock:/var/run/docker.sock ...'

Similarly, the persistent build directories should be bind mounted docker volumes (local driver). 

[1] https://github.com/crops/poky-container
[2] https://hub.docker.com/r/crops/poky
[3] https://github.com/buildbot/buildbot/blob/master/worker/Dockerfile.py3
[4] https://hub.docker.com/r/buildbot/buildbot-worker
[5] https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
Comment 1 Tim Orling 2020-03-03 02:58:42 UTC
Very raw command line experiment, ignoring yocto-autobuilder-helper for now. Assumes buildbot-controller is already running. See https://bugzilla.yoctoproject.org/show_bug.cgi?id=13648

On debian-10 host:

$ sudo apt install jq
$ docker inspect autobuilder-controller | jq .[0] | jq -r .NetworkSettings.Networks.autobuilder.IPAddress
172.19.0.2
$ docker run -it -e BUILDMASTER=172.19.0.2 -e BUILDMASTER_PORT=9989 -e WORKERNAME=ubuntu1804-ty-1 -e WORKERPASS=pass -e WORKER_ENVIRONMENT_BLACKLIST=WORKERPASS --network autobuilder --name ubuntu1804-ty-1 buildbot/buildbot-worker:v2.7.0
Comment 2 Tim Orling 2022-04-07 16:47:15 UTC
Nice idea, but not something I have time to work on.
Comment 3 Randy MacLeod 2023-10-24 14:24:24 UTC
Bulk move from 4.99 or 0.00 to 5.99
Comment 4 Tim Orling 2026-06-25 15:22:10 UTC
Ideas here are probably still valid for future design considerations. Moving to future.