Bug 7527

Summary: Integrate Toaster with Jenkins
Product: [Build System, Metadata & Runtime] Toaster Reporter: Belen Barros Pena <belen.barros.pena>
Component: toasterAssignee: Unassigned <unassigned>
Status: RESOLVED OBSOLETE QA Contact:
Severity: enhancement    
Priority: Medium CC: alexandru.damian, belen.barros.pena, cristiana.voicu, daniel, david.reyna, jessica.zhang, joaohf, joshuagloe, markus.boos, msuraev, randy.macleod, tim.orling, wmills
Version: unspecified   
Target Milestone: Future   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Belen Barros Pena 2015-03-27 11:31:57 UTC
The informal interviews I carry out regularly with users have revealed Jenkins as the most used CI tool by far. It does make sense to start thinking of how to provide Toaster integration with Jenkins.
Comment 1 Alexandru Damian 2015-03-30 14:01:45 UTC
After summarely investigating Jenkins (installation, configuration for building poky, building and inspecting artifacts), I can propose three features that would provide degrees of integration between Toaster and Jenkins.

a). Toaster results uploader; Jenkins supports plugins for doing actions with the artifacts it creates during the build process; we could build a plugin that uploads toaster build logs to a Toaster instance; this is similar to what we currently support for Autobuilder.

b). Exporting/Importing project configurations between Toaster/Jenkins; we could provide a way to export a current Toaster project configuration (Project name, layer set, target set, and project configuration variables) and import this project as a Jenkins Freestyle project that would re-create the artifacts. 

c). Triggering a build for a certain project in Jenkins. Toaster could provide an user interface to trigger a build on a Jenkins remote instance.


Each option in part builds another layer of functionality, and using all three options we can build and provide a Jenkins build executor for Toaster; but I suspect that having modularized options is more valuable for prospective users as they can mix and match based on their needs.

If this makes sense, we need to open up new feature bugs for each of the options I described.
Comment 2 Belen Barros Pena 2015-03-31 15:59:18 UTC
A comment from the community (pasting here to keep track of things):

On 31/03/2015 10:39, "Anders Darander" wrote:

If there's some project configuration that needs to be done in Toaster,
I'd appreciate if as much as possible could be retrieved from our
current Jenkins setup (i.e. exported from the Jenkins plugin to
Toaster). (Not least as we're running a couple of scripts to source the
environment and handling some extra configurations needed; thus I'd
assume we couldn't just take the freestyle Jenkins project talked about
in b) of Alexandru 's first comment in #YOCTO-7527.
Comment 3 Belen Barros Pena 2015-04-09 12:42:22 UTC
Some other comments / requests:

* Show Jenkins job name and build number in Toaster. Those could be retrieved from the environment variables BUILD_NUMBER and JOB_NAME in the Jenkins build.

* Use the above to link back to Jenkins jobs and builds from Toaster
Comment 4 Belen Barros Pena 2015-04-30 12:41:45 UTC
There was some discussion during the Toaster weekly call on April 29th. We identified 2 integration types:

1. Show data coming from Jenkins builds using the Toaster analysis mode

2. Send configuration for builds that will run in Jenkins and receive build artifacts using the Toaster build mode

Although the initial goal of the release was doing number 2, we have settled for the following: we will provide type 1 integration for both Autobuilder and Jenkins, plus the ability to download build artifacts (logs, images and SDKs). We will aim for a generic solution for the latter: an API people can use to pass the artifacts location to Toaster. 

We will continue to explore the problem of the type 2 integration, to come up with a good solution that we might be able to implement in a future release. 

Alex, feel free to split this enhancement if you need to.
Comment 5 Michael Halstead 2015-08-12 15:58:09 UTC
Clearing 2.0 milestone
Comment 6 David Reyna 2018-10-04 13:37:08 UTC
Reviewed 10/18
Comment 7 Tim Orling 2020-06-25 08:18:30 UTC
One approach might be using the CROPS toaster-container
Comment 8 Randy MacLeod 2022-06-30 15:13:35 UTC
Jenkins isn't as popular as it once was. -- YP bug team