Bug 11794

Summary: Add remote HTTP API to enable status aggregation of multiple Toaster instances
Product: [Build System, Metadata & Runtime] Toaster Reporter: Konrad Scherer <konrad.scherer>
Component: toasterAssignee: David Reyna <david.reyna>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: enhancement    
Priority: Medium CC: jessica.zhang, stanciux.mihail
Version: 2.4   
Target Milestone: 2.4 M3   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Konrad Scherer 2017-07-14 18:26:34 UTC 
I am creating a very basic Toaster Aggregation webapp for a build cluster of potentially 100+ machines. The idea is that each build will be running Toaster and the webapp will show a list of all currently running builds with progress information and a direct link to each individual toaster instance.

I suggest the following endpoints:

/health: Returns 200 if Toaster instance is healthy. This will help remove inactive Toaster instances from the webapp quickly.

/builds: Returns json with a list of names of currently running builds

/build/<name>: Returns json with build status for the build with <name>. This should include at minimum the current progress but could include any other relevant status information. If the status information is limited, this endpoint could be merged with the /builds endpoint.
Comment 1 David Reyna 2017-07-14 18:45:50 UTC 
The patterns "/builds" and "build/<name>" are currently already used for the normal full GUI pages, so I will propose alternate names.

FYI, the Toaster URL patterns are encoded here:
    bitbake/lib/toaster/toastergui/urls.py
Comment 2 David Reyna 2017-07-14 19:21:17 UTC 
The counter proposal is:
 /api/builds
 /api/build/<name>

This makes sense in that it follows the Layer Index model for the back end data.
Comment 3 David Reyna 2017-07-18 22:29:30 UTC 
Here are the example API calls and output, with the returned JSON data fanned out and sorted for clarity.

1) "localhost:8000/toastergui/api/building"

{"count": 1, 
 "building": [
   {"id": 22, 
    "name": "20170718220837", 
    "project": "foo", 
    "target": "core-image-base", 
    "distro": "poky", 
    "machine": "qemux86", 
    "start": "2017-07-18T22:07:48.083Z", 
    "seconds": "32.783", 
    "clone": "1:1"}
    "parse": "800:803", 
    "task": "0:0", 
    }]
}

2) "localhost:8000/toastergui/api/builds"

{"count": 1,
 "builds": [
   {"id": 21, 
    "outcome": "Succeeded", 
    "name": "20170630232654", 
    "project": "Command line builds", 
    "target": "core-image-base", 
    "distro": "poky"
    "machine": "qemux86", 
    "start": "2017-06-30T23:26:47.767Z", 
    "stop": "2017-06-30T23:35:32.753Z", 
    "seconds": "524.986", 
    "errors": 0, 
    "warnings": 0, 
   }]
}

3) "localhost:8000/health"

"Ok"

Source:
  <!DOCTYPE html>
  <html lang="en">
    <head><title>Toaster Health</title></head>
    <body>Ok</body>
  </html>
Comment 4 David Reyna 2017-08-01 02:01:37 UTC 
commit 3bc3d26b465b5af3cebc1d4c0d1fa382965c32cf