Bug 6319 - Automated storage and graphing of build performance metrics
Summary: Automated storage and graphing of build performance metrics
Status: RESOLVED FIXED
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 2.4 M1
Assignee: Markus Lehtonen
QA Contact:
URL:
Whiteboard: TRT
Depends on: 10582
Blocks: 5049
  Show dependency tree
 
Reported: 2014-05-13 13:17 UTC by Richard Purdie
Modified: 2017-12-06 16:48 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Purdie 2014-05-13 13:17:51 UTC
Currently, we do have various build performance numbers being run on a regular basis and we do graph those numbers for milestones.

I'd like to:

a) Collect this data together in a database somewhere rather than having an ever growing csv file
b) Have some kind of visualisation of those numbers automatically generated like the bug trend charts.
Comment 1 Richard Purdie 2014-05-13 13:18:49 UTC
Adding Belen as I know there may be some overlap between some functionality of toaster and this.
Comment 2 Alexandru Georgescu 2015-05-04 13:00:04 UTC
This will be integrated in out QA REporting TOol
Comment 3 Benjamin Esquivel 2016-11-29 16:56:03 UTC
removing dependency to bug 9958 as it has been set to future and this bug is targeted to a specific release.
Comment 4 Markus Lehtonen 2017-02-01 15:17:51 UTC
Moving to milestone to 2.4 as the feature would be part of the TestReportTool.

I have implemented an initial prototype web service, based on an older CustomReports project. The code is currently available in GitHub (as it is a fork from CustomReports), see build-perf branch:
https://github.com/marquiz/CustomReports/tree/build-perf
Comment 5 Markus Lehtonen 2017-04-11 12:24:23 UTC
I'm setting the milestone now to 2.4 M1. However, I would be inclined to close the bug as fixed because test results are being automatically stored in Git and graphical test reports are being automatically sent from the testers. 

I'd like to see more fine-grained requirements for TRT when its design/implementation starts.
Comment 6 Markus Lehtonen 2017-05-23 07:49:58 UTC
Build performance test data is being automatically stored in Git:
http://git.yoctoproject.org/cgit/cgit.cgi/yp-qa-build-perf-data/

Also, build performance test reports are now being automatically generated and uploaded to wiki.yoctoproject.org for each test run. Links to the latest test reports are found here:
https://wiki.yoctoproject.org/wiki/Performance_Test#Daily_Test_Results

Closing this bug.