Bug 6319

Summary:	Automated storage and graphing of build performance metrics
Product:	[QA/Testing] Build Testing	Reporter:	Richard Purdie <richard.purdie>
Component:	general	Assignee:	Markus Lehtonen <markus.lehtonen>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	Medium+	CC:	belen.barros.pena, bill.randle, joshuagloe, konrad.scherer, markus.lehtonen, randy.macleod, sgw
Version:	unspecified
Target Milestone:	2.4 M1
Hardware:	x86
OS:	Multiple
Whiteboard:	TRT
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)
Bug Depends on:	10582
Bug Blocks:	5049

Description Richard Purdie 2014-05-13 13:17:51 UTC

Currently, we do have various build performance numbers being run on a regular basis and we do graph those numbers for milestones.

I'd like to:

a) Collect this data together in a database somewhere rather than having an ever growing csv file
b) Have some kind of visualisation of those numbers automatically generated like the bug trend charts.

Comment 1 Richard Purdie 2014-05-13 13:18:49 UTC

Adding Belen as I know there may be some overlap between some functionality of toaster and this.

Comment 2 Alexandru Georgescu 2015-05-04 13:00:04 UTC

This will be integrated in out QA REporting TOol

Comment 3 Benjamin Esquivel 2016-11-29 16:56:03 UTC

removing dependency to bug 9958 as it has been set to future and this bug is targeted to a specific release.

Comment 4 Markus Lehtonen 2017-02-01 15:17:51 UTC

Moving to milestone to 2.4 as the feature would be part of the TestReportTool.

I have implemented an initial prototype web service, based on an older CustomReports project. The code is currently available in GitHub (as it is a fork from CustomReports), see build-perf branch:
https://github.com/marquiz/CustomReports/tree/build-perf

Comment 5 Markus Lehtonen 2017-04-11 12:24:23 UTC

I'm setting the milestone now to 2.4 M1. However, I would be inclined to close the bug as fixed because test results are being automatically stored in Git and graphical test reports are being automatically sent from the testers. 

I'd like to see more fine-grained requirements for TRT when its design/implementation starts.

Comment 6 Markus Lehtonen 2017-05-23 07:49:58 UTC

Build performance test data is being automatically stored in Git:
http://git.yoctoproject.org/cgit/cgit.cgi/yp-qa-build-perf-data/

Also, build performance test reports are now being automatically generated and uploaded to wiki.yoctoproject.org for each test run. Links to the latest test reports are found here:
https://wiki.yoctoproject.org/wiki/Performance_Test#Daily_Test_Results

Closing this bug.