Bug 10931

Summary:	Improve indicators on automated performance reports
Product:	[QA/Testing] Build Testing	Reporter:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Component:	general	Assignee:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	benjamin.esquivel, markus.lehtonen, richard.purdie
Version:	2.3
Target Milestone:	2.4 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Leonardo Sandoval Gonzalez 2017-01-16 22:08:21 UTC

Currently, the QA creates plots at milestone rate [1] containing some stats from profiling data (attached). This is fine but it may be too late to detect performance degradation [for example, see 2]. The proposed task consists on parsing the profiling results and from it compute the mean values for each profiling column, which is actually what [1] is doing but this time, the mean values will be included in the email report sent several times per day. 

[1] https://wiki.yoctoproject.org/charts/perf_milestone_GDC/bitbake-core-image-sato-rootfs.html
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=10884

Comment 1 Markus Lehtonen 2017-03-22 17:04:11 UTC

I think I can take this. I've already implemented a quick prototype for this:
http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/scripts

The prototype contains a scripts for creating an (interactive) HTML report, and converting that into an html-formatted email. There is a long list of streamlining/enhancements to do, the most critical being support for XML formatted test reports (it currently only supports json).

Comment 2 Markus Lehtonen 2017-08-08 15:15:08 UTC

I would propose that we close this bug as fixed. Graphical performance reports are being automatically uploaded to wiki.yoctoproject.org after each test run so the latest data is always available (see [1]).

Assigning back to Leonardo for comments.

[1] https://wiki.yoctoproject.org/wiki/Performance_Test#Daily_Test_Results

Comment 3 Leonardo Sandoval Gonzalez 2017-08-16 19:35:30 UTC

Work completed by Markus L.