Bug 10951

Summary:	Runtime Performance testing framework
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Humberto Ibarra <humberto.ibarra.lopez>
Component:	core	Assignee:	Humberto Ibarra <humberto.ibarra.lopez>
Status:	RESOLVED INVALID	QA Contact:
Severity:	enhancement
Priority:	Medium+	CC:	jose.perez.carranza, meta.mr.watcher, meta.watcher, randy.macleod
Version:	2.3
Target Milestone:	2.3 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know
Bug Depends on:	4527
Bug Blocks:

Description Humberto Ibarra 2017-01-22 12:50:33 UTC

This bug is being opened as a new version of #4144, which was the previous umbrella bug for runtime perfomance. That bug is three years old and the bugs it blocked were not representing what was going to be done in the best of ways.

The plan now is to generate testplan, testcases and specific metrics from #10500 benchmarks as a first point and later extend it as required based on new benchmarks added. The intent is to use phoronix for this.

Comment 1 Humberto Ibarra 2017-01-25 20:18:53 UTC

Setting target for M3.

Comment 2 Humberto Ibarra 2017-02-15 21:56:50 UTC

Marking this as INVALID for now. There are no benchmarks to base testcases or even a testplan on.

Bug 10500 was aiming at getting these benchmarks, but it is not robust enough yet. The server is unstable and that can be tracked in bug 11046.

To get runtime performance off the ground we need solid benchmarks from a good number of attributes, and that needs this server to work and be running reliable tests for a while.

Once we have reliable benchmarks from this server or some other we can reopen and reevaluate runtime performance.