This bug is being opened as a new version of #4144, which was the previous umbrella bug for runtime perfomance. That bug is three years old and the bugs it blocked were not representing what was going to be done in the best of ways. The plan now is to generate testplan, testcases and specific metrics from #10500 benchmarks as a first point and later extend it as required based on new benchmarks added. The intent is to use phoronix for this.
Setting target for M3.
Marking this as INVALID for now. There are no benchmarks to base testcases or even a testplan on. Bug 10500 was aiming at getting these benchmarks, but it is not robust enough yet. The server is unstable and that can be tracked in bug 11046. To get runtime performance off the ground we need solid benchmarks from a good number of attributes, and that needs this server to work and be running reliable tests for a while. Once we have reliable benchmarks from this server or some other we can reopen and reevaluate runtime performance.