Bug 4144

Summary:	Runtime performance testing
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Jason Wessel <jason.wessel>
Component:	core	Assignee:	Humberto Ibarra <humberto.ibarra.lopez>
Status:	RESOLVED OBSOLETE	QA Contact:
Severity:	enhancement
Priority:	Medium+	CC:	bogdanx.a.voiculescu, leonardo.sandoval.gonzalez, meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie
Version:	unspecified
Target Milestone:	2.3 M2
Hardware:	All
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know
Bug Depends on:	5369, 5477, 5478, 6009, 6296
Bug Blocks:

Description Jason Wessel 2013-03-28 18:41:34 UTC

This maybe something that has to move to the meta-yocto layer or infrastructure, but the idea here is that there should be some type of runtime performance testing that happens when builds are successful and also to be able to compare one version of the Yocto Project to the next.

The standard bench marks would be integrated into oe-core or meta-yocto and then run against the booted target system and metrics collected.  The possibilities for benchmark suites include:

lmbench
hackbench
iozone3

As a strech goal start to integrate some or all of the phronix test suite:
http://www.phoronix-test-suite.com/?k=downloads

Comment 1 Alexandru Georgescu 2013-09-19 13:37:51 UTC

This definetly will be future. Maybe 1.6

Comment 2 Alexandru Georgescu 2013-11-04 10:37:58 UTC

Hi Jason,
Do we want the Phoronix Test Suite integrated? Or would be better to include only the benchmarks from it?

Comment 3 Alexandru Georgescu 2014-04-24 11:48:38 UTC

Promoting this to a master enhancement bug that will track ovarll runtime performance testing. This includes:
- bug 4527 - that will cover PTS integration or  tests from PTS in a new layer
- bug 6009 - Toaster backend performance testing
- bug 5478 - Covers both build time performance when using gcc security flags
- bug 5477 - Check for any runtime performance degradations with gcc security flags enabled
- bug 5369 -  performance analysis of x32 runtime

Comment 4 Alexandru Georgescu 2014-05-08 15:01:22 UTC

- bug 6296 - Add support for runtime performance tests to image_tests

Comment 5 Humberto Ibarra 2017-01-22 12:56:48 UTC

The approach for this bug and its blocking bugs is four years old and not the one intended right now. Opened bug #10951 as a new version for this and will track the implementation with that one. Marking this as RESOLVED OBSOLETE