Bug 4527

Summary:	create a Runtime performance test layer to enable runtime performance tests
Product:	[QA/Testing] Runtime Testing	Reporter:	Stoicescu Cornel <corneliux.stoicescu>
Component:	general	Assignee:	Humberto Ibarra <humberto.ibarra.lopez>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	Medium+	CC:	alexandru.c.georgescu, leonardo.sandoval.gonzalez, markus.lehtonen, sgw, todor.minchev
Version:	1.5
Target Milestone:	2.3 M2
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know
Bug Depends on:
Bug Blocks:	10951

Description Stoicescu Cornel 2013-05-22 12:00:46 UTC

We should use the Phoronix test Suite to benchmark Yocto images with common Linux distributions.

http://www.phoronix-test-suite.com

This is the proposed action plan:
1) Select individual tests or test suites (like Timed Code Compilation) that would be of interest. The tests dependencies should be bared in mind when making this selection.  
2) Select the images -hardware and type- (ex: core-image-sato-sdk-atom-pc) that would be tested.


Requirements:
* Tests must be run on the same hardware using the selected Yocto images and latest/LTS common Linux distributions like Ubuntu, Fedora, OpenSUSE and CentOS;

* There should be dedicated machines to run these tests. All distros should be installed on the same HDD using partitions to avoid unwanted errors in the results;

* Keeping in mind the above requirements, multiple hardware can be used in order to test different architectures;

Comment 1 Alexandru Georgescu 2013-09-19 13:54:01 UTC

Moving to 1.6

Comment 2 Alexandru Georgescu 2014-04-23 08:09:46 UTC

still in progress.

Comment 3 Ionut Chisanovici 2014-08-27 09:35:26 UTC

Moved to 1.7

Comment 4 Alexandru Georgescu 2016-02-08 13:50:46 UTC

See lmbench recipe as reference: http://layers.openembedded.org/layerindex/recipe/869/

Comment 5 Humberto Ibarra 2016-12-08 21:31:28 UTC

Have tracked the required recipes. Implementing the layer.

Comment 6 Humberto Ibarra 2016-12-09 20:35:19 UTC

Phoronix might be a bit costly to deploy on certain targets. Either way, not really sure if the extra layer is needed in the first place. Phoronix, lmbench, iozone3 and others are already part of oe-core or meta-oe; a new layer wouldn't really bring anything new into the fold.

Check recipes-extended under oe-core and recipes-benchmark under meta-oe. Runtime performance test could add these recipes directly from meta-oe or meta instead of creating a new layer for it.

Comment 7 Humberto Ibarra 2016-12-12 21:31:31 UTC

There is a related bug that might be implementing this already, bug is 10500. Sent mail about this and waiting for answers.

Comment 8 Humberto Ibarra 2017-01-22 13:16:19 UTC

All other runtime performance bugs were marked obsolete as they didn't reflect the current implementation in the best of ways, however, this one is still valid since we are going to need this layer. The layer will probably be just a skeleton and will be initially added to contrib.

Removing old dependencies to obsolete bugs and adding a 10951 as its umbrella bug.

Comment 9 Humberto Ibarra 2017-01-25 04:42:01 UTC

Added a skeleton layer for this in:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=humberto/runtime-performance

This layer only contains a dependency to the meta-oe layer, which has the phoronix recipe and is going to be used to run performance testcases. More about phoronix settings can be found at the README file of the layer.

Testcases should be implemented for this in future bugs attached to #10951