Bug 12635

Summary: add performance test using phoronix test suite
Product: [QA/Testing] Test Plans/Suite Reporter: Anuj Mittal <anuj.mittal>
Component: test-plans/suiteAssignee: Yang Wang <yang.wang>
Status: RESOLVED WONTFIX QA Contact:
Severity: enhancement    
Priority: Medium+ CC: bluelightning, randy.macleod, stephano, tim.orling, yang.wang
Version: 5.99   
Target Milestone: 5.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Anuj Mittal 2018-03-29 23:09:51 UTC
Evaluate if a subset of phoronix tests can be run as ptests so we can have a set of performance numbers for every release.

We might want to start monitoring these to make sure that we are not regressing w.r.t performance for the default set of features enabled.
Comment 1 Paul Eggleton 2018-04-04 22:46:30 UTC
I guess you mean performance test rather than ptest? It would of course be done at runtime but I don't think the ptest framework is really where this fits.
Comment 2 Anuj Mittal 2018-04-05 05:25:14 UTC
Yes. I was thinking in terms of ptest because I thought it'd be nice if this had the attributes of a test, that is, mark it pass or fail every time we release depending on a baseline.
Comment 3 Richard Purdie 2018-04-05 08:29:50 UTC
I think it may make sense as an extension of ptest in that there are a set of tests and they return a value (a performance number) and rather than creating yet another mechanism, we may want to add this as an extension to the ptest usecase?
Comment 4 Randy MacLeod 2020-04-02 14:45:26 UTC
Ross has a prototype. We'd use parts of meta-oe. Yang are you interested in this work?
Comment 5 Yang Wang 2020-04-02 17:12:59 UTC
Hi Randy,

I would love to, but now there are about 20 tickets in my hand, let me close most of them and then come back to this one, is that okay?
Comment 6 Randy MacLeod 2020-04-02 17:31:14 UTC
Yang, sure. This story isn't even in the schedule (3.99 not 3.2) so there's no hurry. Assigned to you though, thanks!
Comment 7 Randy MacLeod 2022-04-07 15:09:58 UTC
Yang, Do you think you'll have time for this in the coming month or two?
Comment 8 Yang Wang 2022-04-07 15:28:03 UTC
Hi Randy,

I would say it can be difficult, but I will let you know whenever I have time.
Comment 9 Tim Orling 2022-04-07 15:53:37 UTC
I did some work on this a while back:
https://github.com/moto-timo/meta-openembedded/tree/timo/phoronix-8.2.0-isolated
Comment 10 Tim Orling 2022-04-07 16:16:20 UTC
As I recall, after the isolated fetching issue (during do fetch and not at run time), the next issue was that individual test cases have information about dependencies (DEPENDS/RDEPENDS that would be needed to build/run the test) and also arch info (e.g. some only run on x86).
Comment 11 Randy MacLeod 2023-10-24 14:24:34 UTC
Bulk move from 4.99 or 0.00 to 5.99
Comment 12 Randy MacLeod 2026-08-13 15:15:46 UTC
Phoronix may not be the best performance test suite, despite it's name.
This is not something that we plan to implement in oe-core but patches welcome.