This is a candidate task for build performance area. Note we could close this issue as invalid if team doesn't like this idea. Problem Statement: Now the perf test scripts are a part of poky git. This layout potentially invalidate a bisect procedure when the section of git history contains changes to the perf test scripts themselves. We need to use same tool revisions to do benchmark against different commit tips in the source code project so that the data acquired are comparable. Now I have to manipulate the tree or copy the perf test out of tree to achieve this goal. These add another change for human errors. And actually such changes are not noticeable in a bisecting test. Proposed Solution: I suggest we put perf test scripts themselves into another git repository to address this concern.
Assigning this to myself. You have a valid point and it make sense to split it out. Any suggestions where to move it? I wouldn't necessarily create a separate layer just for build perf tests.
Whilst I see the usecase for this, I also see various problems. My personal instinct is not to do this, we have other more important things to deal with.
After sleeping over it I also start to think it might have more problems than upsides. First of all, it would be required to make it self-contained, we couldn't rely on any utilities/library functions in oe-core. Nevertheless, I still see the bisect usecase.
I don't want to argue for urgency point. Making perf self-contained is what I believe (Thanks for Markus to make it clear) too. I have said team can close this if you guys don't like this idea.
Refer to our previous comment.