Bug 11429 - oe-selftest should use its own build directory and write the desired configuration
Summary: oe-selftest should use its own build directory and write the desired configur...
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: Scripts and Tools (show other bugs)
Version: 2.4
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 4.99
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on: 12385
Blocks: 11933 10033 11158 11612
  Show dependency tree
 
Reported: 2017-04-27 14:42 UTC by Joshua Lock
Modified: 2018-09-13 15:17 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
Complete AU log from an internal oe-selftest run (2.85 MB, application/octet-stream)
2017-12-05 15:38 UTC, Leonardo Sandoval Gonzalez
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Lock 2017-04-27 14:42:26 UTC
oe-selftest currently has some checks to ensure the configuration doesn't include values that have an adverse affect on the tests. This provides a jarring user experience, but we also do not want to have oe-selftest modify the user's configuration files itself.

The right way to ensure oe-selftest has the environment it requires is to use a separate build directory, just for use by oe-selftest, and write out the required configuration (whilst inheriting certain values such as DL_DIR and *MIRRORS* from the primary configuration).
Comment 1 Ross Burton 2017-04-27 14:55:56 UTC
And DISTRO, and MACHINE, and...

How do we draw the line between "user configuration which we ignore for tests" and "user configuration that is being tested"?
Comment 2 brian avery 2017-04-27 15:05:15 UTC
perhaps by pointing to s oe-selftest config that makes it explicit which variables we are excluding and including?
Comment 3 Aníbal Limón 2017-06-28 16:46:58 UTC
Already implemented but is part of oe-selftest threaded runs,

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/oe_selftest_threaded&id=97e3066889e16a79e95966d2009c747db974c6af
Comment 4 Aníbal Limón 2017-07-19 21:17:23 UTC
The thing here what will be the right configuration template to use?, the initial created by oe-init-build-env?
Comment 5 Joshua Lock 2017-07-25 13:54:58 UTC
Yes, I think it makes sense for the configuration used by oe-selftest to be a combination of the TEMPLATECONF values and the whitelisted variables (DL_DIR, SSTATE_CACHE, *MIRRORS) from the initiating environment.
Comment 6 Ross Burton 2017-07-25 14:21:34 UTC
What if I want to run selftest for all of the qemu machines?  Will MACHINE be whitelisted to?
Comment 7 Jair Gonzalez 2017-08-29 20:52:43 UTC
(In reply to comment #3)
> Already implemented but is part of oe-selftest threaded runs,
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/
> oe_selftest_threaded&id=97e3066889e16a79e95966d2009c747db974c6af

New commit id,

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/oe_selftest_threaded&id=a3fdad106080bdd5addc5c2bff34534ffb6db4cd
Comment 8 Leonardo Sandoval Gonzalez 2017-10-31 15:02:57 UTC
proposed patch sent to the ML

http://lists.openembedded.org/pipermail/openembedded-core/2017-October/143698.html
Comment 9 Leonardo Sandoval Gonzalez 2017-12-05 15:34:01 UTC
this work is in progress but unfortunately it is still giving some errors in the autobuilder [1]. Current work is at [2]


[1] complete log is attached
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-isolation
Comment 10 Leonardo Sandoval Gonzalez 2017-12-05 15:38:49 UTC
Created attachment 4142 [details]
Complete AU log from an internal oe-selftest run

see comment 9 for more information about this text.
Comment 11 Leonardo Sandoval Gonzalez 2017-12-06 16:18:26 UTC
the reason that this bug depends on bug 12401 is that the current approach to complete this enhancement basically starts metadata/scripts which is not git controlled, so it is the same scenario.
Comment 12 Leonardo Sandoval Gonzalez 2017-12-07 15:22:00 UTC
Last (failed) build:

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/666
Comment 13 Leonardo Sandoval Gonzalez 2017-12-11 18:21:07 UTC
Due to bug 12385 targeted for M2, moving this one also.
Comment 14 Leonardo Sandoval Gonzalez 2017-12-18 17:53:54 UTC
Second revision sent to the mailing list [1]

[1] http://lists.openembedded.org/pipermail/openembedded-core/2017-December/145743.html
Comment 15 Richard Purdie 2018-07-09 23:15:09 UTC
FWIW I'm now toying with the idea of http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=f65664c23e8fbefc821919c730399c5e099842a5 which puts the parallelism into unittest itself...
Comment 16 Richard Purdie 2018-09-13 15:17:44 UTC
Patches to parallelise oe-selftest merged for 2.6.