Bug 11429

Summary: oe-selftest should use its own build directory and write the desired configuration
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Joshua Lock <joshuagloe>
Component: Scripts and ToolsAssignee: Unassigned <unassigned>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: brian.avery, jair.de.jesus.gonzalez.plascencia, juan.m.cruz.alcaraz, richard.purdie, ross.burton
Version: 2.4   
Target Milestone: 4.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on: 12385    
Bug Blocks: 11933, 10033, 11158, 11612    
Attachments:
Description Flags
Complete AU log from an internal oe-selftest run none

Description Joshua Lock 2017-04-27 14:42:26 UTC
oe-selftest currently has some checks to ensure the configuration doesn't include values that have an adverse affect on the tests. This provides a jarring user experience, but we also do not want to have oe-selftest modify the user's configuration files itself.

The right way to ensure oe-selftest has the environment it requires is to use a separate build directory, just for use by oe-selftest, and write out the required configuration (whilst inheriting certain values such as DL_DIR and *MIRRORS* from the primary configuration).
Comment 1 Ross Burton 2017-04-27 14:55:56 UTC
And DISTRO, and MACHINE, and...

How do we draw the line between "user configuration which we ignore for tests" and "user configuration that is being tested"?
Comment 2 brian avery 2017-04-27 15:05:15 UTC
perhaps by pointing to s oe-selftest config that makes it explicit which variables we are excluding and including?
Comment 3 Aníbal Limón 2017-06-28 16:46:58 UTC
Already implemented but is part of oe-selftest threaded runs,

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/oe_selftest_threaded&id=97e3066889e16a79e95966d2009c747db974c6af
Comment 4 Aníbal Limón 2017-07-19 21:17:23 UTC
The thing here what will be the right configuration template to use?, the initial created by oe-init-build-env?
Comment 5 Joshua Lock 2017-07-25 13:54:58 UTC
Yes, I think it makes sense for the configuration used by oe-selftest to be a combination of the TEMPLATECONF values and the whitelisted variables (DL_DIR, SSTATE_CACHE, *MIRRORS) from the initiating environment.
Comment 6 Ross Burton 2017-07-25 14:21:34 UTC
What if I want to run selftest for all of the qemu machines?  Will MACHINE be whitelisted to?
Comment 7 Jair Gonzalez 2017-08-29 20:52:43 UTC
(In reply to comment #3)
> Already implemented but is part of oe-selftest threaded runs,
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/
> oe_selftest_threaded&id=97e3066889e16a79e95966d2009c747db974c6af

New commit id,

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/oe_selftest_threaded&id=a3fdad106080bdd5addc5c2bff34534ffb6db4cd
Comment 8 Leonardo Sandoval Gonzalez 2017-10-31 15:02:57 UTC
proposed patch sent to the ML

http://lists.openembedded.org/pipermail/openembedded-core/2017-October/143698.html
Comment 9 Leonardo Sandoval Gonzalez 2017-12-05 15:34:01 UTC
this work is in progress but unfortunately it is still giving some errors in the autobuilder [1]. Current work is at [2]


[1] complete log is attached
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-isolation
Comment 10 Leonardo Sandoval Gonzalez 2017-12-05 15:38:49 UTC
Created attachment 4142 [details]
Complete AU log from an internal oe-selftest run

see comment 9 for more information about this text.
Comment 11 Leonardo Sandoval Gonzalez 2017-12-06 16:18:26 UTC
the reason that this bug depends on bug 12401 is that the current approach to complete this enhancement basically starts metadata/scripts which is not git controlled, so it is the same scenario.
Comment 12 Leonardo Sandoval Gonzalez 2017-12-07 15:22:00 UTC
Last (failed) build:

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/666
Comment 13 Leonardo Sandoval Gonzalez 2017-12-11 18:21:07 UTC
Due to bug 12385 targeted for M2, moving this one also.
Comment 14 Leonardo Sandoval Gonzalez 2017-12-18 17:53:54 UTC
Second revision sent to the mailing list [1]

[1] http://lists.openembedded.org/pipermail/openembedded-core/2017-December/145743.html
Comment 15 Richard Purdie 2018-07-09 23:15:09 UTC
FWIW I'm now toying with the idea of http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=f65664c23e8fbefc821919c730399c5e099842a5 which puts the parallelism into unittest itself...
Comment 16 Richard Purdie 2018-09-13 15:17:44 UTC
Patches to parallelise oe-selftest merged for 2.6.