Bug 11239 - native recipes should not care about DISTRO_FEATURES / MACHINE_FEATURES
Summary: native recipes should not care about DISTRO_FEATURES / MACHINE_FEATURES
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.3 M4
Assignee: Jussi Kukkonen
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-03-24 14:03 UTC by Jussi Kukkonen
Modified: 2017-04-13 16:12 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jussi Kukkonen 2017-03-24 14:03:14 UTC
Currently a change in DISTRO_FEATURES means we end up rebuilding a bunch native recipes. So far I have not found a single feature where this would be useful: maybe we can just define a constant set of features that will always be used for -native?

I'm looking at this next week, filing now as reminder.

For background, here's a list of distro features we refer to in oe-core (possibly not 100% exhaustive). Most of the libc- ones should probably be removed: none of our libcs respect those.

---
3g
acl
alsa
api-documentation
argp
bluetooth
bluez5
bssplt
cramfs
directfb
efi
ext2
gobject-introspection-data
ipsec
ipv4
ipv6
irda
largefile
ldconfig
ld-is-gold
libc-backtrace
libc-big-macros
libc-bsd
libc-catgets
libc-charsets
libc-crypt
libc-crypt-ufc
libc-cxx-tests
libc-db-aliases
libc-envz
libc-fcvt
libc-fmtmsg
libc-fstab
libc-ftraverse
libc-getlogin
libc-idn
libc-inet
libc-inet-anl
libc-libm
libc-locale-code
libc-locales
libc-memusage
libc-nis
libc-nsswitch
libc-posix-clang-wchar
libc-posix-regexp
libc-posix-regexp-glibc
libc-posix-wchar-io
libc-rcmd
libc-rtld-debug
libc-spawn
libc-streams
libc-sunrpc
libc-utmp
libc-utmpx
libc-wordexp
mplt
multiarch
nfc
nfs
nls
opengl
pam
pci
pcmcia
ppp
ptest
pulseaudio
selinux
smbfs
systemd
sysvinit
usbgadget
usbhost
wayland
wifi
x11
xattr
zeroconf
Comment 1 Richard Purdie 2017-03-27 14:05:46 UTC
I think the main one we might care about in the native case is largefile which we'd always want turn on and we're likely deprecating anyway (in favour of always enabled).

I can't see much of a case for not fixing the native list of features. I also agree that the libc ones are pretty much dead now.
Comment 2 Jussi Kukkonen 2017-04-05 08:23:50 UTC
This really stretches my understanding of bitbake variables... documenting some details:

My initial approach is just to modify DISTRO_FEATURES (and if needed other variables) in native.bbclass. This turned out to be tricky:

* DISTRO_FEATURES = "<features we want for native>" 
This will be overriden by at least DISTRO_FEATURES_append in any configuration as well as DISTRO_FEATURES_BACKFILL. This makes sense to me.

* DISTRO_FEATURES_forcevariable = "<features we want for native>"
Same results -- appends and backfill still get added to the list. I don't really understand why.

* anonymous python with d.setVar("DISTRO_FEATURES", "<features we want for native>")
This is so ugly but does ensure DISTRO_FEATURES is correct when tasks run. Unfortunately there are other anonymous python functions that do d.getVar("DISTRO_FEATURES") and end up getting the wrong value and modifying the build based on that.
Comment 3 Jussi Kukkonen 2017-04-05 17:20:17 UTC
On Richards suggestion I tried d.SetVar() in native_virtclass_handler(): this seems to work. My test case was:
 $ bitbake core-image-minimal
 # append " systemd" to DISTRO_FEATURES
 $ bitbake core-image-minimal

With a native_virtclass_handler patch the second build had 135 tasks less to do. Sadly this doesn't show up in wallclock time: all the native tasks are done while glibc is still bottlenecking everything else. I'm sure there is a effect on cpu-time, I just didn't take note of it.
Comment 4 Jussi Kukkonen 2017-04-05 20:03:42 UTC
I redid the tests with timing:
 $ bitbake core-image-minimal
 # append " systemd" to DISTRO_FEATURES
 $ bitbake core-image-minimal


With master the second build was:
real    15m23.533s
user    135m19.244s
sys     16m55.816s

After patching:
real    14m48.574s
user    94m31.776s
sys     13m54.560s


user+sys difference is -44 minutes or -28% so definitely worth it.
Comment 5 Ross Burton 2017-04-13 16:05:38 UTC
oe-core 731744d5538e315702be828e6f2bd556309dee07 solves this for DISTRO_FEATURES.

oe-core 96c20c9df714cdf3f0e9461ec566c4f5d3bdb5f1 solves the native case for MACHINE_FEATURES.
Comment 6 Ross Burton 2017-04-13 16:12:13 UTC
I *think* we're done with this. Closing!