Bug 3622

Summary:	select preferred provider for multiple runtime providers
Product:	[Build System, Metadata & Runtime] BitBake	Reporter:	Martin Jansa <Martin.Jansa>
Component:	bitbake	Assignee:	Richard Purdie <richard.purdie>
Status:	RESOLVED NOTABUG	QA Contact:
Severity:	normal
Priority:	Medium	CC:	poky.bs.watcher, poky.watcher, sgw
Version:	unspecified
Target Milestone:	1.4
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	---

Description Martin Jansa 2012-12-19 13:17:09 UTC

I know that selecting runtime providers isn't well implemented in bitbake, but I've found interesting case:

When building world I always got:
ERROR: Multiple .bb files are due to be built which each provide udev (/OE/jansa-test/shr-core/meta-openembedded/meta-systemd/recipes-core/systemd/systemd_git.bb /OE/jansa-test/shr-core/openembedded-core/meta/recipes-core/udev/udev_182.bb).
 This usually means one provides something the other doesn't and should.

First part of fix was to define PACKAGES_DYNAMIC:
http://git.openembedded.org/meta-openembedded/commit/?id=39aabff8e0f9b3fa7c729dfc0593b3cea1762784
and then it worked after removing all udev recipes in layers.

python-pyudev has libudev in RDEPENDS
udev recipe has libudev explicitly in PACKAGES
systemd recipe splits all lib* packages dynamically:
  do_split_packages(d, systemdlibdir, '^lib(.*)\.so\.*', 'lib%s', 'Systemd %s
  PACKAGES_DYNAMIC += "^lib(udev|gudev|systemd).*"

But systemd is never selected as runtime provider and is not even listed in eligible providers, because of bitbake/lib/bb/providers.py:getRuntimeProviders optimization to search dynamic package only when other provider wasn't found yet (using PACKAGES and RPROVIDES).

If I remove that optimization I can select systemd as provider:
DEBUG: providers for libudev are: ['systemd', 'udev']
DEBUG: checking PREFERRED_PROVIDER_systemd (value None) against ['systemd', 'udev']
DEBUG: checking PREFERRED_PROVIDER_systemd-git (value None) against ['systemd', 'udev']
DEBUG: checking PREFERRED_PROVIDER_systemd-git-r20 (value None) against ['systemd', 'udev']
DEBUG: checking PREFERRED_PROVIDER_udev (value systemd) against ['systemd', 'udev']
NOTE: selecting systemd to satisfy runtime libudev due to PREFERRED_PROVIDER_udev = systemd
DEBUG: sorted providers for libudev are: ['/OE/jansa-test/shr-core/meta-openembedded/meta-systemd/recipes-core/systemd/systemd_git.bb', '/OE/jansa-test/shr-core/openembedded-core/meta/recipes-core/udev/udev_182.bb']
DEBUG: adding '/OE/jansa-test/shr-core/meta-openembedded/meta-systemd/recipes-core/systemd/systemd_git.bb' to satisfy runtime 'libudev'

Is there some other reason for not looking in dynamic packages other then optimization? It was introduced in first version of getProvidersRun:
commit 02a43ca809811ef6762c3db6a0b1f53b95d03966
Author: Holger Hans Peter Freyther <zecke@selfish.org>
Date:   Thu Jan 12 22:57:25 2006 +0000

I guess I can work around this by adding libudev to PACKAGES directly and remove it from PACKAGES_DYNAMIC. But at least I will use this report in such change to explain why it's needed.

Also with that optimization removed I can see few more notes about multiple providers:
NOTE: multiple providers are available for runtime gtk-theme-e17lookalike (gtk-theme-e17lookalike, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-e17lookalike
NOTE: multiple providers are available for runtime gtk-theme-sato (gtk-engines, gtk-sato-engine)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-sato
NOTE: multiple providers are available for runtime gtk-theme-clearlooks-gtkrc-dev (gtk-engines, gtk-theme-clearlooks-gtkrc)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-clearlooks-gtkrc-dev
NOTE: multiple providers are available for runtime gtk-theme-clearlooks-gtkrc (gtk-theme-clearlooks-gtkrc, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-clearlooks-gtkrc
NOTE: multiple providers are available for runtime libpostproc-dev (libav, libpostproc)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libpostproc-dev
NOTE: multiple providers are available for runtime libpostproc (libpostproc, libav)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libpostproc
NOTE: multiple providers are available for runtime gtk-theme-neo (gtk-theme-neo, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-neo
NOTE: multiple providers are available for runtime pam-plugin-ck-connector (consolekit, libpam)
NOTE: consider defining a PREFERRED_PROVIDER entry to match pam-plugin-ck-connector
NOTE: multiple providers are available for runtime gtk-theme-e17lookalike-dev (gtk-engines, gtk-theme-e17lookalike)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-e17lookalike-dev
NOTE: multiple providers are available for runtime gtk-theme-e17lookalike-gtkrc (gtk-engines, gtk-theme-e17lookalike)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-e17lookalike-gtkrc
NOTE: multiple providers are available for runtime gtk-theme-torturer-dev (gtk-engines, gtk-theme-torturer)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-torturer-dev
NOTE: multiple providers are available for runtime gtk-theme-torturer (gtk-theme-torturer, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-torturer
NOTE: multiple providers are available for runtime gtk-theme-neo-dev (gtk-theme-neo, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-neo-dev
NOTE: multiple providers are available for runtime gtk-theme-neo-gtkrc (gtk-theme-neo, gtk-engines)
NOTE: consider defining a PREFERRED_PROVIDER entry to match gtk-theme-neo-gtkrc

Comment 1 Richard Purdie 2013-01-21 14:27:29 UTC

Its generally thought that if you have two providers of something, they'd happen at equal levels, i.e. either both as packages_dynamic or both as PACKAGES.

The speed of packages dynamic processing is sadly much slower due to the regexps involved.

So I'd consider this more as an interesting corner case than a true error. I'd also note that for other reasons we need to put skipPackage or at the very least EXCLUDE_FROM_WORLD in the udev/systemd recipes to stop them being built in parallel for other reasons.