Bug 9217

It would be helpful to have the logs of both these sets of warnings included in the bug (an attachment) rather than just stating the number of errors...

Right now its difficult to know if there is some underlying issue here or what the patterns may me. Its hard to know if the build-deps and file-rdeps issues are related or not.

Would you please add how to reproduce it please ?

I don't care anymore.

A year later I've found some time to work around first check:

1) e.g.
WARNING: eglinfo-fb-1.0.0-r0 do_package_qa: QA Issue: /usr/bin/eglinfo-fb contained in package eglinfo-fb requires libmali.so, but no providers found in RDEPENDS_eglinfo-fb? [file-rdeps]

eglinfo-fb has dependency on virtual/egl and actual virtual/egl provider also provides libmali.so, but it's not found by insane QA check.

work around:
RPROVIDES_${PN} = "libmali.so libwayland-egl.so libGLESv2.so libEGL.so"
in the recipe providing the virtual/*gl* for you and then explicit RDEPENDS in every recipe linking with one of these, e.g.:

VIRTUAL-RUNTIME_gpu-libs ?= ""
RDEPENDS_${PN} += "${VIRTUAL-RUNTIME_gpu-libs}"

and your DISTRO configuration sets VIRTUAL-RUNTIME_gpu-libs accordingly, it's not nice, but better than thousands warnings in every builds.

2) this was a bit trickier, but after hiding thousands of warnings from 1), I've noticed that build-deps warning about missing bash and stat dependencies was shown only in some builds and on some builders I wasn't able to reproduce it at all.

First I've of course assumed that it's shown only sometimes due to the recipes with issues being reused from sstate. But then I wasn't able to reproduce the QA warning even when doing local builds after cleansstate.

The test case is busybox bbappend with:
RPROVIDES_${PN} += "stat"
RPROVIDES_${PN} += "bash /bin/bash"

and e.g. libtool which  explicitly RDEPENDS_${PN} += "bash" and has bbappend with:
VIRTUAL-RUNTIME_bash ?= "bash"
RDEPENDS_${PN}_append_class-target = " ${VIRTUAL-RUNTIME_bash}"

To resolve the file-rdeps issue from 1). And the DISTRO sets:
VIRTUAL-RUNTIME_bash = "busybox"
it also uses u-a to make sure that busybox's bash is used when both get installed in the "development" image and check for GPLv3 for the "production" images to make sure that real bash doesn't get installed accidentally.

This lead me to interesting discovery in insane.bbclass:
Here in this for loop:
http://git.openembedded.org/openembedded-core/tree/meta/classes/insane.bbclass?id=7a0ababb9f5c7f09bf072412e0744dcf69534129#n880

On the builder where I was seeing the QA warning, I got bash rdep_data on first call of "rdep_data = oe.packagedata.read_subpkgdata(rdepend, d)" on
http://git.openembedded.org/openembedded-core/tree/meta/classes/insane.bbclass?id=7a0ababb9f5c7f09bf072412e0744dcf69534129#n881

and on the builder which wasn't showing the QA warning i got empty rdep_data and then busybox was found as possible provider in the section bellow:
http://git.openembedded.org/openembedded-core/tree/meta/classes/insane.bbclass?id=7a0ababb9f5c7f09bf072412e0744dcf69534129#n887

So basically when bash was built before libtool (for whatever reason), insane.bbclass found bash recipe as first bash provider and complained that bash isn't in the dependencies.

When bash wasn't built before libtool, then bash provider wasn't found with oe.packagedata.read_subpkgdata, so it started searching for other possible providers and found busybox which was in the dependencies and warning wasn't shown.

I'm not sure what's best fix for this:

A) I can PNBLACKLIST bash to make sure it never gets built for given DISTRO (and to make sure that all package-managers will respect my selection to use busybox as preferred provider for bash).

B) I can remove explicit bash dependency in all recipes where I'm adding explicit 
VIRTUAL-RUNTIME_bash (preferably by using such variable in original recipes, but I don't expect warm acceptance of such changes for every bash, stat and whatever else you can RPROVIDE with busybox with proper defconfig).

C) I can change insane.bbclass to search os.listdir("%s/runtime-rprovides/%s/" % (pkgdata_dir, rdepend)) even when the provider was already found with oe.packagedata.read_subpkgdata and complain only when none of the providers is in dependency tree

D) combination of all above, that I'll search provider in both runtime-rprovides and read_subpkgdata, but will show different kind of warning basically saying that there are multiple providers and you need to make sure that your package manager will select the right provider in some deterministic way - so that you need to implement A) or B).

I guess the main issue is that this warning is shown only sometimes (only for recipes where do_package_qa is executed after the alternative provider) and with all the determinism from RSS and HOSTTOOLS this was a bit surprising (until finding the root cause which explains why it's like that).

work around for bash runtime dependencies sent to ML:
http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142237.html

The work around I've shared before works fine for us.

It's used e.g. with Yocto 2.2 in meta-webosose: https://github.com/webosose/meta-webosose
https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-core/busybox/busybox_1.24.1.bbappend
https://github.com/webosose/meta-webosose/blob/master/meta-webos/recipes-devtools/libtool/libtool_%25.bbappend

But for some other builds we started to use multilib and now
RPROVIDES_${PN} += "bash /bin/bash"
gets expanded as:
RPROVIDES_${PN} += "lib32-bash lib32-/bin/bash"
and the QA errors are back.

I haven't found a way to prevent ${LIB32_PREFIX} being added by map_depends_variable even for /bin/bash:

# $RPROVIDES [4 operations]
#   set oe-core/meta/conf/bitbake.conf:272
#     ""
#   set oe-core/meta/conf/documentation.conf:355
#     [doc] "A list of package name aliases that a package also provides. These aliases are useful for satisfying runtime dependencies of other packages both during the build and on the target."
#   override[lib32-busybox]:rename from RPROVIDES_${PN} data.py:116 [expandKeys]
#     " ${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', '/bin/sh', '', d)} stat bash /bin/bash"
#   override[lib32-busybox]:set classextend.py:86 [map_depends_variable]
#     "lib32-/bin/bash lib32-bash lib32-stat"
# pre-expansion value:
#   ""
RPROVIDES=""
#
# $RPROVIDES_lib32-busybox [2 operations]
#   rename from RPROVIDES_${PN} data.py:116 [expandKeys]
#     " ${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', '/bin/sh', '', d)} stat bash /bin/bash"
#   set classextend.py:86 [map_depends_variable]
#     "lib32-/bin/bash lib32-bash lib32-stat"
# pre-expansion value:
#   "lib32-/bin/bash lib32-bash lib32-stat"
RPROVIDES_lib32-busybox="lib32-/bin/bash lib32-bash lib32-stat"

I've tried to prevent it with;
A) RDEPENDS_${LIB32_PREFIX}${BPN}
   but the original RDEPENDS_${PN} replaces it
B) append with virtclass-multilib-lib32
   but it's ignored even when virtclass-multilib-lib32 is shown in OVERRIDES
C) anonymous python to replace lib32-/bin/bash with /bin/bash
   is executed too early before map_depends_variable

after discussing this with Alex Kanavin on IRC he suggested to repeat the exact steps how to reproduce this in oe-core in single comment so that some better solution can be found instead of this work around

To reproduce this apply:
http://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/YOCTO-9217&id=91b432dc3a183e6b2cbe825d3eec69302a61ea5f
in our oe-core or poky and build e.g. libtool (anything with RDEPENDS_${PN} = "bash" will trigger this):

bitbake libtool

ERROR: libtool-2.4.6-r0 do_package_qa: QA Issue: libtool rdepends on bash, but it isn't a build dependency, missing bash in DEPENDS or PACKAGECONFIG? [build-deps]
ERROR: libtool-2.4.6-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: libtool-2.4.6-r0 do_package_qa: Function failed: do_package_qa
ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/libtool/2.4.6-r0/temp/log.do_package_qa.642
ERROR: Task (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/libtool/libtool_2.4.6.bb:do_package_qa) failed with exit code '1'

http://errors.yoctoproject.org/Errors/Details/181381/

from bitbake -e libtool and bitbake -e busybox we can see:

docker-shr qemux86-64@ ~/build/oe-core $ grep ^RPROVIDES env.busybox
RPROVIDES=""
RPROVIDES_busybox="  bash stat"
docker-shr qemux86-64@ ~/build/oe-core $ grep ^RDEPENDS env.libtool
RDEPENDS=""
RDEPENDS_${KERNEL_PACKAGE_NAME}-base=""
RDEPENDS_libtool=" bash"
RDEPENDS_libtool-dev="libtool (= 2.4.6-r0)"
RDEPENDS_libtool-staticdev="libtool-dev (= 2.4.6-r0)"

If I apply the VIRTUAL-RUNTIME_bash work around for libtool as shown in:
http://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/YOCTO-9217&id=9ee0d23d43b4add9d381048ea8499bb250f7e34e

Then the build-deps QA isn't shown anymore, but there might be file-rdeps issue, I've mentioned before, but I currently cannot reproduce the file-rdeps one.

In oe-core build I'm seeing /bin/bash in
pkgdata/qemux86-64/runtime/busybox:FILERPROVIDES_/bin/busybox.nosuid_busybox:
but in internal build with multilib it's not there at all, I'm checking why.

A patch was merged in oe-core 01b9810ff1cea53ae321f6fbe760b83573ed78c0. Is this sufficient to say this is resolved?

01b9810ff1cea53ae321f6fbe760b83573ed78c0 is enough to work around most of these issues, it's not nice to have so many .bbappends replacing bash with ${VIRTUAL-RUNTIME_bash} but I guess I'll have to live with that.

Unfortunately there was only one reply on RFC for bash issue here:
http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142237.html so I'm going to keep all these bbappends in our layers at least for now.

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=99206cb7b41be7a124ea8339bc891077f02128a2 change makes this better by filtering the visible dependencies.

I'll remove my work arounds with VIRTUAL-RUNTIME variables and re-test with latest master, but I think it still might pick wrong provider for /bin/bash when both busybox and bash proper are in the dependencies.