Bug 10932

Summary:	RRECOMMENDS returned by Tinfoil API are not the final ones
Product:	[Build System, Metadata & Runtime] BitBake	Reporter:	Ismo Puustinen <ismo.puustinen>
Component:	bitbake	Assignee:	Richard Purdie <richard.purdie>
Status:	RESOLVED NOTABUG	QA Contact:
Severity:	normal
Priority:	Undecided	CC:	bluelightning, poky.bs.watcher, poky.watcher
Version:	2.3
Target Milestone:	2.0.3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Ismo Puustinen 2017-01-17 09:39:34 UTC

BitBake automatically adds packages to the RDEPENDS field (see http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#automatically-added-runtime-dependencies for details). For example, the shared objects that are linked against are automatically added to RDEPENDS. However, the Tinfoil API does not appear to show this information in the "rundeps" attribute information.

Steps to reproduce:

1. Build a suitable package (bitbake connman)
2. Run the following script to show the rdepends of connman-client. The outcome is just "connman", even though all the libraries that connman-client links against should be listed too.

import sys
import os

bindir = os.path.dirname(__file__)
topdir = os.path.dirname(bindir)

sys.path[0:0] = [os.path.join(topdir, 'lib')]

import bb.tinfoil

tinfoil = bb.tinfoil.Tinfoil(tracking=True)
tinfoil.prepare(False)

rundeps = tinfoil.cooker.recipecaches[''].rundeps
recipe = tinfoil.get_runtime_providers("connman-client")[1]

print("Listing the dependencies for connman-client:")
for d in rundeps[recipe]["connman-client"]:
    print(d)

tinfoil.shutdown()

Comment 1 Paul Eggleton 2017-01-18 10:19:38 UTC

FWIW, tinfoil cannot show you anything that's computed at packaging time, which would include any dependencies generated from shared object links - at least not through the rundeps. For that you need to have actually run do_packagedata for the recipe in question (which is a task run during normal course of building a recipe to the packaging stage) and then you can find the results in pkgdata - see scripts/oe-pkgdata-util for some example code that reads that.

In your separate email you mentioned that bitbake -g was showing something that you didn't find in the rundeps, but that doesn't seem to be mentioned here... ?

Comment 2 Ismo Puustinen 2017-01-18 11:55:35 UTC

Yes. The question is that if this is the right behavior and in line with the docs? If RDEPENDS is automatically updated by BitBake, shouldn't the Tinfoil API show that data too?

I left the bitbake -g description out from this bug report to avoid mixing things up even more. :-) Documentation says about package-depends.dot: "A graph showing known dependencies between runtime targets." I can do this:

--- --- ---- --- --- --- --- --- --- --- ---

$ bitbake -g connman
... skip text ...
NOTE: Package dependencies saved to 'package-depends.dot'

$ grep connman-client package-depends.dot 
"connman-client" [label="connman-client(connman) :1.33-r0\n/home/ipuustin/git/poky/meta/recipes-connectivity/connman/connman_1.33.bb"]
"connman-client" -> "virtual/libc" [style=solid]
"connman-client" -> "autoconf-native" [style=solid]
"connman-client" -> "initscripts" [style=solid]
"connman-client" -> "update-rc.d-native" [style=solid]
"connman-client" -> "update-rc.d" [style=solid]
"connman-client" -> "automake-native" [style=solid]
"connman-client" -> "glib-2.0" [style=solid]
"connman-client" -> "bluez5" [style=solid]
"connman-client" -> "readline" [style=solid]
"connman-client" -> "ppp" [style=solid]
"connman-client" -> "ofono" [style=solid]
"connman-client" -> "gnutls" [style=solid]
"connman-client" -> "iptables" [style=solid]
"connman-client" -> "wpa-supplicant" [style=solid]
"connman-client" -> "pkgconfig-native" [style=solid]
"connman-client" -> "gnu-config-native" [style=solid]
"connman-client" -> "libtool-cross" [style=solid]
"connman-client" -> "virtual/x86_64-poky-linux-compilerlibs" [style=solid]
"connman-client" -> "virtual/x86_64-poky-linux-gcc" [style=solid]
"connman-client" -> "dbus" [style=solid]
"connman-client" -> "libtool-native" [style=solid]
"connman-client" -> "connman" [style=dashed]

--- --- ---- --- --- --- --- --- --- --- ---

This output looks pretty good even though I don't understand why -native packages are included, since they clearly aren't "runtime targets". At least "glib-2.0" and "readline" are there. Having Tinfoil to return these items in the .rundeps field would feel to me like a more useful behavior. The RDEPENDS variable data can be just read out from the parsed recipe, but getting the actual runtime dependencies would require the user to go through the pkgdata (as you mention).

Comment 3 Paul Eggleton 2017-01-18 19:39:57 UTC

Well, the actual value of RDEPENDS only gets extended internally within the packaging code - there's no way for it to be "permanently" extended such that reading it via tinfoil gives you the full answer because the packaging code doesn't run at that time, and in any case the source for the information (i.e. build output) might not be present. We could look at providing some library functions to read pkgdata, but these would be part of OE and not tinfoil (though they'd be just as easily accessible from a tinfoil-using script).

At the moment I can't explain the difference between rundeps and -g, but the fact that native recipes are in there doesn't suggest that the results are quite right. It seems to me that build-time dependencies are being copied in there.

Comment 4 Richard Purdie 2017-01-23 14:08:40 UTC

To be honest, I'd not trust anything you see in package-depends.dot or pn-depends.dot. They're legacy left from a previous era and prompted by this bug I'm tempted to remove them. They were replaced by task-depends.dot which is accurate and definitive.

Regardless of what the docs say, the output in package-depends.dot is a mix of DEPENDS and RDEPENDS data which is why you see -native things in there. You should be able to pull similar data out of tinfoil. For the reason Paul mentions, accurate runtime data for dependencies is only available after do_package executes. That data is written out by the do_packagedata task and can be accessed with oe-pkgdata-util.

Comment 5 Ismo Puustinen 2017-01-23 14:13:00 UTC

Ok, this is fair enough.