At the moment we have to explain to users how to investigate why a particular recipe is being built or package is being included in their image and it's not a straightforward process. We have bitbake -g, however the graphs this produces are not easy to visualise due to their size. The depexp UI helps with this a little but it may not be showing the complete picture. Some way of filtering the graphs might be useful. A large proportion of this may just end up being documentation since most of the tools are already there. Buildhistory can be useful for graphing the dependencies during image creation, but buildhistory is not yet documented. We may want to look at bundling xdot either in bitbake or in the metadata as this is the most performant way I have found to view large graphs (although it is still not usable for the graphs produced with -g, it can be useful for viewing the dependency graphs produced by buildhistory). As a bonus it's written in Python (and PyGTK+): http://code.google.com/p/jrfonseca/wiki/XDot
Some of the webhob visualized analysis features will fill the requirements for this
Some developments since this bug was originally written: 1) Toaster (at one time known as "Web hob") provides the ability to drill down through dependency relationships after a build 2) depexp has been replaced by taskexp, which shows only task dependencies 3) buildhistory got documented: http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality 4) There's a section in the Technical FAQ (draft) that I've compiled that covers various dependency situations: https://wiki.yoctoproject.org/wiki/Technical_FAQ#Dependencies I think all we're really missing is a proper how-to guide for investigating dependencies. The technical FAQ section linked above would be good starter material.
taking a look
I think there may be a new approach we could take with this now. We could write a small tinfoil based tool which could blacklist a dependency and when you do this you do get much more useful information from bitbake about where the dependency is needed. Many people already use this technique to debug this problem, this bug would be able writing a tool to automate the process.
When looking for a reason why a specific package is built or included in the image, reverse dependencies are *invaluable*. There is an old and seemingly unmaintained tool, `oe-deptools`, which parsed pn-depends.dot and was capable of printing reverse dependencies that way. First of all, a reverse dependency lookup in Yocto wouldn't make much sense without an extra context - such as an upstream dependency (most typically an image, but might also be a packagegroup or another package). Otherwise there's just too many of them. This could either be a separate tool, or built into BitBake (`-G TASK` ?). I do not know anything about BitBake internals to say how difficult this would turn out to be. Updating (if necessary) `oe-deptools` should probably be quite easy, although I am worried about the fragility of it's extremely simplistic parser - any change to BitBake's output could brake it. And in my use case it is currently blocked by #14621
Forgot to link oe-deptools: https://github.com/scottellis/oe-deptools
Is this now covered by this tool? https://git.yoctoproject.org/poky/tree/scripts/oe-depends-dot
also agree this should be covered by oe-depends-dot, that might need some documentation? Or do I miss something? Here is an example First I did build "bitbake -g core-image-minimal" to create the task-depends.dot, and then I did run this: oe-depends-dot task-depends.dot -w -k kmod Because: core-image-minimal eudev util-linux bash btrfs-tools e2fsprogs fontconfig glib-2 mdadm shadow acl coreutils dbus ptest-runner attr bc bluez5 busybox bzip2 curl diffutils elfutils expat findutils gawk gdbm gnutls liberror-perl libgcrypt libgpg-error libseccomp libxml2 nettl e openssl perl python3 sed strace wayland zlib binutils cairo dropbear file freetype gcc gobject-introspection kmod libarchive libpcre2 libpng mesa pixman vulkan-loader grep libical python3-pygobject libmicrohttpd python3-pycairo iproute2 opkg-utils python3-dbus vulkan-headers libe dit ncurses packagegroup-core-boot procps psmisc readline sysvinit unzip vala which xz socat ca-certificates shared-mime-info createrepo-c-native desktop-file-utils gnome-desktop-testing gobject-introspection-native libdnf-native libical-native libmodulemd-native librepo-native lib slirp-native qemu-native qemu-system-native shared-mime-info-native vala-native qemu-helper-native libdrm qemuwrapper-cross wayland-protocols xorgproto libice libsm libx11 libxau libxcb libxdamage libxdmcp libxext libxfixes libxrandr libxrender libxxf86vm xrandr libxshmfence dnf-na tive bash-completion iptables core-image-minimal -> mdadm -> eudev -> util-linux -> kmod core-image-minimal -> mdadm -> strace -> bluez5 -> eudev -> util-linux -> kmod core-image-minimal -> packagegroup-core-boot -> eudev -> util-linux -> kmod core-image-minimal -> libarchive -> acl -> e2fsprogs -> bzip2 -> opkg-utils -> bash -> coreutils -> attr -> gawk -> ptest-runner -> shadow -> util-linux -> kmod core-image-minimal -> binutils -> elfutils -> libarchive -> acl -> e2fsprogs -> bzip2 -> opkg-utils -> python3 -> bash -> coreutils -> attr -> gawk -> ptest-runner -> shadow -> util-linux -> kmod core-image-minimal -> mesa -> vulkan-loader -> wayland -> binutils -> elfutils -> libarchive -> acl -> e2fsprogs -> bzip2 -> opkg-utils -> python3 -> expat -> bash -> coreutils -> attr -> gawk -> ptest-runner -> shadow -> util-linux -> kmod core-image-minimal -> bluez5 -> python3-pygobject -> python3-pycairo -> cairo -> mesa -> vulkan-loader -> wayland -> binutils -> elfutils -> libarchive -> acl -> e2fsprogs -> bzip2 -> opkg-utils -> python3 -> expat -> bash -> coreutils -> attr -> gawk -> ptest-runner -> shadow -> util-linux -> kmod
Richard may comment again...