Bug 10241

Please also see:
http://stackoverflow.com/questions/38824707/missing-files-in-rootfs-when-adding-links-bin-usr-bin-and-lib-usr-lib-i
for nicer formatting.

When building a custom kernel I want a file system where /bin is a soft link to /usr/bin and /lib is a soft link to /usr/lib. I'm using Yocto as build system.

I have changing package base-files, so that the /bin directory is soft linked to /usr/bin and the /lib directory is soft linked to /usr/lib, the resulting image/rootfs directory structure is missing a lot of files, most of which are unrelated to any of the directories mentioned above. E.g. file://etc/fstab is missing. When inspecting the base-files .rpm it contains the file structure​ that is expected. For some reason the package install order changes when building rootfs when the links from /lib and /usr are present.

I know too little about Yocto/kernel image building to be able to debug this effectively, since there is no errors reported and the build process is quite complex. Any solutions or help with where to continue my investigations or an explanation that this is the wro would be highly appreciated.

Actions to reproduce: I first detected the problem in an adaptation of Yocto, but I have reproduced it in the project described in the quick start: http://www.yoctoproject.org/docs/1.8/yocto-project-qs/yocto-project-qs.html

Set up a new Yocto project as described in the link above
Set up a new layer with a .bbappendfile as follows
--../recepies-linkdir/base-files/base-files_%.bbappend ----------------------------------------
dirs755_remove = "/bin /lib"

do_install_extra_links() {
    ln -srvf ${D}/usr/bin ${D}/bin
    ln -srvf ${D}/usr/lib ${D}/lib
}

addtask install_extra_links after do_install before do_populate_sysroot do_package

-- ../meta-linkdir/conf/layer.conf ----------------------------------------
# We have a conf and classes directory, add to BBPATH 
BBPATH =. "${LAYERDIR}:" 
# We have a packages directory, add to BBFILES 
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ 
            ${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "linkdir" 
BBFILE_PATTERN_linkdir := "^${LAYERDIR}/" 
BBFILE_PRIORITY_linkdir := "6"


Build the software
$ bitbake core-image-sato

Inspect the .rpm
$ rpm -lvpq ./tmp/work/qemux86-poky-linux/base-files/3.0.14-r89/deploy-rpms/qemux86/base-files-dev-3.0.14-r89.qemux86.rpm
Note that: /bin -> /usr/bin, /lib -> /usr/lib and that e.g. /etc/fstab is present.

Save the generated file-structure
$ pushd ./tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/rootfs 
$ LC_ALL=C find . -exec "stat -c '%N' {} \;" > /tmp/yocto_link_fs.txt
$ popd

Save the build log from rootfs build
$ cp .tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/log.do_rootfs s/tmp/log.do_rootfs.link

To be able to compare with the original filesystem, disable the changes in the .bb-append file, i.e. edit it to look like this:
-- ../recepies-linkdir/base-files/base-files_%.bbappend -------------------  

# dirs755_remove = "/bin /lib"

do_install_extra_links() {
    ln -srvf ${D}/usr/bin ${D}/bin
    ln -srvf ${D}/usr/lib ${D}/lib
}

# addtask install_extra_links after do_install before do_populate_sysroot do_package

Rebuild the project
$ bitbake -c cleanall base-files
$ bitbake core-image-sato
Inspect the .rpm
rpm -lvpq ./tmp/work/qemux86-poky-linux/base-files/3.0.14-r89/deploy-rpms/qemux86/base-files-dev-3.0.14-r89.qemux86.rpm
Note that: /bin and /lib are normal directories and that /etc/fstab is present

Save the generated file-structure
$ pushd ./tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/rootfs
$ LC_ALL=C find . -exec "stat -c '%N' {} \;" > /tmp/yocto_orig_fs.txt
$ popd

Save the build log from rootfs build
$ cp .tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/log.do_rootfs /tmp/log.do_rootfs.orig

Compare the difference and note that files are exclusively missing from the build where the soft-links where added.
$ diff /tmp/yocto_orig_fs.txt /tmp/yocto_links_fs.txt
<!-- language: lang-diff -->
2d1
< './proc'
16d14
< './usr/share/info'
3643d3640
< './usr/share/dict'
5068,5070d5064
< './usr/share/dbus-1/interfaces/org.freedesktop.ConsoleKit.Seat.xml'
< './usr/share/dbus-1/interfaces/org.freedesktop.ConsoleKit.Session.xml'
< './usr/share/dbus-1/interfaces/org.freedesktop.ConsoleKit.Manager.xml'
5075d5068
< './usr/share/dbus-1/system-services/org.freedesktop.ConsoleKit.service'
5382d5374
< './usr/share/man'
5386d5377
< './usr/src'
5488d5478
< './usr/bin/ck-launch-session'
5603d5592
< './usr/bin/ck-list-sessions'
5674d5662
< './usr/bin/ck-history'
5708d5695
< './usr/sbin/ck-log-system-restart'
5710d5696
< './usr/sbin/ck-log-system-stop'
5758d5743
< './usr/sbin/ck-log-system-start'
5771d5755
< './usr/sbin/console-kit-daemon'
6119d6102
< './usr/lib/libck-connector.so.0.0.0'
6287d6269
< './usr/lib/libck-connector.so.0' -> 'libck-connector.so.0.0.0'
7570,7571d7551
< './usr/lib/ConsoleKit/scripts/ck-system-stop'
< './usr/lib/ConsoleKit/scripts/ck-system-restart'
7597,7599d7576
< './usr/lib/consolekit/ck-get-x11-display-device'
< './usr/lib/consolekit/ck-get-x11-server-pid'
< './usr/lib/consolekit/ck-collect-session-info'
7739,7740d7715
< './dev'
< './run'
7812,7813d7786
< './home'
< './home/root'
7818d7790
< './etc/nsswitch.conf'
7823d7794
< './etc/default/usbd'
7910d7880
< './etc/fstab'
7936d7905
< './etc/hostname'
7983d7951
< './etc/motd'
8020d7987
< './etc/profile'
8086d8052
< './etc/mtab' -> '/proc/mounts'
8098d8063
< './etc/filesystems'
8138d8102
< './etc/issue.net'
8163d8126
< './etc/host.conf'
8227d8189
< './etc/dbus-1/system.d/ConsoleKit.conf'
8434d8395
< './etc/issue'
8446d8406
< './etc/ConsoleKit/seats.d/00-primary.seat'
8460,8465d8419
< './etc/inputrc'
< './etc/skel'
< './etc/skel/.bashrc'
< './etc/skel/.profile'
< './tmp'
< './sys'
8533d8486
< './media'
8535d8487
< './var/lock' -> '../run/lock'
8544,8549c8496
< './var/volatile'
< './var/run' -> '../run'
< './var/tmp' -> 'volatile/tmp'
< './var/backups'
< './var/local'
< './var/log' -> 'volatile/log'
---
> './var/log'
8562d8508
< './var/lib/misc'
Compare the package install order (for some strange reason) has changed by comparing the rootfs-logs

diff /tmp/log.do_rootfs .tmp/work/q1775-poky-linux/axis-image-cvp/1.0-r0/temp/log.do_rootfs
Output is not pasted in here for brewity. For some strange reason, the package install order has changed. Possibly this is the root cause of the problem
Conclusion/Expected results

The resulting file-system is not the intended, since there are many missing files. The base-files .rpm package contains the intended file structure, but is seems like Yocto (or possibly some .rpm package) is removing files.

Regardless of the root cause of the bad file-system, no error messages are presented by Yocto or the package installation process. It is unclear for me whether the latter depends on bugs in Yocto, the rpm packaging system, the packages. It is also a possibility that Yocto eats the error reporting from rpm.

I't would have been easier to comprehend that files intended for /usr or /lib was missing, but looking at the diff above files like /etc/fstab and /media is missing which seems to transcend normal reasoning.