1784 2011-11-24 00:30:26 +0000 busybox not working with pseudo:libpseudo.so cannot be preloaded: ignored 2012-04-03 08:34:57 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core unspecified x86 Multiple RESOLVED WONTFIX (1.2) Medium normal 1.2 M4 1 dexuan.cui bluelightning dexuan.cui edwin.zhai jessica.zhang meta.mr.watcher meta.watcher richard.purdie sgw shane.wang --- --- --- oldest_to_newest 17360 0 dexuan.cui 2011-11-24 00:30:26 +0000 When doing the Build Appliance work(https://wiki.yoctoproject.org/wiki/Build_Appliance_Design), I found in target qemu (or BSP like n450 or sugarbay), the logs of many tasks have such a line: ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored Further investigation shows it's because the busybox utilities(like gzip, gunzip, sed, cpio, mktemp, hostname) don't work with pseudo. E.g, in the strace log, we can see when gzip is executed, somehow it tries to load the libpseudo.so in host's /lib and /usr/lib... This is wrong (actually the host doesn't have such a .so file) Here gzip is symbol of busybox. And before the gunzip's execution, other programs(like tar, git, grep, tr, uname, etc -- they are not a symbol to busybox) run with ${STAGING_NATIVE_LIBDIR}'s libpseudo.so loaded properly. The below method is an easier way to reproduce the issue: sugarbay:/raid/pe2/build-3# id uid=1000(poky) gid=1000(poky) groups=1000(poky) sugarbay:/raid/pe2/build-3# tmp/sysroots/x86_64-linux/usr/bin/pseudo gunzip ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored. gunzip: applet not found sugarbay:/raid/pe2/build-3# tmp/sysroots/x86_64-linux/usr/bin/pseudo tar tar: You must specify one of the `-Acdtrux' or `--test-label' options Try `tar --help' or `tar --usage' for more information. BTW, the info of busybox on sugarbay is: sugarbay:/raid/pe2/build-3$ which busybox /bin/busybox sugarbay:/raid/pe2/build-3$ ldd /bin/busybox linux-vdso.so.1 => (0x00007fffad86f000) libc.so.6 => /lib/libc.so.6 (0x00007fdd552c7000) /lib/ld-linux-x86-64.so.2 (0x00007fdd55651000) sugarbay:/raid/pe2/build-3 $ file /bin/busybox /bin/busybox: setuid ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, stripped For now, I use the the utilities of non-busybox verson as a workaround, but in the long run, we may want to fix this bug. -------- From Mark: I wonder if busybox is playing with the LD_LIBRARY_PATH or something. That should be set and informing the loader to look in the alternative location(s). (If the busybox shell is playing with the environment variable storage and retrieval this could also affect things as well.) 17502 1 dexuan.cui 2011-12-05 22:48:05 +0000 BTW: e.g., the utility "hostname" of busybox version has the "ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored" issue, too. In the self-hosted-image work, I found the below tasks are affected by the "hostname": tmp/work/i586-poky-linux/elfutils-0.148-r3/temp/log.do_configure tmp/work/i586-poky-linux/libgpg-error-1.10-r1/temp/log.do_configure tmp/work/i586-poky-linux/libtool-2.4-r2/temp/log.do_configure tmp/work/i586-poky-linux/libtool-cross-2.4-r2/temp/log.do_configure tmp/work/x86_64-linux/elfutils-native-0.148-r3/temp/log.do_configure tmp/work/x86_64-linux/perl-native-5.12.3-r5/temp/log.do_configure tmp/work/qemux86-poky-linux/linux-yocto-3.0.4+git1+d05450e4aef02c1b7137398ab3a9f8f96da74f52_1+72671808fdbe69a9fe03fd8f094e7c59da04a28c-r2/temp/log.do_compile When this bug is fixed, I'll check the the logs of these tasks. 19792 2 bluelightning 2012-04-02 14:54:57 +0000 Looks like busybox is clearing out LD_LIBRARY_PATH - leaving pseudo aside, if you set it and then invoke busybox as "sh", it is not set within the resulting shell; this might explain the problem we're having. The reason for this is that busybox is marked suid root; clearing this variable is an automatic security feature. It seems to me either we make busybox non-suid-root (and break any busybox modules that need it, e.g. passwd, login) or install real executables for all commands we might want to work properly under pseudo. 19793 3 dexuan.cui 2012-04-02 15:39:23 +0000 (In reply to comment #2) > Looks like busybox is clearing out LD_LIBRARY_PATH - leaving pseudo aside, if > you set it and then invoke busybox as "sh", it is not set within the resulting > shell; this might explain the problem we're having. The reason for this is that > busybox is marked suid root; clearing this variable is an automatic security > feature. Hi Paul, thanks for the explanation! It's reasonable to me. BTW, out of curiosity, how can busybox clear out LD_LIBRATY_PATH? I didn't think an application could do that: before an application starts to run, the LD_LIBRARY_PATH variable has been used by the loader (e.g., /lib/ld-linux.so.2)? > It seems to me either we make busybox non-suid-root (and break any busybox > modules that need it, e.g. passwd, login) or install real executables for all > commands we might want to work properly under pseudo. It seems we can only choose to install real executables. However, in the cases mentioned in comment #1, the real executable version of hostname supplied by coreutils is said broken, so we have to use the busybox version: $ grep hostname meta/recipes-core/coreutils -nr meta/recipes-core/coreutils/coreutils_8.14.bb:32:# hostname gets a special treatment and is not included in this meta/recipes-core/coreutils/coreutils_8.14.bb:79: update-alternatives --remove hostname hostname.${PN} meta/recipes-core/coreutils/coreutils_6.9.bb:41:# hostname gets a special treatment and is not included in this meta/recipes-core/coreutils/coreutils_6.9.bb:63: # hostname and uptime separated. busybox's versions are preferred ... Luckily the messages mentioned in comment #1 are actually warnings. So I think we can safely ignore them. :-) So maybe we can close the bug and mark it as WON'T FIX. 19794 4 bluelightning 2012-04-02 15:53:02 +0000 In this case it's not really busybox clearing the variables, I think it may be the loader. In any case I believe it is automatic for any executable that is suid/sgid. 19796 5 dexuan.cui 2012-04-02 16:14:57 +0000 (In reply to comment #4) > In this case it's not really busybox clearing the variables, I think it may be > the loader. In any case I believe it is automatic for any executable that is > suid/sgid. Oh, got it! Thanks for the explanation! I didn't realize the relation between LD_LIBRARY_PATH and setuid/setgid. :-) I found the article that explains the issue in detail: http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html 3.3.1. LD_LIBRARY_PATH ... 3.3.3. Other Environment Variables ... Permitting user control over dynamically linked libraries would be disastrous for setuid/setgid programs if special measures weren't taken. Therefore, in the GNU loader (which loads the rest of the program on program start-up), if the program is setuid or setgid these variables (and other similar variables) are ignored or greatly limited in what they can do. The loader determines if a program is setuid or setgid by checking the program's credentials; if the uid and euid differ, or the gid and the egid differ, the loader presumes the program is setuid/setgid (or descended from one) and therefore greatly limits its abilities to control linking. If you read the GNU glibc library source code, you can see this; see especially the files elf/rtld.c and sysdeps/generic/dl-sysdep.c. This means that if you cause the uid and gid to equal the euid and egid, and then call a program, these variables will have full effect. Other Unix-like systems handle the situation differently but for the same reason: a setuid/setgid program should not be unduly affected by the environment variables set. 19807 6 bluelightning 2012-04-03 08:34:57 +0000 Marking WONTFIX as suggested, since currently the effects are not harmful to the build process in the self-hosted image - if we want to revisit this in 1.3 we can open a new bug.