With the upgrade to glibc 2.34, testing with buildtools tarball reveals a problem. This can be seen with:

bitbake buildtools-extended-tarball
bitbake buildtools-extended-tarball -c testsdk

where linux-libc-headers:do_install will fail due to being unable to load pseudo. The real error is in bitbake-cookerdaemon.log where the issue with symbol mismatches will be seen. There are some fixes in master-next

pseudo: Fix to work with glibc 2.34 systems
gcc: also relocate the musl loader
gcc: Fix nativesdk builds and multilib fixes with gcc
glibc-package.inc: Fix builds of pseudo with glibc 2.34

however these aren't enough and don't full avoid the problems.

The issue is that pseudo-native is compiled against the buildtools compiler which means glibc 2.34. If a binary is then run by the host system which doesn't use glibc 2.34, the symbol differences become apparent. This happens with libpseudo.so since it is an LD_PRELOAD.

In particular, libpseudo.so uses a pthread mutex and dlerror(), dlsym() and dlvsym() calls. In 2.34 these moved from libpthread and libdl to libc. In linking with newer libs, the lib references are dropped. Even adding DT_NEEDED sections isn't enough since the linker helpfully optimises through to libc.so's symbols.

The huge problem is that as soon as distros start shipping with glibc 2.34, pseudo-native will work with the glibc it was built with but not the other set. We therefore have a problem which will soon cause us a lot of pain.

In the past we made pseudo-native host specific. The broke eSDKs in ways which ended with us reverting that change.

I did ask for ideas on the libc list:

https://sourceware.org/pipermail/libc-help/2021-August/005985.html

I have tried:

* adding sysroot-uninative to libpseudo.so's RUNPATH but it is not used by the linker for loading libc.so for host binaries
* injecting the sysroot-uninative path into pseudo's LD_LIBRARY_PATH and this does load the right libc.so however since the loader was the host's ld.so, this errors
* the pseudo patch referenced above which changes the symbol versioning of dlerror/dlsym/dlvsym however the linker still resolves the symbols to libc.so rather than libdl.so

I'd note that the pthread symbols don't use 2.34 versioning so are slightly less problematic but that the pthread APIs look to have changed meaning that using an older pthread lib with newer compiled calls may be highly suspect.

Other ideas:

* adding a dummy libdl to link against to fool libpseudo into looking as we need it
* downloading older binaries to link pseudo-native against