Bug 15032 - Failed to build libusb-compat-native with bitbake in crops/poky:ubuntu-18.04 container
Summary: Failed to build libusb-compat-native with bitbake in crops/poky:ubuntu-18.04 ...
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: oe-core other (show other bugs)
Version: 4.2
Hardware: x86 x86_64
: Medium+ normal
Target Milestone: 4.3 M1
Assignee: Yoann Congal
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2023-02-10 09:59 UTC by Abderrahim BENAOUDA
Modified: 2023-03-28 22:29 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
logfile of failure (1.19 KB, application/octet-stream)
2023-02-10 10:52 UTC, Abderrahim BENAOUDA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Abderrahim BENAOUDA 2023-02-10 09:59:08 UTC
Reproducing the Bug:
In crops/poky:ubuntu-18.04 container add meta-oe layer (found in meta-openembedded master branch), build libusb-compat-native package with bitbake

below is the build log:

NOTE: Executing Tasks
ERROR: libusb-compat-native-1_0.1.7-r0 do_compile: oe_runmake failed
ERROR: libusb-compat-native-1_0.1.7-r0 do_compile: ExecutionError('/workdir/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/temp/run.do_compile.175', 1, None, None)
ERROR: Logfile of failure stored in: /workdir/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/temp/log.do_compile.175
Log data follows:
| DEBUG: Executing python function autotools_aclocals
| DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc', 'bit-64', 'x86_64-linux', 'common']
| DEBUG: Python function autotools_aclocals finished
| DEBUG: Executing shell function do_compile
| NOTE: make -j 8
| Makefile:282: *** missing separator.  Stop.
| ERROR: oe_runmake failed
Comment 1 Abderrahim BENAOUDA 2023-02-10 10:52:43 UTC
Created attachment 4930 [details]
logfile of failure
Comment 2 Abderrahim BENAOUDA 2023-02-10 11:41:02 UTC
The generated Makefile by autotools is not correctly formatted. See lines 281 the "./confest:\n" should not be there.

280 LIBUSB_1_0_LIBS = -L/workdir/build/tmp/work/x86_64-linux/libusb-compat-nativ    e/1_0.1.7-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/lib -lusb-    1.0
281 LIBUSB_1_0_SONAME = ./conftest:
282 libusb-1.0.so.0
283 LIPO =

The package libusb-compat-native builds successfully in the base container crops/yocto:ubuntu-18.04-base, with a correct Makefile:

280 LIBUSB_1_0_LIBS = -L/workdir/build/tmp/work/x86_64-linux/libusb-compat-nativ    e/1_0.1.7-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/lib -lusb-    1.0
281 LIBUSB_1_0_SONAME = libusb-1.0.so.0
282 LIPO =

Also builds successfully on crops/poky:debian-10 container 

280 LIBUSB_1_0_LIBS = -L/workdir/build/tmp/work/x86_64-linux/libusb-compat-nativ    e/1_0.1.7-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/lib -lusb-    1.0
281 LIBUSB_1_0_SONAME = 
282 LIPO =
Comment 3 Ross Burton 2023-02-16 15:34:01 UTC
"./conftest:" looks like the configure test is failing but the Makefile (or configure script) isn't catching the failure.  Reading the config.log would be interesting to see if that has any failures.
Comment 4 Yoann Congal 2023-02-16 15:35:21 UTC
Note: libusb-compat v0.1.8 released on Nov 18, 2022 : https://github.com/libusb/libusb-compat-0.1/releases/tag/v0.1.8

Maybe try to upgrade?
Comment 5 Yoann Congal 2023-02-18 16:00:12 UTC
> Maybe try to upgrade [to v0.1.8] ?

Note: upgrading to v0.1.8 does not solve this problem.

> "./conftest:" looks like the configure test is failing but the Makefile (or configure script) isn't catching the failure.  Reading the config.log would be interesting to see if that has any failures.

log.do_configure :
checking for SONAME of libusb-1.0... ./conftest:
libusb-1.0.so.0
checking for GNU extensions of errno.h... no
configure: WARNING: cache variable au_cv_lib_soname_LIBUSB_1_0 contains a newline


config.log:
configure:12984: checking for SONAME of libusb-1.0
configure:13016: gcc  -o conftest -isystem...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/include -O2 -pipe -isystem...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/include -L...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/lib                         -L...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/lib                         -Wl,-rpath,...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath,...poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=...poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 conftest.c   -lusb-1.0 >&5
configure:13016: $? = 0
configure:13044: result: ./conftest:
libusb-1.0.so.0

After that the other conftest build fail because of the ill-formed #define LIBUSB_1_0_SONAME
Comment 6 Yoann Congal 2023-02-19 17:25:20 UTC
libusb-compat tries to compute the libusb SO name by compiling a linking against is and then parsing the output of ldd on the resulting binary.

See https://github.com/libusb/libusb-compat-0.1/blob/v0.1.7/m4/au_check_lib_soname.m4#L29.

The suspicious line is :
  ldd conftest$ac_exeext | grep 'lib[$2]'$shrext_regexp | sed 's/^@<:@ \t@:>@*lib[$2]'$shrext_regexp'/lib[$2]'$shrext_regexp'/;s/@<:@ \t@:>@.*$//'

Here's the output of ldd on an impacted system :
1 ./conftest: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./conftest)
2 ./conftest: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by .../poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/lib/libusb-1.0.so.0)
3 linux-vdso.so.1 (0x00007ffdd2bb6000)
4 libusb-1.0.so.0 => .../poky/build/tmp/work/x86_64-linux/libusb-compat-native/1_0.1.7-r0/recipe-sysroot-native/usr/lib/libusb-1.0.so.0 (0x00007fcd4b799000)
5 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcd4b1a4000)
6 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcd4af85000)
7 .../poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fcd4b595000)

The code clearly look for the 4th line but the 2nd is also caught by the grep/sed regexp.

So, there are 2 problems:
1. ldd can't find test binary's glibc. I don't know if it's really a problem or not since this binary is never run.
2. The grep/sed regexp does not handle the problem 1 => This part can be fixed by patching the sources.
Comment 7 Yoann Congal 2023-02-20 08:41:01 UTC
Note: Related fix by our Buildroot friends :
https://git.buildroot.net/buildroot/commit/package/libusb-compat/libusb-compat.mk?id=3a9261ddd917007e19b56b4bfe48ccc0861dd716

They force LIBUSB_1_0_SONAME to prevent the "broken" libusb-compat SONAME detection from running...
Comment 8 Ross Burton 2023-02-20 13:05:22 UTC
The loader thing is odd, but the readelf workaround in buildroot would likely work for us (and arguably could be upstreamed as a better solution than running ldd).
Comment 9 Yoann Congal 2023-02-20 14:05:20 UTC
(In reply to comment #8)
> The loader thing is odd, but the readelf workaround in buildroot would
> likely work for us (and arguably could be upstreamed as a better solution
> than running ldd).

I arrived at the same conclusion. Thank you for the confirmation!

Instead of readelf, I've planned to use objdump (but the idea is the same)

I have a working patch, I just need to clean it up to push it to libusb and (I guess) to the recipe.
Comment 10 Yoann Congal 2023-02-27 07:50:18 UTC
libusb-compat used involuntarly host libs to compute libusb1 soname. Where host libc is recent enough, the computation works but if the libc is old, the computation fails. Using buildtools kinda hide this because the buildtools libc is recent enough.

Since https://git.yoctoproject.org/poky/commit/bitbake?id=7d67b6356033b5ffeb37d13bf748632ce917289b we need the buildtools to build on old distros so this bug may be hard to catch on master.

It does clearly appears when build libusb-compat-native on crops/poky:ubuntu-18.04 without installing the buildtools.
Comment 11 Yoann Congal 2023-02-28 21:14:25 UTC
Changing the Product to OE-Core : This is a problem of the libusb-compat recipe in oe-core and not linked to CROPS.
Resetting the ticket to NEW state to get triage team to check if that's OK.
 
Meanwhile, fix for this bug is in this PR : https://github.com/openembedded/meta-openembedded/pull/660