Bug 11308

Summary: Consider replacing pkg-config with pkgconf
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Joshua Lock <joshuagloe>
Component: coreAssignee: Unassigned <unassigned>
Status: RESOLVED OBSOLETE QA Contact:
Severity: enhancement    
Priority: Medium CC: brian.avery, kergoth, maxin.john, meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie, ross.burton, sgw, stanley.cheong.kwan.phoong
Version: 2.4   
Target Milestone: 4.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Error log pkgconf bitbake world none

Description Joshua Lock 2017-04-07 10:01:36 UTC
Fedora 26 replaces the system pkg-config implementation with pkgconf[1] because it "provides better support for handling .pc files and a stable library ABI/API for integrating into applications."[2] and is actively maintained, unlike pkg-config.

pkgconf aims to offer many improvements over pkg-config such as faster/more efficient dependency resolver which "allows for the user to more conservatively link their binaries -- which may be helpful in some environments, such as when prelink(1) is being used."[3]

pkgconf also aims to provide a more complete implementation of pkg-config.

A full feature breakdown is available on the pkgconf website[4].

The features most likely to benefit the Yocto Project build system are the faster/more efficient dependency resolution[3] and linker flag optimisation[5].

1. http://pkgconf.org/
2. https://fedoraproject.org/wiki/Changes/pkgconf_as_system_pkg-config_implementation
3. https://github.com/pkgconf/pkgconf#comparison-of-pkgconf-and-pkg-config-dependency-resolvers
4. http://pkgconf.org/features.html
5. https://github.com/pkgconf/pkgconf#linker-flags-optimization
Comment 1 Christopher Larson 2017-04-07 14:03:40 UTC
https://github.com/kergoth/meta-kergoth-wip/tree/master/meta-pkgconf may be of interest to anyone wanting to work on this, as a starting point, otherwise I'll see if I can find the time. I haven't had time to mess with it in a while.
Comment 2 Christopher Larson 2017-04-07 18:05:00 UTC
Updated the meta-pkgconf recipe to 1.3.5 and successfully built core-image-minimal, core-image-base, and core-image-sato with it so far.
Comment 3 brian avery 2017-04-13 14:39:27 UTC
We should check and see if this could allow us to avoid the pkgconfig-native vs pkgconfig in the esdk. That separation breaks menuconfig in the esdk Bug #11155
Comment 4 Christopher Larson 2017-04-13 15:15:35 UTC
I've often thought it'd be nice to have a single pkgconfig install, but with wrappers where appropriate that internally set variables to adjust search paths, thereby avoiding both pkgconfig-native and global exports of the pkgconfig variables in our classes, but not sure whether it's viable or not.
Comment 5 Christopher Larson 2017-04-13 15:16:37 UTC
Successfully built oe-core world with pkgconf, using the latest version which is now in meta-pkgconf. Further testing should be done with other layers, and it could use more runtime testing. It would also be worthwhile to add ptest support to the recipe, but that requires use of kyua, the test runner used by the pkgconf project, which I haven't had time to dig into.
Comment 6 Stanley Phoong 2017-05-18 00:24:25 UTC
Hi Christopher, I'm trying to help perform some testings, what I did was, bitbake core-image-minimal with intel-corei7-64 and this seems to work but if I tried bitbake world, I would get different failures. Could you advice on your configuration that you made to get the bitbake world to work?
Comment 7 Stanley Phoong 2017-05-22 08:24:45 UTC
Created attachment 3769 [details]
Error log pkgconf bitbake world

Tried bitbake world for the pkgconf but it seems to show this error. I've attached the error log on this to provide you with more detail on this issue.
Comment 8 Stanley Phoong 2017-06-30 07:06:33 UTC
Some updates for this bug,

1. pkg-conf is stated to be a one-to-one replacement for pkg-config, behavior is expected to be the same.

2. during bitbake world, pkg-config succeeded however, pkg-conf fails
The error shown by pkg-conf is failure from libx11. *so instead of bitbake world you can just test this out merely by bitbake libx11* (saves you lots of time)

3. libx11 states this error:
| /workdir/11308/tmp/work/i586-poky-linux/libx11/1_1.6.5-r0/recipe-sysroot/usr/include/bits/long-double.h
:57:33: fatal error: bits/long-double-64.h: No such file or directory
| compilation terminated.

Which sounds like a missing library. I tested added that library e.g -l<path>/long-double-64.h into the recipe but even more errors started showing, by which I started debugging and resolving each of these errors which came out more errors. I think something is definitely not right, so don't try this route.

4. Tested building pkg-conf and drop down into devshell and build a pkg-config and drop down into devshell to compare the differences in behavior since pkg-conf and pkg-config is expected to react similarly. Found some differences in aclocal.m4. I tried removing those differences from aclocal.m4 which doesn't affect the output as well.

5. As I tried comparing more things, it seems that I missed out one thing which I just detected recently, I was compiling for qemux86 however, the error message prompted from bitbake shows that "bits/long-double-64.h" is missing. Technically, it should be building 
"bits/long-double-32.h". Probably somewhere inside the config or some settings is building the wrong arch type?

6. Checking inside 11308/tmp/work/i586-poky-linux/libx11/1_1.6.5-r0/recipe-sysroot/usr/include/bits/long-double.h, 

#elif __MHWORDSIZE == 64
#include <bits/long-double-64.h>
#else
#error "Unknown __WORDSIZE detected"
#endif /* matches #if __WORDSIZE == 32 */

it seems to show that if the __WORDSIZE == 64, it'll choose the bits/long-double-64.h, so I am in the midst of investigating this as to whether the __WORDSIZE is truly supposed to be 64 under qemux86, rather than 32 and if it is not supposed to be 64 then why is it that when pkg-conf is selected, the __WORDSIZE is has changed? 


I've not yet found the answer these two questions yet. 

Possible next step:
Either gdb into recipe-sysroot/usr/include/bits/long-double.h and print __WORDSIZE or create a print function inside pkg-config to answer the first question and verify if the __WORDSIZE == 32 or 64.
Comment 9 Maxin B. John 2017-07-03 12:12:51 UTC
As discussed with Ross.
Comment 10 Maxin B. John 2017-07-10 10:05:03 UTC
(In reply to comment #8)
> Some updates for this bug,
> 
> 1. pkg-conf is stated to be a one-to-one replacement for pkg-config,
> behavior is expected to be the same.
> 
> 2. during bitbake world, pkg-config succeeded however, pkg-conf fails
> The error shown by pkg-conf is failure from libx11. *so instead of bitbake
> world you can just test this out merely by bitbake libx11* (saves you lots
> of time)
> 
> 3. libx11 states this error:
> |
> /workdir/11308/tmp/work/i586-poky-linux/libx11/1_1.6.5-r0/recipe-sysroot/usr/
> include/bits/long-double.h
> :57:33: fatal error: bits/long-double-64.h: No such file or directory
> | compilation terminated.

We can fix this build error with this patch:

diff --git a/meta/recipes-graphics/xorg-lib/libx11_1.6.5.bb b/meta/recipes-graphics/xorg-lib/libx11_1.6.5.bb
index 71cbf67..50e8837 100644
--- a/meta/recipes-graphics/xorg-lib/libx11_1.6.5.bb
+++ b/meta/recipes-graphics/xorg-lib/libx11_1.6.5.bb
@@ -5,6 +5,9 @@ BBCLASSEXTEND = "native nativesdk"
 
 SRC_URI += "file://disable_tests.patch \
            "
+do_configure_append () {
+    sed -i -e "/X11_CFLAGS/d" ${B}/src/util/Makefile
+}
---

src/util/makekeys should be compiled for the host since it is executed at build time to generate ks_tables.h. Since we have the X11 include files in the standard path of oe-core, we can remove the X11_CFLAGS from the Makefile.

Shamelessly adapted from buildroot:
https://git.buildroot.net/buildroot/tree/package/x11r7/xlib_libX11/xlib_libX11.mk
Comment 11 Maxin B. John 2017-07-11 13:01:33 UTC
patch sent to oe-core:
http://lists.openembedded.org/pipermail/openembedded-core/2017-July/139381.html
Comment 12 Maxin B. John 2017-08-23 11:18:06 UTC
pkgconf is available in oe-core:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=ee8b56528dacf9b2821d537074fa40ad2d1baa7a
Comment 13 Maxin B. John 2017-08-23 11:55:44 UTC
As Jussi suggested, keeping it open till we actually remove/replace pkgconfig with pkgconf.
Comment 14 Maxin B. John 2018-02-21 07:50:30 UTC
patch set sent to oe-core:
http://lists.openembedded.org/pipermail/openembedded-core/2018-February/147567.html
Comment 15 Randy MacLeod 2018-03-02 16:00:41 UTC
Reset milestone to 2.5-M3 since a patch has been sent to the list.

I think Robert Jordan's web browser inadvertently set the milestone. I've had the same problem but we don't understand how it happens. It may be that a full page re-load will avoid the problem.
Comment 16 Ross Burton 2018-03-02 16:01:36 UTC
Moving back to 2.6 as we triaged this and had punted it to 2.6.
Comment 17 Randy MacLeod 2018-03-02 16:07:15 UTC
Ah, ok. I see now that there are a several bugs/enhancements being moved so I'll leave them alone rather than assume they are victims of the browser/Bugzilla bug that has been plaguing me! :)
Comment 18 Maxin B. John 2018-09-18 08:27:37 UTC
pkgconfig and pkgconf has some behaviour changes which prevents it from becoming a drop-in replacement.

Changes are available here:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=maxin/pkg-config
Comment 19 Richard Purdie 2021-06-24 15:08:00 UTC
The two are incompatible and pkgconfig works so we may as well just stick with that. We can reopen if there is a pressing reason to change.