Bug 15361

Summary: libxml2 fails with missing python
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Patrick Williams <patrick>
Component: coreAssignee: Unassigned <unassigned>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: Undecided CC: Martin.Jansa, meta.mr.watcher, meta.watcher, richard.purdie
Version: 5.99   
Target Milestone: 0.0.0   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Patch to aggravate issue.
none
Revert inherit_defer for libxml. none

Description Patrick Williams 2024-01-19 18:31:04 UTC
I think this problem is related to the `inherit_defer` changes that went in recently.  It doesn't seem like they're working appropriately for the necessary inherit of python3native.

Observed this as of poky commit 6e00774a79b3df07c5e6732c6929ee2e77950592 when building libxml2:

```
10:22:14 | checking for python2... no
10:22:14 | checking for python3... /data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/hosttools/python3
10:22:14 | checking for python version... 3.10
10:22:14 | checking for python platform... linux
10:22:14 | checking for GNU default python prefix... ${prefix}
10:22:14 | checking for GNU default python exec_prefix... ${exec_prefix}
10:22:14 | checking for python script directory (pythondir)... ${libdir}/python3.10/site-packages
10:22:14 | checking for python extension module directory (pyexecdir)... ${libdir}/python3.10/site-packages
10:22:14 | checking for python-3.10... no
10:22:14 | configure: error: Package requirements (python-3.10) were not met:
10:22:14 | 
10:22:14 | No package 'python-3.10' found
```

Looking earlier in the log, I notice that the `--with-python` is pointing to the `/usr/bin/python` which is incorrect:

```
10:22:14 | NOTE: Running ../libxml2-2.11.5/configure  --build=x86_64-linux 		  --host=x86_64-linux 		  --target=x86_64-linux 		  --prefix=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr 		  --exec_prefix=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr 		  --bindir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin 		  --sbindir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/sbin 		  --libexecdir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/libexec 		  --datadir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share 		  --sysconfdir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/etc 		  --sharedstatedir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/com 		  --localstatedir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/var 		  --libdir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/lib 		  --includedir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/include 		  --oldincludedir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/include 		  --infodir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share/info 		  --mandir=/data0/jenkins/workspace/ci-openbmc-2/distro/ubuntu/label/docker-builder/target/p10bmc/build/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share/man 		  --disable-silent-rules 		  --disable-dependency-tracking 		   --without-legacy --with-c14n --without-lzma --with-zlib --disable-static  --enable-ipv6 --with-python=/usr/bin/python3
```

The libxml2 `--with-python` comes from the PACKAGECONFIG in the recipe:
```
meta/recipes-core/libxml/libxml2_2.11.5.bb
33:PACKAGECONFIG[python] = "--with-python=${PYTHON},--without-python,python3"
```


The python3native.bbclass should have been inherited, which would have set PYTHON to the python3-native version:
```
meta/classes-recipe/python3native.bbclass
9:PYTHON="${STAGING_BINDIR_NATIVE}/python3-native/python3"
```
Comment 1 Martin Jansa 2024-01-22 16:16:37 UTC
Works for me with latest oe-core/master

...
NOTE: Running ../libxml2-2.11.5/configure  --build=x86_64-linux                   --host=x86_64-linux             --target=x86_64-linux                   --prefix=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr                 --exec_prefix=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr            --bindir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin             --sbindir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/sbin                   --libexecdir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/libexec             --datadir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share                  --sysconfdir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/etc             --sharedstatedir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/com                 --localstatedir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/var                  --libdir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/lib             --includedir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/include             --oldincludedir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/include                  --infodir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share/info             --mandir=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/share/man               --disable-silent-rules                  --disable-dependency-tracking                    --without-legacy --with-c14n --without-lzma --with-zlib --disable-static  --enable-ipv6 --with-python=/OE/build/luneos-scarthgap/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin/python3-native/python3
...
checking for python version... 3.12
checking for python platform... linux
checking for GNU default python prefix... ${prefix}
checking for GNU default python exec_prefix... ${exec_prefix}
checking for python script directory (pythondir)... ${libdir}/python3.12/site-packages
checking for python extension module directory (pyexecdir)... ${libdir}/python3.12/site-packages
checking for python-3.12... yes

Use bitbake -e or bitbake-getvar to figure out why PYTHON variable is incorrectly set in your build.
Comment 2 Patrick Williams 2024-01-22 16:28:12 UTC
The previous job was ran on CI server, but I recreated it locally with similar behavior.

# $PYTHON
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/bitbake.conf:575
#     "${@sys.executable}"
PYTHON="/usr/lib/python-exec/python3.11/python3"

# $PACKAGECONFIG_CONFARGS [2 operations]
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/classes-global/base.bbclass:18
#     [_defaultval] ""
#   append base.bbclass:474 [appendVar]
#     " --enable-ipv6 --with-python=${PYTHON}"
# pre-expansion value:
#   " --enable-ipv6 --with-python=${PYTHON}"
PACKAGECONFIG_CONFARGS=" --enable-ipv6 --with-python=/usr/lib/python-exec/python3.11/python3"

# $EXTRA_OECONF [8 operations]
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/bitbake.conf:594
#     ""
#   :append /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/distro/include/no-static-libs.inc:29
#     "${DISABLE_STATIC}"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/documentation.conf:166
#     [doc] "Additional configure script options."
#   :append /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/classes-recipe/autotools.bbclass:141
#     " ${PACKAGECONFIG_CONFARGS}"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:54
#     "--without-debug --without-legacy --with-catalog --with-c14n --without-lzma --with-fexceptions"
#   override[class-native]:set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:55
#     "--without-legacy --with-c14n --without-lzma --with-zlib"
#   override[class-nativesdk]:set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:56
#     "--without-legacy --with-c14n --without-lzma --with-zlib"
#   override[linuxstdbase]:set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:57
#     "--with-debug --with-legacy --with-c14n --without-lzma --with-zlib"
# pre-expansion value:
#   "--without-legacy --with-c14n --without-lzma --with-zlib${DISABLE_STATIC} ${PACKAGECONFIG_CONFARGS}"
EXTRA_OECONF="--without-legacy --with-c14n --without-lzma --with-zlib --disable-static  --enable-ipv6 --with-python=/usr/lib/python-exec/python3.11/python3"
Comment 3 Martin Jansa 2024-01-22 16:42:10 UTC
oe-core $ bitbake-getvar -r libxml2-native PYTHON
#
# $PYTHON [3 operations]
#   set /OE/build/oe-core/openembedded-core/meta/conf/bitbake.conf:575
#     "${@sys.executable}"
#   set /OE/build/oe-core/openembedded-core/meta/classes-recipe/python3native.bbclass:9
#     "${STAGING_BINDIR_NATIVE}/python3-native/python3"
#   exported ast.py:52 [eval]
#     [export] "1"
# pre-expansion value:
#   "${STAGING_BINDIR_NATIVE}/python3-native/python3"
export PYTHON="/OE/build/oe-core/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin/python3-native/python3"

oe-core $ bitbake-getvar -r libxml2-native --value BBINCLUDED | sed "s/ /\n/g" | grep python
/OE/build/oe-core/classes-recipe/python3targetconfig.bbclass
/OE/build/oe-core/meta-openembedded/meta-oe/classes-recipe/python3targetconfig.bbclass
/OE/build/oe-core/openembedded-core/meta/classes-recipe/python3targetconfig.bbclass
/OE/build/oe-core/classes-recipe/python3native.bbclass
/OE/build/oe-core/meta-openembedded/meta-oe/classes-recipe/python3native.bbclass
/OE/build/oe-core/openembedded-core/meta/classes-recipe/python3native.bbclass
/OE/build/oe-core/classes-recipe/python3-dir.bbclass
/OE/build/oe-core/meta-openembedded/meta-oe/classes-recipe/python3-dir.bbclass
/OE/build/oe-core/openembedded-core/meta/classes-recipe/python3-dir.bbclass

oe-core $ ls -lah /OE/build/oe-core/meta-openembedded/meta-oe/classes-recipe/python3native.bbclass /OE/build/oe-core/openembedded-core/meta/classes-recipe/python3native.bbclass /OE/build/oe-core/classes-recipe/pyt
hon3native.bbclass
ls: cannot access '/OE/build/oe-core/meta-openembedded/meta-oe/classes-recipe/python3native.bbclass': No such file or directory
ls: cannot access '/OE/build/oe-core/classes-recipe/python3native.bbclass': No such file or directory
-rw-r--r-- 1 martin martin 863 Jul 26 10:36 /OE/build/oe-core/openembedded-core/meta/classes-recipe/python3native.bbclass
Comment 4 Patrick Williams 2024-01-22 17:14:52 UTC
I added some dirty debug statements to bitbake:

--- a/poky/bitbake/lib/bb/parse/parse_py/BBHandler.py
+++ b/poky/bitbake/lib/bb/parse/parse_py/BBHandler.py
@@ -45,6 +45,8 @@ def inherit(files, fn, lineno, d, deferred=False):
     __inherit_cache = d.getVar('__inherit_cache', False) or []
     #if "${" in files and not deferred:
     #    bb.warn("%s:%s has non deferred conditional inherit" % (fn, lineno))
+    if "${" in files:
+        bb.warn("HERE: files = %s, expand = %s, packageconfig = %s" % (files, d.expand(files), d.getVar("PACKAGECONFIG")))
     files = d.expand(files).split()
     for file in files:
         classtype = d.getVar("__bbclasstype", False)

This results in:

WARNING: HERE: files = ${USERADDEXTENSION}, expand = , packageconfig = None
Loading cache...done.
Loaded 4612 entries from dependency cache.
Parsing recipes...done.
Parsing of 2840 .bb files complete (2839 cached, 1 parsed). 4613 targets, 599 skipped, 0 masked, 0 errors.
WARNING: HERE: files = ${USERADDEXTENSION}, expand = , packageconfig = None
NOTE: Resolving any missing task queue dependencies
WARNING: HERE: files = ${GNOMEBASEBUILDCLASS}, expand = autotools, packageconfig = 
WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3targetconfig', '', d)}, expand = , packageconfig = 

But... in the -e output:

# $PACKAGECONFIG [7 operations]
#   :append[pn-qemu-system-native] /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/.build-yosemite/conf/local.conf:219
#     " sdl"
#   :remove[pn-btrfs-tools] /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta-phosphor/conf/distro/include/phosphor-pkg-tweaks.inc:7
#     "python"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/documentation.conf:320
#     [doc] "This variable provides a means of enabling or disabling features of a recipe on a per-recipe basis."
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:32
#     [_defaultval] "python     ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} "
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:33
#     [python] "--with-python=${PYTHON},--without-python,python3"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/recipes-core/libxml/libxml2_2.11.5.bb:34
#     [ipv6] "--enable-ipv6,--disable-ipv6,"
#   override[openbmc-phosphor:class-target]:set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta-phosphor/recipes-core/libxml2/libxml2_%.bbappend:2
#     ""
# pre-expansion value:
#   "python     ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} "
PACKAGECONFIG="python     ipv6 "

Not sure how we end up with `d.expand()` not evaluating properly.
Comment 5 Patrick Williams 2024-01-22 17:16:02 UTC
New data point:

We have a bbappend for this recipe:

PACKAGECONFIG:openbmc-phosphor:class-target = ""


If I remove that then the expand works correctly:

WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3targetconfig', '', d)}, expand = python3targetconfig, packageconfig = python     ipv6 


Something must be wrong in the `expand` call that isn't including the overrides?
Comment 6 Patrick Williams 2024-01-22 17:21:33 UTC
Adding another hint:
        bb.warn("HERE2: %s" % (d.getVar("CLASSOVERRIDE")))

With the bbappend removed:

WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3targetconfig', '', d)}, expand = python3targetconfig, packageconfig = python     ipv6 
WARNING: HERE2: class-target

With the bbappend present:

WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3targetconfig', '', d)}, expand = , packageconfig = 
WARNING: HERE2: class-target


This is a native recipe... how'd we get class-target?

# $CLASSOVERRIDE [4 operations]
#   set? /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/bitbake.conf:815
#     "class-target"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/documentation.conf:111
#     [doc] "An internal variable specifying the special class override that should currently apply (e.g. "class-target", "class-native", and so forth)."
#   set? /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/classes-global/base.bbclass:8
#     "class-target"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/classes-recipe/native.bbclass:113
#     "class-native"
# pre-expansion value:
#   "class-native"
CLASSOVERRIDE="class-native"
Comment 7 Richard Purdie 2024-01-22 19:20:54 UTC
(In reply to Patrick Williams from comment #6)
> Adding another hint:
>         bb.warn("HERE2: %s" % (d.getVar("CLASSOVERRIDE")))
> 
> With the bbappend removed:
> 
> WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python',
> 'python3targetconfig', '', d)}, expand = python3targetconfig, packageconfig
> = python     ipv6 
> WARNING: HERE2: class-target
> 
> With the bbappend present:
> 
> WARNING: HERE: files = ${@bb.utils.contains('PACKAGECONFIG', 'python',
> 'python3targetconfig', '', d)}, expand = , packageconfig = 
> WARNING: HERE2: class-target
> 
> 
> This is a native recipe... how'd we get class-target?
> 
> # $CLASSOVERRIDE [4 operations]
> #   set?
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/
> bitbake.conf:815
> #     "class-target"
> #   set
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/
> documentation.conf:111
> #     [doc] "An internal variable specifying the special class override that
> should currently apply (e.g. "class-target", "class-native", and so forth)."
> #   set?
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/
> classes-global/base.bbclass:8
> #     "class-target"
> #   set
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/
> classes-recipe/native.bbclass:113
> #     "class-native"
> # pre-expansion value:
> #   "class-native"
> CLASSOVERRIDE="class-native"


classes-global/base.bbclass:CLASSOVERRIDE ?= "class-target"

i.e. the default is class-target

and:

classes-recipe/native.bbclass:CLASSOVERRIDE = "class-native"

so native.bbclass sets it later. It is possible to get the target version if you expand before the native class is inherited. That inherit happens right at the of parsing with BBCLASSEXTEND.

The trouble is that there is immediate expansion involved to decide whether to process the inherit. That was the case before inherit_defer and is the case afterwards, the point of the inherit just becomes more deterministic and is later with inherit_defer.


So whilst inherit_defer improves many things, it looks like it still does not interact well with class-XXX overrides in variables.

It isn't a bug in bitbake, it is just unfortunate the way things are working toegther.
Comment 8 Patrick Williams 2024-01-22 20:29:16 UTC
> So whilst inherit_defer improves many things, it looks like it still does not interact well with class-XXX overrides in variables.

Practically speaking, what should be done with libxml2?  If I revert to `inherit` directly the parsing works and with `inherit_defer` it doesn't.  I assume we don't want this to be inherit though.  I'm not following why `inherit_defer` makes things worse in this particular instance.


If I change it back to inherit I get PYTHON correct:

# $PYTHON [3 operations]
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/bitbake.conf:575
#     "${@sys.executable}"
#   set /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/classes-recipe/python3native.bbclass:9
#     "${STAGING_BINDIR_NATIVE}/python3-native/python3"
#   exported ast.py:52 [eval]
#     [export] "1"
# pre-expansion value:
#   "${STAGING_BINDIR_NATIVE}/python3-native/python3"
export PYTHON="/home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/.build-yosemite/tmp/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin/python3-native/python3"
Comment 9 Richard Purdie 2024-01-22 21:26:14 UTC
(In reply to Patrick Williams from comment #8)
> > So whilst inherit_defer improves many things, it looks like it still does not interact well with class-XXX overrides in variables.
> 
> Practically speaking, what should be done with libxml2?  If I revert to
> `inherit` directly the parsing works and with `inherit_defer` it doesn't.  I
> assume we don't want this to be inherit though.  I'm not following why
> `inherit_defer` makes things worse in this particular instance.
> 
> 
> If I change it back to inherit I get PYTHON correct:
> 
> # $PYTHON [3 operations]
> #   set
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/conf/
> bitbake.conf:575
> #     "${@sys.executable}"
> #   set
> /home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/meta/
> classes-recipe/python3native.bbclass:9
> #     "${STAGING_BINDIR_NATIVE}/python3-native/python3"
> #   exported ast.py:52 [eval]
> #     [export] "1"
> # pre-expansion value:
> #   "${STAGING_BINDIR_NATIVE}/python3-native/python3"
> export
> PYTHON="/home/apwillia/local/sync/openbmc-sources/openbmc-config/openbmc/.
> build-yosemite/tmp/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-
> native/usr/bin/python3-native/python3"

I'm not convinced we're comparing the same things before and after, particularly as it now appears there is some bbappend at play.

Can you turn this into some simple reproducer for OE-Core which shows the issue?
Comment 10 Patrick Williams 2024-01-22 21:44:33 UTC
Created attachment 5006 [details]
Patch to aggravate issue.
Comment 11 Patrick Williams 2024-01-22 21:44:53 UTC
Created attachment 5007 [details]
Revert inherit_defer for libxml.
Comment 12 Patrick Williams 2024-01-22 21:47:05 UTC
> Can you turn this into some simple reproducer for OE-Core which shows the issue?

Applying https://bugzilla.yoctoproject.org/attachment.cgi?id=5006 and running `bitbake libxml2-native -ne` results in:

# $PYTHON
#   set /home/apwillia/local/sync/openbmc-sources/openembedded-core/meta/conf/bitbake.conf:575
#     "${@sys.executable}"
PYTHON="/usr/lib/python-exec/python3.11/python3"

Applying https://bugzilla.yoctoproject.org/attachment.cgi?id=5007 on top of that results in:

# $PYTHON [3 operations]
#   set /home/apwillia/local/sync/openbmc-sources/openembedded-core/meta/conf/bitbake.conf:575
#     "${@sys.executable}"
#   set /home/apwillia/local/sync/openbmc-sources/openembedded-core/meta/classes-recipe/python3native.bbclass:9
#     "${STAGING_BINDIR_NATIVE}/python3-native/python3"
#   exported ast.py:52 [eval]
#     [export] "1"
# pre-expansion value:
#   "${STAGING_BINDIR_NATIVE}/python3-native/python3"
export PYTHON="/home/apwillia/local/sync/openbmc-sources/openembedded-core/build/tmp-glibc/work/x86_64-linux/libxml2-native/2.11.5/recipe-sysroot-native/usr/bin/python3-native/python3"
Comment 13 Richard Purdie 2024-01-22 22:29:06 UTC
The patches you attached were the same thing.

Anyhow, adding:

PACKAGECONFIG:remove:class-target = "python"

to the libxml recipe was enough to see what you're referring to. You can swap the inherit_defer back to inherit to restore the old behaviour. It really is just happening to work through luck :/.

With inherit, the class is always inherited even in the target case. This is because at the time the line is evaluated, the PACKAGECONFIG in the bbappend (or at the end of the bb file) hasn't been seen.

The configure command is dynamically adjusted rather than immediately evaluated so it can change to --without-python even though the class was innherited and the DEPENDS were added.

With inherit_defer, the line is parsed after the bbappend or bb but before native.bbclass. This means that it has seen the PACKAGECONFIG line but class-native isn't set yet.

So yes, there is a change of behaviour but it wasn't doing what you thought before. I agree the new behaviour isn't ideal either though. We should be clear that there was a bug even before this though which inherit_defer does improve by not adding the DEPENDS.
Comment 14 Patrick Williams 2024-01-23 15:56:02 UTC
Marking as WONTFIX for now.  I'll just add an explicit inherit in the bbappend on our end, which gets us past the problem, and refer to this issue for reference.

Thank you Richard for the explanation.  I understand now what is going on and why the observed change of behavior.