Bug 15638

Summary: bitbake-getvar doesn't have identical output to bitbake -e
Product: [Build System, Metadata & Runtime] BitBake Reporter: Ross Burton <ross.burton>
Component: bitbakeAssignee: Yoann Congal <yoann.congal>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: adrian.freihofer, poky.bs.watcher, poky.watcher, randy.macleod, yoann.congal
Version: 5.1   
Target Milestone: 5.2 M1   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15652
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Ross Burton 2024-10-29 14:55:08 UTC
$ bitbake-getvar DISTRO_FEATURES   --value
acl alsa bluetooth debuginfod ext2 ipv4 ipv6 pcmcia usbgadget usbhost wifi xattr nfs zeroconf pci 3g nfc x11 vfat seccomp opengl ptest multiarch wayland vulkan sysvinit

$ bitbake -e |grep DISTRO_FEATURES=
DISTRO_FEATURES="acl alsa bluetooth debuginfod ext2 ipv4 ipv6 pcmcia usbgadget usbhost wifi xattr nfs zeroconf pci 3g nfc x11 vfat seccomp opengl ptest multiarch wayland vulkan sysvinit pulseaudio gobject-introspection-data ldconfig"

Notably, the backfill hasn't happened.
Comment 1 Adrian 2024-10-29 15:44:23 UTC
It's also interesting that bitbake-getvar behaves differently if called with -r or without (as in the example here). With -r it's more verbose than without.
Comment 2 Richard Purdie 2024-10-29 17:46:27 UTC
cooker.py in the -e codepath does:

            data.expandKeys(envdata)
            parse.ast.runAnonFuncs(envdata)

on the data. Adding the expandKeys is straightforward enough for bitbake-getvar but the runAnonFuncs will need a new API call adding for tinfoil to use.
Comment 4 Yoann Congal 2024-11-14 18:28:18 UTC
(In reply to Yoann Congal from comment #3)
> Patches sent:
> https://lists.openembedded.org/g/bitbake-devel/message/16793
> https://lists.openembedded.org/g/bitbake-devel/message/16794
> https://lists.openembedded.org/g/openembedded-core/message/207069

Those patches make runqemu.QemuTest.test_qemu_can_boot_nfs_and_shutdown fail.
Reverting the getvar patch fix the issue. I can reproduce locally, I'll investigate.

Idea from #yp-infra:
[23:18:41] <RP> yocton: I think something in the testing code uses bitbake-getvar to get a value and your fix is breaking it!
[23:19:11] <yocton> Same idea. I'll try to locate this getvar call
Comment 6 Yoann Congal 2024-11-14 22:25:21 UTC
v2 sent

[PATCH v2 1/2] tinfoil: add new "finalizeData" API
https://lists.openembedded.org/g/bitbake-devel/message/16801
[PATCH v2 2/2] bitbake-getvar: use finalizeData tinfoil API to get identical result to "bitbake -e"
https://lists.openembedded.org/g/bitbake-devel/message/16802
Comment 7 Yoann Congal 2024-11-19 15:41:17 UTC
Merged:

bitbake: tinfoil: add new "finalizeData" API - poky - Poky Build Tool and Metadata
https://git.yoctoproject.org/poky/commit/?id=30a3ac4a2e81cb13c3113e15b38444490450a70f
bitbake: bitbake-getvar: use finalizeData tinfoil API to get identical result to "bitbake -e" - poky - Poky Build Tool and Metadata
https://git.yoctoproject.org/poky/commit/?id=4be9d612252286b4f920adb729d28b28e2372dc4
oeqa/selftest: add a test for bitbake "-e" and "-getvar" difference - poky - Poky Build Tool and Metadata
https://git.yoctoproject.org/poky/commit/?id=131872001abf6c76bd1ed7e31cc68c88ede8ca2c