Bug 16281

Summary: AB-INT: File exists: libglib-2.0.so.0.8800.0-gdb.py
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Mathieu Dubois-Briand <mathieu.dubois-briand>
Component: coreAssignee: Yoann Congal <yoann.congal>
Status: RESOLVED FIXED QA Contact: Richard Purdie <richard.purdie>
Severity: normal    
Priority: Medium+ CC: meta.mr.watcher, meta.watcher, paul, richard.purdie, yoann.congal
Version: 5.99   
Target Milestone: 6.0.2   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Mathieu Dubois-Briand 2026-05-11 09:35:09 UTC
We got this intermittent failure a few times on the autobuilder. First in my branch, then on Richard branch.
It never occurred on master so far, but we haven't found the commit introducing this issue so far, so we fear it might get merged at some point.

2026-05-02 22:40:48,184 - oe-selftest - INFO - runtime_test.Postinst.test_postinst_rootfs_and_boot_sysvinit (subunit.RemotedTestCase)
2026-05-02 22:40:48,189 - oe-selftest - INFO -  ... FAIL
...
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:
...
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/classes-global/staging.bbclass', lineno: 621, function: extend_recipe_sysroot
     0617:                    if "/bin/" in l or "/sbin/" in l:
     0618:                        # defer /*bin/* files until last in case they need libs
     0619:                        binfiles[l] = (targetdir, dest)
     0620:                    else:
 *** 0621:                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
     0622:
     0623:    # Handle deferred binfiles
     0624:    for l in binfiles:
     0625:        (targetdir, dest) = binfiles[l]
File: '/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/classes-global/staging.bbclass', lineno: 165, function: staging_copyfile
     0161:        os.symlink(linkto, dest)
     0162:        #bb.warn(c)
     0163:    else:
     0164:        try:
 *** 0165:            os.link(c, dest)
     0166:        except OSError as err:
     0167:            if err.errno == errno.EXDEV:
     0168:                bb.utils.copyfile(c, dest)
     0169:            else:
Exception: FileExistsError: [Errno 17] File exists: '/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1165004/tmp/sysroots-components/aarch64/glib-2.0-native/usr/share/gdb/auto-load/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1164956/tmp/work/aarch64-linux/glib-2.0-native/2.88.0/recipe-sysroot-native/usr/lib/libglib-2.0.so.0.8800.0-gdb.py' -> '/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1165004/tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/recipe-sysroot-native/usr/share/gdb/auto-load/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1164956/tmp/work/aarch64-linux/glib-2.0-native/2.88.0/recipe-sysroot-native/usr/lib/libglib-2.0.so.0.8800.0-gdb.py'
Comment 1 Mathieu Dubois-Briand 2026-05-11 09:36:23 UTC
oe-selftest-armhost ubuntu2404-vk-arm2 master-next&master completed at 2026-05-02 22:57:18+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3888/steps/15/logs/stdio

multilib fedora43-vk-2 master-next&master completed at 2026-05-02 16:40:39+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3680/steps/24/logs/stdio

multilib alma8-vk-2 mathieu/master-next&mathieu/master-next-tests completed at 2026-04-21 18:02:19+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3616/steps/25/logs/stdio

multilib alma8-vk-2 mathieu/master-next completed at 2026-04-22 00:24:20+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3618/steps/25/logs/stdio

multilib opensuse160-vk-1 mathieu/master-next&mathieu/master-next-tests completed at 2026-04-22 08:40:27+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3621/steps/24/logs/stdio

multilib ubuntu2604-vk-2 mathieu/master-next completed at 2026-04-24 11:26:53+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3646/steps/24/logs/stdio

multilib debian12-vk-7 mathieu/master-next-tests&mathieu/master-next completed at 2026-04-26 19:17:22+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3652/steps/24/logs/stdio
Comment 2 Richard Purdie 2026-05-13 16:14:49 UTC
I debugged this failure a bit and you need a few things to happen. The first is to have Adams "extend_recipe_sysroot" series included, which means changes in PACKAGE_CLASSES cause do_fetch for recipes to rerun.

The second thing required is a recipe like glib-2.0-native which is accidentally encoding build paths into it's output. In this case, the gdb plguins are installed into ${D}{datadir}/gdb/ but have the full host install path included after that. This looks ugly and wrong but is probably correct for the way the plugins are meant to work.

Whilst we detect and remap path references in files, we don't detect the build path being encoded into the target install directory structure. We probably should.

That still leaves the question about how that causes the sstate logic to break.

I suspect what happens is we have one local build of glib-2.0-native which has a given sstate taskhash, then, the build swaps over to an install of that sstate object from SSTATE_DIR. Since the build path is encoded there, you can have two sstate objects with the same taskhash which have two differing sets of path contents as the build paths were different.

The sstate code will probably treat the two as equivalent when the contents was not, leading to the manifest mismatches.

We can stop installing these files in glib-2.0-native, bit that does leave a few other open questions...
Comment 3 Mathieu Dubois-Briand 2026-05-15 08:53:29 UTC
multilib ubuntu2204-vk-4 master-next completed at 2026-05-13 15:15:57+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/92/builds/3745/steps/24/logs/stdio

oe-selftest-armhost ubuntu2504-vk-arm1 master-next completed at 2026-05-13 21:34:17+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3950/steps/15/logs/stdio

pkgman-non-rpm debian11-vk-3 master-next completed at 2026-05-13 18:50:40+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/67/builds/3787/steps/21/logs/stdio
Comment 4 Yoann Congal 2026-05-19 14:36:36 UTC
Merged patches related to this bug:
glib-2.0-native: Remove problematic path reference
https://git.openembedded.org/openembedded-core/commit/?id=117ed64803dc2c8176125835b2bbcc08225f45da
sstate: Improve confusing debug message
https://git.openembedded.org/openembedded-core/commit/?id=47136a5068245348bbe186a40d4b6b7d0664e22d
sstate: Detect broken sstate paths containing tmpdir
https://git.openembedded.org/openembedded-core/commit/?id=907af8fb448e2f9ecf8e0439f2d8c7c397fb873f

@RP: What do you think about backporting those to Wrynose?
Comment 5 Yoann Congal 2026-05-21 15:00:09 UTC
(In reply to Yoann Congal from comment #4)
> @RP: What do you think about backporting those to Wrynose?

Note from bug triage call: bug is rare but hard to debug, check has low chances of false positive. So, let's backport the 3 patches to wrynose.