Bug 16078

qemux86-world-alt fedora39-vk-1 mathieu/master-next completed at 2025-11-26 11:12:43+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/17/builds/2617/steps/12/logs/stdio

Let's see if it happens again.

This time it's on dnf recipe
qemux86-world rocky9-vk-1 mathieu/master-next completed at 2026-01-29 17:06:13+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/59/builds/3109/steps/12/logs/stdio

And now on pulseaudio
qemux86-64-alt rocky9-vk-2 master completed at 2026-02-04 02:02:46+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/3089/steps/14/logs/stdio

Very odd that this would segfault so raise to a High.
It could be related to:
16058 	AB-INT: rust do_test_compile/do_install segfault

I do not think it's related to rust.

opkg-build is a bash script, and line 338 is:

( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )

The last failure also prints a bit more:

Subprocess output:malloc(): smallbin double linked list corrupted

I don't see a possibility to investigate this further right now. The artifacts from the last fail have long been removed.

We need the core dump to at least see what executable is crashing exactly. So I'd wait until that happens again, and get the coredump ASAP when it does.

Other suggestions welcome!

So the crash seems to be happening in a 'find' executable from the host - probably a rarely triggered bug in older versions of findutils? 

RHEL8 derivatives use buildtools, which has newer findutils from oe-core, but it would be good also to collect more data points.

An urgent AB-INT issue but rare enough that it won't block 6.0-M2,
so moving to M3. 
- YP bug triage team

I'd note these are in pseudo context so this could be the pseudo intercepts breaking something somehow

Paul to investigate while Alex is away.

Paul was not able to reproduce the problem.
Nothing in the pseudo changes jumped out as being related.

It hasn't happened again so lower priority to M+ and move to M4.
Paul has done lots of investigation.
Let's see if it happens again.

Bulk move of unassigned 6.0-M* bugs to 6.1-M2.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/3906 - I did ssh in and rerun the command and the vte recipe tasks completed successfully (not from sstate). Even with the failed build, it doesn't reproduce.

[rpurdie@rocky9-vk-1 ~]$ coredumpctl gdb 4110612
           PID: 4110612 (opkg-build)
           UID: 6000 (pokybuild)
           GID: 6000 (pokybuild)
        Signal: 11 (SEGV)
     Timestamp: Mon 2026-06-15 12:07:09 UTC (53min ago)
  Command Line: /bin/bash /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/work/cortexa57-poky-linux/vte/0.82.2/recipe-sysroot-native/usr/bin/opkg-build -Z zstd -a $'--threads=8 -3' vte-locale-en-ca /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/work/cortexa57-poky-linux/vte/0.82.2/deploy-ipks/cortexa57
    Executable: /usr/bin/bash
 Control Group: /system.slice/buildbot-worker.service
          Unit: buildbot-worker.service
         Slice: system.slice
       Boot ID: b70b446282d74961b3dbd8215a4b7f05
    Machine ID: 89b51523a00d46d8a1522245f0a8cd29
      Hostname: rocky9-vk-1
       Storage: /var/lib/systemd/coredump/core.opkg-build.6000.b70b446282d74961b3dbd8215a4b7f05.4110612.1781525229000000.zst (inaccessible)
       Message: Process 4110612 (opkg-build) of user 6000 dumped core.
                
                Stack trace of thread 4110612:
                #0  0x00007fa93289a4d4 _int_malloc (libc.so.6 + 0x9a4d4)
                #1  0x00007fa93289b1f2 __libc_malloc (libc.so.6 + 0x9b1f2)
                #2  0x00007fa932c1a824 pseudo_setupenvp (libpseudo.so + 0x44824)
                #3  0x00007fa932bec07f n/a (libpseudo.so + 0x1607f)
                #4  0x00007fa932bf0f83 execve (libpseudo.so + 0x1af83)
                #5  0x000055b0ac93aeb6 shell_execve (bash + 0x52eb6)
                #6  0x000055b0ac941e3e execute_disk_command.lto_priv.0 (bash + 0x59e3e)
                #7  0x000055b0ac932587 execute_simple_command (bash + 0x4a587)
                #8  0x000055b0ac9345aa execute_command_internal (bash + 0x4c5aa)
                #9  0x000055b0ac9375d6 execute_pipeline (bash + 0x4f5d6)
                #10 0x000055b0ac935d74 execute_command_internal (bash + 0x4dd74)
                #11 0x000055b0ac936748 execute_command (bash + 0x4e748)
                #12 0x000055b0ac936121 execute_command_internal (bash + 0x4e121)
                #13 0x000055b0ac936e3b execute_in_subshell (bash + 0x4ee3b)
                #14 0x000055b0ac933ca4 execute_command_internal (bash + 0x4bca4)
                #15 0x000055b0ac936748 execute_command (bash + 0x4e748)
                #16 0x000055b0ac928109 reader_loop (bash + 0x40109)
                #17 0x000055b0ac91997e main (bash + 0x3197e)
                #18 0x00007fa93282a610 __libc_start_call_main (libc.so.6 + 0x2a610)
                #19 0x00007fa93282a6c0 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2a6c0)
                #20 0x000055b0ac919d75 _start (bash + 0x31d75)
                ELF object binary architecture: AMD x86-64

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/work/cortexa57-poky-linux/vte/0.82.2/recipe-sysroot-native/usr/bin/opkg-build /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/work/cortexa57-poky-linux/vte/0.82.2/recipe-sysroot-native/usr/bin/opkg-build -Z zstd -a --threads=8\ -3 vte-locale-en-ca /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/work/cortexa57-poky-linux/vte/0.82.2/deploy-ipks/cortexa57'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fa93289a4d4 in _int_malloc (av=av@entry=0x7fa9329fbca0 <main_arena>, bytes=bytes@entry=24) at malloc.c:3800
3800			      bck->fd = bin;
(gdb) l
3795			      bck = tc_victim->bk;
3796			      set_inuse_bit_at_offset (tc_victim, nb);
3797			      if (av != &main_arena)
3798				set_non_main_arena (tc_victim);
3799			      bin->bk = bck;
3800			      bck->fd = bin;
3801	
3802			      tcache_put (tc_victim, tc_idx);
3803		            }
3804

(gdb) i locals
tc_victim = 0x55b0aceafc40
tc_idx = 0
p = <optimized out>
nb = 32
idx = 2
bin = 0x7fa9329fbd10 <main_arena+112>
victim = 0x55b0aceb11e0
size = <optimized out>
victim_index = <optimized out>
remainder = <optimized out>
remainder_size = <optimized out>
block = <optimized out>
bit = <optimized out>
map = <optimized out>
fwd = <optimized out>
bck = 0x6f732e6f
tcache_unsorted_count = <optimized out>
tcache_nb = <optimized out>
tc_idx = <optimized out>
return_cached = <optimized out>
__PRETTY_FUNCTION__ = "_int_malloc"
(gdb) f
#0  0x00007fa93289a4d4 in _int_malloc (av=av@entry=0x7fa9329fbca0 <main_arena>, bytes=bytes@entry=24) at malloc.c:3800
3800			      bck->fd = bin;
(gdb) f 2
#2  0x00007fa932c1a824 in pseudo_setupenvp () from /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so
(gdb) i locals
No symbol table info available.
(gdb) f 1
#1  0x00007fa93289b1f2 in __GI___libc_malloc (bytes=24) at malloc.c:3184
3184	      victim = tag_new_usable (_int_malloc (&main_arena, bytes));
(gdb) i locals
ar_ptr = <optimized out>
victim = <optimized out>
tbytes = <optimized out>
tc_idx = <optimized out>
__PRETTY_FUNCTION__ = "__libc_malloc"

We can obtain valgrind debug with:

diff --git a/meta/recipes-support/vte/vte_0.82.2.bb b/meta/recipes-support/vte/vte_0.82.2.bb
index d5dced4ce6..56f4b4503f 100644
--- a/meta/recipes-support/vte/vte_0.82.2.bb
+++ b/meta/recipes-support/vte/vte_0.82.2.bb
@@ -63,3 +63,5 @@ FILES:${PN}-dev += "${datadir}/glade/"
 # qemu: uncaught target signal 6 (Aborted) - core dumped
 # https://gitlab.gnome.org/GNOME/vte/-/issues/2910
 GI_DATA_ENABLED:toolchain-clang:arm = "False"
+
+OPKGBUILDCMD:prepend = "/usr/bin/valgrind --leak-check=yes --show-leak-kinds=all  --track-origins=yes --verbose --log-file=/tmp/rp.log " 

https://valkyrie.yocto.io/pub/shared-failure-data/pseudo-valgrind.log

Valgrind, asan or glibc malloc debug is the way to go. We may have better luck with asan if we disable looking for leaks and focus on use-after-free/double-free.

Throwing Claude at this produced some pointers to what's happening but not a root cause:

  - Faulting alloc is malloc(24) = "LD_PRELOAD=libpseudo.so", i.e. the scrubbed-environment branch of pseudo_setupenvp (child exec'd with no LD_PRELOAD). Branch is correctly sized — not an overflow there.
  - Corrupted 32-byte smallbin chunk: fd valid (main_arena+112), bk = 0x6f732e6f = "o.so" — the tail of a freed "libpseudo.so" string. Adjacent size/fd intact ⇒ not a buffer overflow.
  - The stale ASCII in bk is the fingerprint of a double-free / use-after-free of a 32-byte-class object: freed into the fastbin (writes fd only, leaves bk), then the duplicate gets onto a doubly-linked bin where the smallbin→tcache refill
  loop dereferences the stale bk.

Rarity: needs scrubbed-LD_PRELOAD branch + tcache(32) full so free spills to fastbin + a second free before reuse + later same-size malloc. Layout/glibc-version dependent ⇒ RHEL/F39 only.

  Open: locate the actual second free/UAF (small 32-byte object, not yet found by static review).

  Minor bug found en route (fix regardless): pseudo_client_path_set does free(slot) then strdup(path); UAF if path aliases the slot (reachable via OP_DUP with dirfd==fd, e.g. dup2(fd,fd)).

The root cause is probably bash and pseudo arguing over the environ array.

Richard shared a valgrind log containing the following:

Invalid read of size 8
  at 0x49AF085: memmove (vg_replace_strmem.c:1415)
  by 0x49FB548: pseudo_setupenvp (in /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so)
  by 0x49CD07E: ??? (in /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so)
  by 0x49D1F82: execve (in /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so)
  by 0x4052EB5: shell_execve (execute_cmd.c:5817)
  by 0x4059E3D: execute_disk_command.lto_priv.0 (execute_cmd.c:5609)
  by 0x404A586: execute_simple_command (execute_cmd.c:4680)
  by 0x404C5A9: execute_command_internal (execute_cmd.c:858)
  by 0x404F5D5: execute_pipeline (execute_cmd.c:2555)
  by 0x404DD73: execute_connection (execute_cmd.c:2739)
  by 0x404DD73: execute_command_internal (execute_cmd.c:1032)
  by 0x404E747: execute_command (execute_cmd.c:399)
  by 0x404E120: execute_connection (execute_cmd.c:2805)
  by 0x404E120: execute_command_internal (execute_cmd.c:1032)
Address 0x4d9eca0 is 0 bytes inside a block of size 132 free'd
  at 0x49A4B4C: free (vg_replace_malloc.c:990)
  by 0x405F35C: strvec_flush (stringvec.c:89)
  by 0x405F35C: strvec_flush (stringvec.c:80)
  by 0x405F35C: maybe_make_export_env.part.0 (variables.c:5033)
  by 0x4049ED2: maybe_make_export_env (variables.c:5030)
  by 0x4049ED2: execute_simple_command (execute_cmd.c:4325)
  by 0x404C5A9: execute_command_internal (execute_cmd.c:858)
  by 0x404F5D5: execute_pipeline (execute_cmd.c:2555)
  by 0x404DD73: execute_connection (execute_cmd.c:2739)
  by 0x404DD73: execute_command_internal (execute_cmd.c:1032)
  by 0x404E747: execute_command (execute_cmd.c:399)
  by 0x404E120: execute_connection (execute_cmd.c:2805)
  by 0x404E120: execute_command_internal (execute_cmd.c:1032)
  by 0x404EE3A: execute_in_subshell (execute_cmd.c:1697)
  by 0x404BCA3: execute_command_internal (execute_cmd.c:670)
  by 0x404E747: execute_command (execute_cmd.c:399)
  by 0x4040108: reader_loop (eval.c:171)
Block was alloc'd at
  at 0x49A182F: malloc (vg_replace_malloc.c:447)
  by 0x4A9EB3E: __add_to_environ (setenv.c:251)
  by 0x49B0F9F: setenv (vg_replace_strmem.c:2528)
  by 0x49FB272: pseudo_setupenv (in /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so)
  by 0x49D2397: fork (in /srv/pokybuild/yocto-worker/qemuarm64-alt/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so)
  by 0x4064FA3: make_child (jobs.c:2175)
  by 0x404BC66: execute_command_internal (execute_cmd.c:646)
  by 0x404E747: execute_command (execute_cmd.c:399)
  by 0x4040108: reader_loop (eval.c:171)
  by 0x403197D: main (shell.c:821)


We can see free() called on the memory buffer from bash code, and then it is later accessed in pseudo_setupenvp(). There are other similar cases, one of them is likely to result in the malloc corruption which leads to the segfault.

The question is, what do we do about this? We probably need to separate environ handling when the process we're attached to is bash. We already have some workarounds for this, but they're clearly not enough.

qemuarm64-alt rocky9-vk-1 master&master-next completed at 2026-06-15 12:12:24+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/9/builds/3906/steps/15/logs/stdio

qemux86-64-alt alma9-vk-2 master completed at 2026-07-11 01:59:24+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/4059/steps/15/logs/stdio

Fixed with the exec changes in pseudo, pulled in with https://git.openembedded.org/openembedded-core/commit/

(In reply to Richard Purdie from comment #23)
> Fixed with the exec changes in pseudo, pulled in with
> https://git.openembedded.org/openembedded-core/commit/

For the records, a more permanent link:
pseudo: Add in openat2, exec and linkat fixes
https://git.openembedded.org/openembedded-core/commit/?id=90f823defa32477c9dbd91d264f581fdf0ee4068