We occasionally see a segfault in matchbox-windowmanager in libstartup-notification, I think we've seen with systemd and sysvinit. An example is: https://autobuilder.yoctoproject.org/typhoon/#/builders/109/builds/820 which in dmesg has: [ 9.602790] NFSD: starting 90-second grace period (net f0000098) [ 9.734875] matchbox-window[249]: segfault at 8 ip 00007f03a364473c sp 00007fffcfc61620 error 4 in libstartup-notification-1.so.0.0.0[7f03a3644000+4000] [ 9.734894] Code: ee 48 89 df e8 e5 fa ff ff 48 83 c5 01 41 39 6c 24 50 49 89 45 00 7f de 49 8b 7c 24 10 44 89 fe 31 d2 e8 47 fc ff ff 48 89 c7 <8b> 40 08 41 89 44 24 20 e8 17 f9 ff ff 49 8b 7c 24 10 44 89 f6 31 [ 655.381774] hellomod: loading out-of-tree module taints kernel. [ 655.403754] Hello world! [ 655.869939] Cleaning up hellomod. since we keep seeing this I did a little digging. Rerunning that same image on the autobuilder did not show the issue so its some kind of startup race. We did have build information available so I put debug symbols into the image and checked: root@qemux86-64:~# cat /proc/307/maps 55cb908c8000-55cb908cd000 r--p 00000000 fd:00 29506 /usr/bin/matchbox-window-manager 55cb908cd000-55cb908e3000 r-xp 00005000 fd:00 29506 /usr/bin/matchbox-window-manager 55cb908e3000-55cb908e9000 r--p 0001b000 fd:00 29506 /usr/bin/matchbox-window-manager 55cb908ea000-55cb908eb000 r--p 00021000 fd:00 29506 /usr/bin/matchbox-window-manager 55cb908eb000-55cb908ec000 rw-p 00022000 fd:00 29506 /usr/bin/matchbox-window-manager 55cb908ec000-55cb909ae000 rw-p 00000000 00:00 0 [heap] 7ff150753000-7ff1507aa000 r--p 00000000 fd:00 61854 /usr/share/fonts/ttf/LiberationSans-Bold.ttf 7ff1507aa000-7ff1507ae000 rw-p 00000000 00:00 0 7ff1507ae000-7ff1507b5000 r--p 00000000 fd:00 960 /lib/libpthread-2.31.so 7ff1507b5000-7ff1507c4000 r-xp 00007000 fd:00 960 /lib/libpthread-2.31.so 7ff1507c4000-7ff1507c9000 r--p 00016000 fd:00 960 /lib/libpthread-2.31.so 7ff1507c9000-7ff1507ca000 r--p 0001a000 fd:00 960 /lib/libpthread-2.31.so 7ff1507ca000-7ff1507cb000 rw-p 0001b000 fd:00 960 /lib/libpthread-2.31.so 7ff1507cb000-7ff1507cf000 rw-p 00000000 00:00 0 7ff1507cf000-7ff1507d1000 r--p 00000000 fd:00 990 /lib/libuuid.so.1.3.0 7ff1507d1000-7ff1507d5000 r-xp 00002000 fd:00 990 /lib/libuuid.so.1.3.0 7ff1507d5000-7ff1507d6000 r--p 00006000 fd:00 990 /lib/libuuid.so.1.3.0 7ff1507d6000-7ff1507d7000 r--p 00006000 fd:00 990 /lib/libuuid.so.1.3.0 7ff1507d7000-7ff1507d8000 rw-p 00007000 fd:00 990 /lib/libuuid.so.1.3.0 7ff1507d8000-7ff1507da000 r--p 00000000 fd:00 53339 /usr/lib/libXdmcp.so.6.0.0 7ff1507da000-7ff1507dc000 r-xp 00002000 fd:00 53339 /usr/lib/libXdmcp.so.6.0.0 7ff1507dc000-7ff1507de000 r--p 00004000 fd:00 53339 /usr/lib/libXdmcp.so.6.0.0 7ff1507de000-7ff1507df000 r--p 00005000 fd:00 53339 /usr/lib/libXdmcp.so.6.0.0 7ff1507df000-7ff1507e0000 rw-p 00006000 fd:00 53339 /usr/lib/libXdmcp.so.6.0.0 7ff1507e0000-7ff1507e1000 r--p 00000000 fd:00 53327 /usr/lib/libXau.so.6.0.0 7ff1507e1000-7ff1507e2000 r-xp 00001000 fd:00 53327 /usr/lib/libXau.so.6.0.0 7ff1507e2000-7ff1507e3000 r--p 00002000 fd:00 53327 /usr/lib/libXau.so.6.0.0 7ff1507e3000-7ff1507e4000 r--p 00002000 fd:00 53327 /usr/lib/libXau.so.6.0.0 7ff1507e4000-7ff1507e5000 rw-p 00003000 fd:00 53327 /usr/lib/libXau.so.6.0.0 7ff1507e5000-7ff1507f4000 r--p 00000000 fd:00 922 /lib/libm-2.31.so 7ff1507f4000-7ff15088c000 r-xp 0000f000 fd:00 922 /lib/libm-2.31.so 7ff15088c000-7ff150925000 r--p 000a7000 fd:00 922 /lib/libm-2.31.so 7ff150925000-7ff150926000 r--p 0013f000 fd:00 922 /lib/libm-2.31.so 7ff150926000-7ff150927000 rw-p 00140000 fd:00 922 /lib/libm-2.31.so 7ff150927000-7ff150934000 r--p 00000000 fd:00 53782 /usr/lib/libfreetype.so.6.17.2 7ff150934000-7ff1509a9000 r-xp 0000d000 fd:00 53782 /usr/lib/libfreetype.so.6.17.2 7ff1509a9000-7ff1509d1000 r--p 00082000 fd:00 53782 /usr/lib/libfreetype.so.6.17.2 7ff1509d1000-7ff1509d8000 r--p 000a9000 fd:00 53782 /usr/lib/libfreetype.so.6.17.2 7ff1509d8000-7ff1509d9000 rw-p 000b0000 fd:00 53782 /usr/lib/libfreetype.so.6.17.2 7ff1509d9000-7ff1509db000 rw-p 00000000 00:00 0 7ff1509db000-7ff1509e2000 r--p 00000000 fd:00 53770 /usr/lib/libfontconfig.so.1.12.0 7ff1509e2000-7ff150a08000 r-xp 00007000 fd:00 53770 /usr/lib/libfontconfig.so.1.12.0 7ff150a08000-7ff150a20000 r--p 0002d000 fd:00 53770 /usr/lib/libfontconfig.so.1.12.0 7ff150a20000-7ff150a22000 r--p 00044000 fd:00 53770 /usr/lib/libfontconfig.so.1.12.0 7ff150a22000-7ff150a23000 rw-p 00046000 fd:00 53770 /usr/lib/libfontconfig.so.1.12.0 7ff150a23000-7ff150a25000 r--p 00000000 fd:00 53369 /usr/lib/libXrender.so.1.3.0 7ff150a25000-7ff150a2c000 r-xp 00002000 fd:00 53369 /usr/lib/libXrender.so.1.3.0 7ff150a2c000-7ff150a2e000 r--p 00009000 fd:00 53369 /usr/lib/libXrender.so.1.3.0 7ff150a2e000-7ff150a2f000 r--p 0000a000 fd:00 53369 /usr/lib/libXrender.so.1.3.0 7ff150a2f000-7ff150a30000 rw-p 0000b000 fd:00 53369 /usr/lib/libXrender.so.1.3.0 7ff150a30000-7ff150a31000 r--p 00000000 fd:00 53318 /usr/lib/libX11-xcb.so.1.0.0 7ff150a31000-7ff150a32000 r-xp 00001000 fd:00 53318 /usr/lib/libX11-xcb.so.1.0.0 7ff150a32000-7ff150a33000 r--p 00002000 fd:00 53318 /usr/lib/libX11-xcb.so.1.0.0 7ff150a33000-7ff150a34000 r--p 00002000 fd:00 53318 /usr/lib/libX11-xcb.so.1.0.0 7ff150a34000-7ff150a35000 rw-p 00003000 fd:00 53318 /usr/lib/libX11-xcb.so.1.0.0 7ff150a35000-7ff150a38000 r--p 00000000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a38000-7ff150a39000 r-xp 00003000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a39000-7ff150a3a000 r--p 00004000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a3a000-7ff150a3b000 ---p 00005000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a3b000-7ff150a3c000 r--p 00005000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a3c000-7ff150a3d000 rw-p 00006000 fd:00 54510 /usr/lib/libxcb-util.so.1.0.0 7ff150a3d000-7ff150a3e000 r--p 00000000 fd:00 910 /lib/libdl-2.31.so 7ff150a3e000-7ff150a3f000 r-xp 00001000 fd:00 910 /lib/libdl-2.31.so 7ff150a3f000-7ff150a40000 r--p 00002000 fd:00 910 /lib/libdl-2.31.so 7ff150a40000-7ff150a41000 r--p 00002000 fd:00 910 /lib/libdl-2.31.so 7ff150a41000-7ff150a42000 rw-p 00003000 fd:00 910 /lib/libdl-2.31.so 7ff150a42000-7ff150a4e000 r--p 00000000 fd:00 54537 /usr/lib/libxcb.so.1.1.0 7ff150a4e000-7ff150a62000 r-xp 0000c000 fd:00 54537 /usr/lib/libxcb.so.1.1.0 7ff150a62000-7ff150a6b000 r--p 00020000 fd:00 54537 /usr/lib/libxcb.so.1.1.0 7ff150a6b000-7ff150a6c000 r--p 00028000 fd:00 54537 /usr/lib/libxcb.so.1.1.0 7ff150a6c000-7ff150a6d000 rw-p 00029000 fd:00 54537 /usr/lib/libxcb.so.1.1.0 7ff150a6d000-7ff150a6f000 rw-p 00000000 00:00 0 7ff150a6f000-7ff150a72000 r--p 00000000 fd:00 994 /lib/libz.so.1.2.11 7ff150a72000-7ff150a80000 r-xp 00003000 fd:00 994 /lib/libz.so.1.2.11 7ff150a80000-7ff150a86000 r--p 00011000 fd:00 994 /lib/libz.so.1.2.11 7ff150a86000-7ff150a87000 ---p 00017000 fd:00 994 /lib/libz.so.1.2.11 7ff150a87000-7ff150a88000 r--p 00017000 fd:00 994 /lib/libz.so.1.2.11 7ff150a88000-7ff150a89000 rw-p 00018000 fd:00 994 /lib/libz.so.1.2.11 7ff150a89000-7ff150a8f000 r--p 00000000 fd:00 54255 /usr/lib/libpng16.so.16.37.0 7ff150a8f000-7ff150ab3000 r-xp 00006000 fd:00 54255 /usr/lib/libpng16.so.16.37.0 7ff150ab3000-7ff150abe000 r--p 0002a000 fd:00 54255 /usr/lib/libpng16.so.16.37.0 7ff150abe000-7ff150abf000 r--p 00034000 fd:00 54255 /usr/lib/libpng16.so.16.37.0 7ff150abf000-7ff150ac0000 rw-p 00035000 fd:00 54255 /usr/lib/libpng16.so.16.37.0 7ff150ac0000-7ff150ac4000 r--p 00000000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150ac4000-7ff150b07000 r-xp 00004000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150b07000-7ff150b41000 r--p 00047000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150b41000-7ff150b42000 ---p 00081000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150b42000-7ff150b43000 r--p 00081000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150b43000-7ff150b44000 rw-p 00082000 fd:00 54032 /usr/lib/libjpeg.so.62.3.0 7ff150b44000-7ff150b48000 r--p 00000000 fd:00 53351 /usr/lib/libXft.so.2.3.3 7ff150b48000-7ff150b57000 r-xp 00004000 fd:00 53351 /usr/lib/libXft.so.2.3.3 7ff150b57000-7ff150b5b000 r--p 00013000 fd:00 53351 /usr/lib/libXft.so.2.3.3 7ff150b5b000-7ff150b5c000 r--p 00016000 fd:00 53351 /usr/lib/libXft.so.2.3.3 7ff150b5c000-7ff150b5d000 rw-p 00017000 fd:00 53351 /usr/lib/libXft.so.2.3.3 7ff150b5d000-7ff150b82000 r--p 00000000 fd:00 897 /lib/libc-2.31.so 7ff150b82000-7ff150cc4000 r-xp 00025000 fd:00 897 /lib/libc-2.31.so 7ff150cc4000-7ff150d0f000 r--p 00167000 fd:00 897 /lib/libc-2.31.so 7ff150d0f000-7ff150d10000 ---p 001b2000 fd:00 897 /lib/libc-2.31.so 7ff150d10000-7ff150d13000 r--p 001b2000 fd:00 897 /lib/libc-2.31.so 7ff150d13000-7ff150d16000 rw-p 001b5000 fd:00 897 /lib/libc-2.31.so 7ff150d16000-7ff150d1a000 rw-p 00000000 00:00 0 7ff150d1a000-7ff150d1d000 r--p 00000000 fd:00 53333 /usr/lib/libXcursor.so.1.0.2 7ff150d1d000-7ff150d23000 r-xp 00003000 fd:00 53333 /usr/lib/libXcursor.so.1.0.2 7ff150d23000-7ff150d25000 r--p 00009000 fd:00 53333 /usr/lib/libXcursor.so.1.0.2 7ff150d25000-7ff150d26000 r--p 0000a000 fd:00 53333 /usr/lib/libXcursor.so.1.0.2 7ff150d26000-7ff150d27000 rw-p 0000b000 fd:00 53333 /usr/lib/libXcursor.so.1.0.2 7ff150d27000-7ff150d29000 rw-p 00000000 00:00 0 7ff150d29000-7ff150d2b000 r--p 00000000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d2b000-7ff150d2e000 r-xp 00002000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d2e000-7ff150d2f000 r--p 00005000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d2f000-7ff150d30000 ---p 00006000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d30000-7ff150d31000 r--p 00006000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d31000-7ff150d32000 rw-p 00007000 fd:00 53345 /usr/lib/libXfixes.so.3.1.0 7ff150d32000-7ff150d35000 r--p 00000000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d35000-7ff150d39000 r-xp 00003000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d39000-7ff150d3b000 r--p 00007000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d3b000-7ff150d3c000 ---p 00009000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d3c000-7ff150d3d000 r--p 00009000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d3d000-7ff150d3e000 rw-p 0000a000 fd:00 54334 /usr/lib/libstartup-notification-1.so.0.0.0 7ff150d3e000-7ff150d42000 r--p 00000000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d42000-7ff150d5e000 r-xp 00004000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d5e000-7ff150d68000 r--p 00020000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d68000-7ff150d69000 ---p 0002a000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d69000-7ff150d6b000 r--p 0002a000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d6b000-7ff150d6c000 rw-p 0002c000 fd:00 53739 /usr/lib/libexpat.so.1.6.11 7ff150d6c000-7ff150d70000 r--p 00000000 fd:00 53342 /usr/lib/libXext.so.6.4.0 7ff150d70000-7ff150d7b000 r-xp 00004000 fd:00 53342 /usr/lib/libXext.so.6.4.0 7ff150d7b000-7ff150d7f000 r--p 0000f000 fd:00 53342 /usr/lib/libXext.so.6.4.0 7ff150d7f000-7ff150d80000 r--p 00012000 fd:00 53342 /usr/lib/libXext.so.6.4.0 7ff150d80000-7ff150d81000 rw-p 00013000 fd:00 53342 /usr/lib/libXext.so.6.4.0 7ff150d81000-7ff150d9d000 r--p 00000000 fd:00 53321 /usr/lib/libX11.so.6.3.0 7ff150d9d000-7ff150e27000 r-xp 0001c000 fd:00 53321 /usr/lib/libX11.so.6.3.0 7ff150e27000-7ff150ebb000 r--p 000a6000 fd:00 53321 /usr/lib/libX11.so.6.3.0 7ff150ebb000-7ff150ebe000 r--p 00139000 fd:00 53321 /usr/lib/libX11.so.6.3.0 7ff150ebe000-7ff150ec2000 rw-p 0013c000 fd:00 53321 /usr/lib/libX11.so.6.3.0 7ff150ec2000-7ff150ec8000 r--p 00000000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150ec8000-7ff150ed7000 r-xp 00006000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150ed7000-7ff150edb000 r--p 00015000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150edb000-7ff150edc000 ---p 00019000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150edc000-7ff150edd000 r--p 00019000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150edd000-7ff150ede000 rw-p 0001a000 fd:00 54097 /usr/lib/libmb.so.1.0.9 7ff150ede000-7ff150ee0000 rw-p 00000000 00:00 0 7ff150ef1000-7ff150ef3000 r--p 00000000 fd:00 888 /lib/ld-2.31.so 7ff150ef3000-7ff150f12000 r-xp 00002000 fd:00 888 /lib/ld-2.31.so 7ff150f12000-7ff150f1a000 r--p 00021000 fd:00 888 /lib/ld-2.31.so 7ff150f1b000-7ff150f1c000 r--p 00029000 fd:00 888 /lib/ld-2.31.so 7ff150f1c000-7ff150f1d000 rw-p 0002a000 fd:00 888 /lib/ld-2.31.so 7ff150f1d000-7ff150f1e000 rw-p 00000000 00:00 0 7fff07bdc000-7fff07bfd000 rw-p 00000000 00:00 0 [stack] 7fff07c1d000-7fff07c20000 r--p 00000000 00:00 0 [vvar] 7fff07c20000-7fff07c21000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall] (gdb) info symbol 0x7ff150d3573c sn_xcb_display_new + 268 in section .text of /usr/lib/libstartup-notification-1.so.0 (gdb) disassemble /m 0x7ff150d3573c Dump of assembler code for function sn_xcb_display_new: 109 ../../startup-notification-0.12/libsn/sn-common.c: No such file or directory. 110 in ../../startup-notification-0.12/libsn/sn-common.c 111 in ../../startup-notification-0.12/libsn/sn-common.c 112 in ../../startup-notification-0.12/libsn/sn-common.c 113 in ../../startup-notification-0.12/libsn/sn-common.c 114 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35630 <+0>: push %r15 0x00007ff150d35632 <+2>: lea 0x39c7(%rip),%rcx # 0x7ff150d39000 0x00007ff150d35639 <+9>: push %r14 0x00007ff150d3563b <+11>: push %r13 0x00007ff150d3563d <+13>: mov %rsi,%r13 0x00007ff150d35640 <+16>: xor %esi,%esi 0x00007ff150d35642 <+18>: push %r12 0x00007ff150d35644 <+20>: push %rbp 0x00007ff150d35645 <+21>: mov %rdx,%rbp 0x00007ff150d35648 <+24>: mov $0xb,%edx 0x00007ff150d3564d <+29>: push %rbx 0x00007ff150d3564e <+30>: mov %rdi,%rbx 0x00007ff150d35651 <+33>: sub $0x18,%rsp 0x00007ff150d35655 <+37>: callq 0x7ff150d35240 <xcb_intern_atom@plt> 0x00007ff150d3565a <+42>: xor %esi,%esi 0x00007ff150d3565c <+44>: mov $0x17,%edx 0x00007ff150d35661 <+49>: mov %rbx,%rdi 0x00007ff150d35664 <+52>: lea 0x39a1(%rip),%rcx # 0x7ff150d3900c 0x00007ff150d3566b <+59>: mov %eax,%r15d 115 in ../../startup-notification-0.12/libsn/sn-common.c 116 in ../../startup-notification-0.12/libsn/sn-common.c 117 in ../../startup-notification-0.12/libsn/sn-common.c 118 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3566e <+62>: callq 0x7ff150d35240 <xcb_intern_atom@plt> 0x00007ff150d35673 <+67>: xor %esi,%esi 0x00007ff150d35675 <+69>: mov $0x11,%edx 0x00007ff150d3567a <+74>: mov %rbx,%rdi 0x00007ff150d3567d <+77>: lea 0x39a0(%rip),%rcx # 0x7ff150d39024 0x00007ff150d35684 <+84>: mov %eax,%r14d 119 in ../../startup-notification-0.12/libsn/sn-common.c 120 in ../../startup-notification-0.12/libsn/sn-common.c 121 in ../../startup-notification-0.12/libsn/sn-common.c 122 in ../../startup-notification-0.12/libsn/sn-common.c 123 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35687 <+87>: callq 0x7ff150d35240 <xcb_intern_atom@plt> 0x00007ff150d3568c <+92>: mov $0xf,%edx 0x00007ff150d35691 <+97>: xor %esi,%esi 0x00007ff150d35693 <+99>: mov %rbx,%rdi 0x00007ff150d35696 <+102>: lea 0x3999(%rip),%rcx # 0x7ff150d39036 0x00007ff150d3569d <+109>: mov %eax,0x8(%rsp) 124 in ../../startup-notification-0.12/libsn/sn-common.c --Type <RET> for more, q to quit, c to continue without paging-- 125 in ../../startup-notification-0.12/libsn/sn-common.c 126 in ../../startup-notification-0.12/libsn/sn-common.c 127 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356a1 <+113>: callq 0x7ff150d35240 <xcb_intern_atom@plt> 0x00007ff150d356a6 <+118>: mov $0x68,%edi 0x00007ff150d356ab <+123>: mov %eax,0xc(%rsp) 128 in ../../startup-notification-0.12/libsn/sn-common.c 129 in ../../startup-notification-0.12/libsn/sn-common.c 130 in ../../startup-notification-0.12/libsn/sn-common.c 131 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356af <+127>: callq 0x7ff150d353e0 <sn_malloc0@plt> 0x00007ff150d356b4 <+132>: mov %rbx,%rdi 0x00007ff150d356b7 <+135>: mov %rbx,0x10(%rax) 0x00007ff150d356bb <+139>: mov %rax,%r12 132 in ../../startup-notification-0.12/libsn/sn-common.c 133 in ../../startup-notification-0.12/libsn/sn-common.c 134 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356be <+142>: callq 0x7ff150d35440 <xcb_get_setup@plt> 0x00007ff150d356c3 <+147>: mov %rax,%rdi 0x00007ff150d356c6 <+150>: callq 0x7ff150d352f0 <xcb_setup_roots_length@plt> 0x00007ff150d356cb <+155>: mov %eax,0x50(%r12) 135 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356d0 <+160>: cltq 0x00007ff150d356d2 <+162>: lea 0x0(,%rax,8),%rdi 0x00007ff150d356da <+170>: callq 0x7ff150d351e0 <sn_malloc@plt> 0x00007ff150d356df <+175>: mov 0x50(%r12),%edx 0x00007ff150d356e4 <+180>: mov %r13,0x40(%r12) 0x00007ff150d356e9 <+185>: mov %rax,0x18(%r12) 136 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356ee <+190>: movl $0x1,(%r12) 137 in ../../startup-notification-0.12/libsn/sn-common.c 138 in ../../startup-notification-0.12/libsn/sn-common.c 139 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356f6 <+198>: mov %rbp,0x48(%r12) 140 in ../../startup-notification-0.12/libsn/sn-common.c 141 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d356fb <+203>: test %edx,%edx 0x00007ff150d356fd <+205>: jle 0x7ff150d3572a <sn_xcb_display_new+250> 0x00007ff150d356ff <+207>: xor %ebp,%ebp 0x00007ff150d35701 <+209>: jmp 0x7ff150d3570d <sn_xcb_display_new+221> 0x00007ff150d35703 <+211>: nopl 0x0(%rax,%rax,1) 0x00007ff150d35708 <+216>: mov 0x18(%r12),%rax 0x00007ff150d35728 <+248>: jg 0x7ff150d35708 <sn_xcb_display_new+216> 142 in ../../startup-notification-0.12/libsn/sn-common.c --Type <RET> for more, q to quit, c to continue without paging-- 0x00007ff150d3570d <+221>: lea (%rax,%rbp,8),%r13 0x00007ff150d35711 <+225>: mov %ebp,%esi 0x00007ff150d35713 <+227>: mov %rbx,%rdi 0x00007ff150d35716 <+230>: callq 0x7ff150d35200 <xcb_aux_get_screen@plt> 0x00007ff150d3571b <+235>: add $0x1,%rbp 0x00007ff150d3571f <+239>: cmp %ebp,0x50(%r12) 0x00007ff150d35724 <+244>: mov %rax,0x0(%r13) 143 in ../../startup-notification-0.12/libsn/sn-common.c 144 in ../../startup-notification-0.12/libsn/sn-common.c 145 in ../../startup-notification-0.12/libsn/sn-common.c 146 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3572a <+250>: mov 0x10(%r12),%rdi 0x00007ff150d3572f <+255>: mov %r15d,%esi 0x00007ff150d35732 <+258>: xor %edx,%edx 0x00007ff150d35734 <+260>: callq 0x7ff150d35380 <xcb_intern_atom_reply@plt> 0x00007ff150d35739 <+265>: mov %rax,%rdi 147 in ../../startup-notification-0.12/libsn/sn-common.c 148 in ../../startup-notification-0.12/libsn/sn-common.c 149 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3573c <+268>: mov 0x8(%rax),%eax 0x00007ff150d3573f <+271>: mov %eax,0x20(%r12) 150 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35744 <+276>: callq 0x7ff150d35060 <free@plt> 151 in ../../startup-notification-0.12/libsn/sn-common.c 152 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35749 <+281>: mov 0x10(%r12),%rdi 0x00007ff150d3574e <+286>: mov %r14d,%esi 0x00007ff150d35751 <+289>: xor %edx,%edx 0x00007ff150d35753 <+291>: callq 0x7ff150d35380 <xcb_intern_atom_reply@plt> 0x00007ff150d35758 <+296>: mov %rax,%rdi 153 in ../../startup-notification-0.12/libsn/sn-common.c 154 in ../../startup-notification-0.12/libsn/sn-common.c 155 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3575b <+299>: mov 0x8(%rax),%eax 0x00007ff150d3575e <+302>: mov %eax,0x2c(%r12) 156 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35763 <+307>: callq 0x7ff150d35060 <free@plt> 157 in ../../startup-notification-0.12/libsn/sn-common.c 158 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35768 <+312>: mov 0x8(%rsp),%esi 0x00007ff150d3576c <+316>: xor %edx,%edx 0x00007ff150d3576e <+318>: mov 0x10(%r12),%rdi 0x00007ff150d35773 <+323>: callq 0x7ff150d35380 <xcb_intern_atom_reply@plt> 0x00007ff150d35778 <+328>: mov %rax,%rdi --Type <RET> for more, q to quit, c to continue without paging-- 159 in ../../startup-notification-0.12/libsn/sn-common.c 160 in ../../startup-notification-0.12/libsn/sn-common.c 161 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3577b <+331>: mov 0x8(%rax),%eax 0x00007ff150d3577e <+334>: mov %eax,0x28(%r12) 162 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35783 <+339>: callq 0x7ff150d35060 <free@plt> 163 in ../../startup-notification-0.12/libsn/sn-common.c 164 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d35788 <+344>: mov 0x10(%r12),%rdi 0x00007ff150d3578d <+349>: xor %edx,%edx 0x00007ff150d3578f <+351>: mov 0xc(%rsp),%esi 0x00007ff150d35793 <+355>: callq 0x7ff150d35380 <xcb_intern_atom_reply@plt> 0x00007ff150d35798 <+360>: mov %rax,%rdi 165 in ../../startup-notification-0.12/libsn/sn-common.c 166 in ../../startup-notification-0.12/libsn/sn-common.c 167 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d3579b <+363>: mov 0x8(%rax),%eax 0x00007ff150d3579e <+366>: mov %eax,0x24(%r12) 168 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d357a3 <+371>: callq 0x7ff150d35060 <free@plt> 169 in ../../startup-notification-0.12/libsn/sn-common.c 170 in ../../startup-notification-0.12/libsn/sn-common.c 0x00007ff150d357a8 <+376>: add $0x18,%rsp 0x00007ff150d357ac <+380>: mov %r12,%rax 0x00007ff150d357af <+383>: pop %rbx 0x00007ff150d357b0 <+384>: pop %rbp 0x00007ff150d357b1 <+385>: pop %r12 0x00007ff150d357b3 <+387>: pop %r13 0x00007ff150d357b5 <+389>: pop %r14 0x00007ff150d357b7 <+391>: pop %r15 0x00007ff150d357b9 <+393>: retq 0x00007ff150d357ba: nopw 0x0(%rax,%rax,1) End of assembler dump. (where I corrected for the differing load address) so it was likely from sn_xcb_display_new + 268 in section .text of /usr/lib/libstartup-notification-1.so.0 which is https://cgit.freedesktop.org/startup-notification/tree/libsn/sn-common.c#n146
In 165 tests, this only happened once but three have been other cases in the YP AB. Add debug code in case it happens again.
Failure from qa-extras2 step 5c this time: https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1963
Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qa-extras2/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qa-extras2/build/meta/lib/oeqa/runtime/cases/parselogs.py", line 375, in test_parselogs self.assertEqual(errcount, 0, msg=self.msg) AssertionError: 1 != 0 : Log: /home/pokybuild/yocto-worker/qa-extras2/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/target_logs/dmesg_output.log ----------------------- Central error: [ 8.089153] matchbox-window[231]: segfault at 8 ip 00007f6e2896073c sp 00007ffef993a0a0 error 4 in libstartup-notification-1.so.0.0.0[7f6e28960000+4000] *********************** [ 5.692134] systemd[1]: Started udev Kernel Device Manager. [ 5.710011] systemd[1]: Finished udev Coldplug all Devices. [ 5.740845] systemd[1]: Finished Load/Save Random Seed. [ 6.220586] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. [ 6.230456] systemd[1]: Finished Rebuild Dynamic Linker Cache. [ 6.241901] systemd[1]: Started Journal Service. [ 6.254118] systemd-journald[137]: Received client request to flush runtime journal. [ 7.997486] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory [ 7.999978] NFSD: Using legacy client tracking operations. [ 8.005390] NFSD: starting 90-second grace period (net f0000098) [ 8.089153] matchbox-window[231]: segfault at 8 ip 00007f6e2896073c sp 00007ffef993a0a0 error 4 in libstartup-notification-1.so.0.0.0[7f6e28960000+4000] [ 8.089164] Code: ee 48 89 df e8 e5 fa ff ff 48 83 c5 01 41 39 6c 24 50 49 89 45 00 7f de 49 8b 7c 24 10 44 89 fe 31 d2 e8 47 fc ff ff 48 89 c7 <8b> 40 08 41 89 44 24 20 e8 17 f9 ff ff 49 8b 7c 24 10 44 89 f6 31 *********************** 1 errors found in logs. Stdout: Machine information: ******************************* Machine name: qemux86-64 CPU: Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz Arch: x86_64 Physical cores: 1 Logical cores: 1 *******************************
https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/892
startup-notification hasn't seen any commits since 2014: https://cgit.freedesktop.org/startup-notification/log/ and is implementing notifications over X protocol between X clients. Which begs the question: should we take the easy way out and just disable it in oe-core, specifically matchbox? It's all obsolete technology.
startup-notification is only obsolete if you consider X obsolete. The protocol is 'done' but it's a key part of X desktops, its just that mutter/kde don't use libstartupnotification directly.
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2095/steps/8/logs/step5c
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2102/steps/8/logs/step7c
Seems like the error only occurs when systemd is in use? So far all error logs I have seen are either qa-extras2 steps 5/7 or altcfg builds. I have read through the code, and: a) startup notification does not handle errors, and instead proceed directly to accessing a struct member ---> crash (boo!): atom_reply = xcb_intern_atom_reply(display->xconnection, atom_utf8_string_c, NULL); display->UTF8_STRING = atom_reply->atom; free(atom_reply); (atom_reply can be NULL!) b) the only way the error can happen where it does is if the X server can't allocate memory. I suspect that because systemd starts everything at once, there is a brief situation where memory is exhausted. How about bumping RAM again for core-image-sato, and observing for a while if the problem wont appear anymore?
Patch to aid debugging: https://lists.openembedded.org/g/openembedded-core/message/140067
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2108 is most recent.
Alex and I both tried some debugging of this over the weekend. Alex ran 10 builds with 512MB ram and 10 builds with 1024MB whilst there was a master-next build running on the autobuilder. We saw no failures in either set. I also tried running 6 qemu images (mixtures of core-image-sato and core-image-sato-sdk) booted locally, either unbound, or all pinned to a single cpu core whilst a world build of OE-Core was in progress, including a webkit build. No reproduction of the failure either (I have seen it locally once).
Merged http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=27f59b5138d734aa466d7465f9438cd93e9ba861 on the offchance it might help. Debug patch from Alex has been in master-next for a while but has never reproduced with that present.
Closing as this has stopped happening. Might be a weird 32/64 bit issue in time_t, might have been a bug in another piece of the stack that has been since fixed. Will reopen if happens again.
Happening again.
*** Bug 14647 has been marked as a duplicate of this bug. ***
Ross, Moar logging?
Moved en masse from 4.2 to 4.3 by Randy.
dunfell https://autobuilder.yoctoproject.org/typhoon/#/builders/109/builds/7103/steps/14/logs/stdio qemux86-64-alt fedora38-ty-2
Have we seen this recently?
qemux86-64-alt ubuntu2204-vk-2 mathieu/master-next completed at 2025-02-04T08:31:17Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/912/steps/13/logs/stdio
*** Bug 15741 has been marked as a duplicate of this bug. ***
qemux86-alt opensuse155-vk-1 mathieu/master-next completed at 2025-02-13T10:31:34Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/978/steps/14/logs/stdio
qa-extras2 debian12-vk-3 master completed at 2025-04-04T02:19:47Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/1314/steps/23/logs/stdio
qemux86-64-alt debian11-vk-2 master-next completed at 2025-05-14T16:27:29Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/1586/steps/14/logs/stdio Two failures so close in time: this seems a bit strange, maybe these are related to the content of the master-next branch. qemux86-64-alt debian12-vk-9 master-next completed at 2025-05-14T21:13:39Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/1589/steps/14/logs/stdio
qemux86-alt opensuse156-vk-1 master-next completed at 2025-06-02T08:01:48Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/1716/steps/15/logs/stdio
qemux86-alt debian12-vk-5 mathieu/master-next completed at 2025-08-27 08:46:59+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2254/steps/13/logs/stdio
qemux86-alt ubuntu2504-vk-1 mathieu/master-next completed at 2025-09-29 17:14:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2447/steps/14/logs/stdio
qa-extras2 ubuntu2404-vk-2 master-next completed at 2025-11-03 12:37:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2654/steps/24/logs/stdio
qa-extras2 debian12-vk-4 master completed at 2025-12-26 01:34:28+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2914/steps/30/logs/stdio qemux86-alt debian12-vk-4 mathieu/master-next completed at 2025-12-26 17:01:06+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2901/steps/14/logs/stdio
qa-extras2 debian12-vk-5 stable/2.16-nut&stable/whinlatter-nut&whinlatter-next completed at 2026-01-08 07:02:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/2973/steps/24/logs/stdio
qemux86-64-alt rocky9-vk-2 mathieu/master-next completed at 2026-01-08 18:34:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/2957/steps/14/logs/stdio
Definitely notable that the failures are all in -altcfg builds (systemd) or systemd-using builds in qa-extras.
Some hunches: There's long-standing time_t problems in libstartup-notification where it's possible that some client code is using a 32-bit int and chaos follows. See: - https://gitlab.freedesktop.org/xdg/startup-notification/-/merge_requests/1 - https://gitlab.freedesktop.org/xdg/startup-notification/-/issues/6 That doesn't explain why it's only seen on systemd builds. I wondered if the recent race/dependency fix (https://lore.kernel.org/openembedded-core/20251129040503.3958950-3-Qi.Chen@windriver.com/) was related, but we've had reports of this problem after that merged so that isn't the fix. I do wonder if the problem is that the X server restarts on some boots and the clients are no handling that gracefully. Obviously in this case, stopping it restarting is the answer.
qa-extras2 fedora42-vk-1 mathieu/master-next completed at 2026-03-10 16:39:19+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/3310/steps/24/logs/stdio
qemux86-64-alt ubuntu2404-vk-2 mathieu/master-next completed at 2026-04-07 15:22:20+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/3503/steps/14/logs/stdio
qemux86-64-alt ubuntu2404-vk-3 mathieu/master-next completed at 2026-04-09 12:11:42+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/95/builds/3514/steps/14/logs/stdio
qemux86-alt ubuntu2404-vk-3 master completed at 2026-07-12 01:16:25+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4069/steps/15/logs/stdio
Potentially interesting/related: matchbox-panel crashing in libstartup-notification. https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4069
qa-extras2 debian13-vk-1 master-next completed at 2026-07-21 23:48:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/4146/steps/24/logs/stdio
qemux86-alt rocky9-vk-2 mathieu/master-next completed at 2026-07-23 03:10:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4134/steps/15/logs/stdio
I have a lead, one of Ross' theories is looking correct and I have evidence. Whilst https://valkyrie.yocto.io/pub/non-release/20260726-112/artefacts/qemux86-alt/oeqa-target-artefacts-20260726-0kkx0pfa/tests_artifacts.tar.gz doesn't contain any coredump/stacktrace, It does contain both an Xorg.log and an Xorg.log.old. Xorg.log.old spans 4.116 -> 4.561s and ends with: [ 4.559] (II) UnloadModule: "libinput" [ 4.560] (II) UnloadModule: "libinput" [ 4.561] (WW) xf86CloseConsole: KDSETMODE failed: Input/output error [ 4.561] (WW) xf86CloseConsole: VT_GETMODE failed: Input/output error [ 4.561] (WW) xf86OpenConsole: VT_GETSTATE failed: Input/output error [ 4.561] (II) Server terminated successfully (0). Closing log file. and then Xorg.log starts at 4.947s. Looksing at the failed build: https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4163 8[ 4.790969] matchbox-panel[355]: segfault at 8 ip b56237d2 sp bfc46880 error 4 in libstartup-notification-1.so.0.0.0[37d2,b5623000+5000] likely on CPU 1 (core 1, socket 0) [ 4.797202] Code: 08 55 83 c5 01 ff 74 24 10 e8 0a fa ff ff 83 c4 10 39 6e 30 89 07 7f e0 8b 6c 24 10 83 ec 04 6a 00 55 ff 76 08 e8 9e fb ff ff <8b> 50 08 89 56 10 89 04 24 e8 10 f9 ff ff 83 c4 0c 6a 00 ff 74 24 i.e. the crash is neatly between xserver restarts.
qemux86-alt ubuntu2510-vk-1 master&master-next completed at 2026-07-26 18:45:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4163/steps/15/logs/stdio
qa-extras2 debian12-vk-6 wrynose completed at 2026-07-27 13:16:29+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/4188
qemux86-alt debian12-vk-7 master&master-next completed at 2026-08-18 13:41:27+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4359/steps/17/logs/stdio
qemux86-alt debian12-vk-7 mathieu/master-next completed at 2026-08-20 18:41:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4383/steps/16/logs/stdio
qemux86-alt ubuntu2204-vk-1 master-next&master completed at 2026-08-30 08:53:13+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4466/steps/17/logs/stdio
qemux86-alt debian12-vk-5 master completed at 2026-09-25 02:05:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/4647/steps/17/logs/stdio
qa-extras2 debian13-vk-2 mathieu/master-next completed at 2026-10-18 14:09:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/26/builds/4808/steps/26/logs/stdio
It looks like xserver typically restarts during boot, I suspect the crash is when some bits of the X session have started already and don't handle the xserver disappearing. Unsetting SPLASH, to remove psplash, appears to remove the restart. This suggests a problem with the handoff from psplash to xserver.