oe-selftest-debian ubuntu2204-vk-1 wrynose still running https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4334 Hanged with: 2026-07-20 22:50:13,082 - oe-selftest - INFO - 13: 67/67 639/676 (29.32s) (0 failed) (wic.Wic2.test_wic_image_type) 2026-07-21 00:13:33,083 - oe-selftest - INFO - Keepalive message 2026-07-21 01:36:53,083 - oe-selftest - INFO - Keepalive message 2026-07-21 03:00:13,083 - oe-selftest - INFO - Keepalive message 2026-07-21 04:23:33,084 - oe-selftest - INFO - Keepalive message 2026-07-21 05:46:53,084 - oe-selftest - INFO - Keepalive message 2026-07-21 07:10:13,084 - oe-selftest - INFO - Keepalive message 2026-07-21 08:33:33,084 - oe-selftest - INFO - Keepalive message Note: test_wic_image_type was the last test to succeed. When investigating the worker: $ pstree -ap ├─buildbot-worker,6927 /srv/pokybuild/buildbot-venv/bin/buildbot-worker start --nodaemon /srv/pokybuild/yocto-worker │ ├─python3,41934 /srv/pokybuild/yocto-worker/oe-selftest-debian/yocto-autobuilder-helper/scripts/run-config oe-selftest-debian ... │ │ └─python3,41935 /srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/scripts/oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg ... │ │ ├─(KnottyUI,41983) │ │ ├─(KnottyUI,41985) │ │ ├─(KnottyUI,41986) │ │ ├─(KnottyUI,41992) │ │ ├─(KnottyUI,41996) │ │ ├─(KnottyUI,41999) │ │ ├─(KnottyUI,42003) │ │ ├─(KnottyUI,42007) │ │ ├─(KnottyUI,42014) │ │ ├─(KnottyUI,42028) │ │ ├─(KnottyUI,42034) │ │ ├─(KnottyUI,42044) │ │ ├─(KnottyUI,42059) │ │ ├─(KnottyUI,42071) │ │ ├─python3,41984 /srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/scripts/oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg ... │ │ │ ├─sh,865316 -c bitbake -c clean selftest-ed-native │ │ │ │ └─python3,865317 /srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/bin/bitbake -c clean selftest-ed-native │ │ │ │ ├─{python3},865729 │ │ │ │ └─{python3},899872 │ │ │ └─{python3},865320 │ │ ├─{python3},41961 │ │ └─{python3},42091 │ └─{buildbot-worker},6931 The python process 865317 (bitbake -c clean selftest-ed-native) is hanged. Here is the backtraces of the threads: (buildbot-venv) pokybuild@ubuntu2204-vk-1:~/yocto-worker/oe-selftest-debian/build$ gdb -p 865317 (gdb) thread 1 [Switching to thread 1 (Thread 0x7fe09a3fc000 (LWP 865317))] #0 __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7fe094000b70) at ./nptl/futex-internal.c:57 (gdb) py-bt Traceback (most recent call first): File "/usr/lib/python3.10/threading.py", line 1567, in _shutdown lock.acquire() (gdb) thread 2 [Switching to thread 2 (Thread 0x7fe09881d640 (LWP 865729))] #0 0x00007fe09a515c4f in __GI___poll (fds=0x7fe09a09e2e0, nfds=1, timeout=timeout@entry=250) at ../sysdeps/unix/sysv/linux/poll.c:29 (gdb) py-bt Traceback (most recent call first): File "/usr/lib/python3.10/selectors.py", line 416, in select fd_event_list = self._selector.poll(timeout) File "/usr/lib/python3.10/multiprocessing/connection.py", line 931, in wait ready = selector.select(timeout) File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/server/process.py", line 799, in wait return multiprocessing.connection.wait([self.reader], timeout) File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/server/process.py", line 780, in startCallbackHandler ready = self.reader.wait(0.25) File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap self._bootstrap_inner() (gdb) thread 3 [Switching to thread 3 (Thread 0x7fe093fff640 (LWP 899872))] #0 0x00007fe09a515c4f in __GI___poll (fds=0x7fe09a09e2f0, nfds=1, timeout=timeout@entry=250) at ../sysdeps/unix/sysv/linux/poll.c:29 29 in ../sysdeps/unix/sysv/linux/poll.c (gdb) py-bt Traceback (most recent call first): File "/usr/lib/python3.10/selectors.py", line 416, in select fd_event_list = self._selector.poll(timeout) File "/usr/lib/python3.10/multiprocessing/connection.py", line 931, in wait ready = selector.select(timeout) File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/server/process.py", line 799, in wait return multiprocessing.connection.wait([self.reader], timeout) File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/server/process.py", line 780, in startCallbackHandler ready = self.reader.wait(0.25) File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap self._bootstrap_inner() (buildbot-venv) pokybuild@ubuntu2204-vk-1:~/yocto-worker/oe-selftest-debian/build$ python3 --version Python 3.10.12
We've removed all but one of the ubuntu-22.04 (python3.10) workers since the distro has not fixed python. Let's see how often it happens.