14223 2021-02-05 18:16:33 +0000 valgrind ptest failing on qemux86-64 2026-01-22 16:05:29 +0000 1 1 10 QA/Testing Package Testing (ptest) ptest 3.3 Other x86_64 NEW Medium+ enhancement 6.99 1 jon.mason mingli.yu alexandre.belloni flowergom Jagadheesan_Duraisamy joaomarcos.costa leonardo.sandoval raj.khem randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 89152 0 jon.mason 2021-02-05 18:16:33 +0000 Seeing valgrind ptest failing in multiple runs with the new glibc. For example: WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} Cannot run ptests without @expectedFailure as ptests are expected to fail RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.61s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.17s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (10957.28s) SUMMARY: core-image-sato-sdk-ptest () - Ran 4 tests in 10958.078s core-image-sato-sdk-ptest - OK - All required tests passed (successes=2, skipped=1, failures=0, errors=0) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 910 tasks of which 909 didn't need to be rerun and all succeeded. See https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1769 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1772 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1774 89159 1 flowergom 2021-02-08 03:23:15 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1777/steps/12/logs/stdio WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} Cannot run ptests without @expectedFailure as ptests are expected to fail 89163 2 flowergom 2021-02-08 08:10:02 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1781/steps/12/logs/stdio WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} 89170 3 flowergom 2021-02-09 01:01:30 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1785/steps/12/logs/stdio Bitbake still alive (5000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} 89171 4 flowergom 2021-02-09 01:03:51 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1789/steps/12/logs/stdio WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} 89172 5 flowergom 2021-02-09 01:05:41 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1782/steps/12/logs/stdio 89205 6 flowergom 2021-02-11 13:45:47 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1795/steps/12/logs/stdio 89208 7 flowergom 2021-02-11 13:48:18 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1792/steps/12/logs/stdio 89234 8 yf3yu 2021-02-11 17:17:25 +0000 When I quickly ran locally, maybe not with the most up todate version of poky I see the test was skipped. my head is sitting at f5188da2f12bee03e087c20d90720250e65e2922 SKIP: gdbserver_tests/nlcontrolc 89239 9 yf3yu 2021-02-11 20:52:05 +0000 reproduced... last comment was on arm silly me. 2021-02-11T20:36 BEGIN: /usr/lib/valgrind/ptest Hide valgrind tests that are non-deterministic Reported at https://bugs.kde.org/show_bug.cgi?id=430321 mv: cannot stat 'drd/tests/bar_bad.vgtest': No such file or directory mv: cannot stat 'drd/tests/bar_bad_xml.vgtest': No such file or directory timed out PASS: gdbserver_tests/hginfo timed out PASS: gdbserver_tests/hgtls PASS: gdbserver_tests/mcblocklistsearch PASS: gdbserver_tests/mcbreak PASS: gdbserver_tests/mcclean_after_fork PASS: gdbserver_tests/mchelp timed out PASS: gdbserver_tests/mcinfcallRU timed out FAIL: gdbserver_tests/mcinfcallWSRU PASS: gdbserver_tests/mcinvokeRU PASS: gdbserver_tests/mcinvokeWS PASS: gdbserver_tests/mcleak PASS: gdbserver_tests/mcmain_pic PASS: gdbserver_tests/mcsignopass PASS: gdbserver_tests/mcsigpass PASS: gdbserver_tests/mcvabits timed out FAIL: gdbserver_tests/mcwatchpoints PASS: gdbserver_tests/mssnapshot timed out FAIL: gdbserver_tests/nlcontrolc PASS: gdbserver_tests/nlfork_chain seems to be a timeout issue... again 89244 10 yf3yu 2021-02-12 01:19:28 +0000 downgraded to glibc2.32: no timeout failures seems related to the recent glibc2.33 uprev 89250 11 yf3yu 2021-02-12 16:10:14 +0000 on poky commit: (prior to glibc2.33) commit 03bf6b3ec0aaa24e9f81b3b0024c59f7cc77b410 (HEAD) Author: Mingli Yu <mingli.yu@windriver.com> Date: Thu Feb 4 17:07:14 2021 +0800 timed out FAIL: drd/tests/annotate_barrier === Test Summary === TOTAL: 726 PASSED: 687 FAILED: 1 SKIPPED: 38 DURATION: 1260 89255 12 yf3yu 2021-02-12 20:03:25 +0000 nlcontrolc hangs on archlinux with glibc2.33 and valgrind 3.16.1 built from source trying to see if valgrind master fixed this issue 89256 13 yf3yu 2021-02-12 20:05:41 +0000 clarification: archlinux keeps most up to date packages unlike other distro only valgrind was built from source, every other tool was from their most up-date repo. https://archlinux.org/packages/core/x86_64/glibc/ 89257 14 yf3yu 2021-02-12 21:19:07 +0000 filed a bug report, a similar bug was reported couple months ago for arm64 https://bugs.kde.org/show_bug.cgi?id=432870 89269 15 Jagadheesan_Duraisamy 2021-02-16 10:57:54 +0000 Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1809/steps/12/logs/stdio WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: 'valgrind': ['gdbserver_tests/nlcontrolc']} Cannot run ptests without @expectedFailure as ptests are expected to fail 89274 16 yf3yu 2021-02-16 17:56:03 +0000 this commit for glibc (present in 2.33 but not 2.32) is causing this test to fail when i rebuilt glibc with a patch to revert this commit, the test *passes*. ``` commit 2433d39b69743f100f972e7886f91a2e21795ef0 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jul 6 16:06:51 2020 -0300 linux: Add time64 select support ``` since this commit is the root cause, dropping that particular patch as a temporary workaround is the easiest solution as it is a high priority bug (until a fix for either valgrind or glibc is implemented) 89277 17 yf3yu 2021-02-17 04:53:46 +0000 nlcontrolc basically means sending a "ctrl-c" to the process in gdb. the test runs `LOOP` amount of `select` syscall to select which does nothing other than wait for the `TIMEOUT` (4th arg) to expire. Also note that 3 other threads (4 total) are also doing the exact same thing. `TIMEOUT` and `LOOP` are very large numbers, so the test would hang... unless the user sends a ctrl-c with gdb (which basically means causing a SIGTRAP) -> giving control back to gdb. we change TIMEOUT to 0 (for all 4 threads) and let the process run again if we are able to do it successfully, the test would complete much faster and on the next SIGTRAP we send, we should be no longer stuck in the the `select` syscall. Obviously in our bug, we probably weren't able to successfully get out of `select`, maybe because in linux(glibc), `select` actually updates the remaining amount of timeout in the same spot that gdb is trying to overwrite (so another thread/process can read that value) Looking into this interaction: https://man7.org/linux/man-pages/man2/select.2.html: ``` On Linux, select() modifies timeout to reflect the amount of time not slept; most other implementations do not do this. (POSIX.1 permits either behavior.) ``` 89278 18 yf3yu 2021-02-17 05:44:22 +0000 Update: `LOOP` doesn't even matter, the burn time and the sleep(select timeout) is long enough already. setting the timeout argument to 0 will just make the threads go to burn (also taking a long time) Another update: seems like we have an compiler optimization problem after running the test manually: ./sleepers 1000000000 1000000000 1000000000 BSBSBSBS 1 info threads ``` * 1 Thread 0x7ffff7ddb740 (LWP 340) "sleepers" 0x00007ffff7ecf619 in __GI___select (nfds=0, readfds=0x0, writefds=0x0, exceptfds=0x0, timeout=<optimized out>) ``` not really sure why it would happen when we add that glibc patch, how does glibc deal with timeout? testing with volatile 89279 19 yf3yu 2021-02-17 06:17:23 +0000 looking more like a glibc issue (at least for the glibc 2.32->2.33 failure on x86-64) if the timeout parameter gets optimized out by the compiler. There is no reason for the compiler to optimize it out since the timeout parameter gets filled by a value coming from argv. the only reason I suspect that would happen is a problem down in glibc where timeout doesn't get used at all. could it be that valgrind calls it the "wrong" way? very *unlikely* because its an API call defined by the POSIX standard. It might be doing something fishy by changing the last argument during the call (setting `timeout->sec_tv` 0) using gdb. here is syscall trace on glibc2.33 (with the bug present) : 5th argument seems to be garbage (expected to be in sequence t[0], t[1] t[2], t[3]) ``` root@qemux86-64:/usr/lib/valgrind/ptest/gdbserver_tests# valgrind --trace-syscalls=yes ./sleepers 1000000000 1000000000 1000000000 BSBSBSBS 2>&1 | grep pselect6 SYSCALL[365,2](270) sys_pselect6 ( 0, 0x0, 0x0, 0x0, 0x5634e00, 0x0 ) --> [async] ... SYSCALL[365,3](270) sys_pselect6 ( 0, 0x0, 0x0, 0x0, 0x5e35e00, 0x0 ) --> [async] ... SYSCALL[365,1](270) sys_pselect6 ( 0, 0x0, 0x0, 0x0, 0x1fff000a50, 0x0 ) --> [async] ... SYSCALL[365,4](270) sys_pselect6 ( 0, 0x0, 0x0, 0x0, 0x6636e00, 0x0 ) --> [async] ... things to investigate further: * turning off compiler optimization on glibc or valgrind 89285 20 yf3yu 2021-02-17 17:15:42 +0000 nlcontrolc expects to change the behaviour of a syscall by changing the argument `timeout` during that syscall (while it is blocked and probably getting woken up periodically by the system timer) Here is what i think is the root cause: From looking at that particular glibc commit, this is dependent on the compiler's optimization. - glibc creates a wrapper around the real syscall and converts the timeout type `timeval` to `timespec`, a new variable. Compiler can decide to optimize for space and actually have them be the same variable. nlcontrolc takes advantage of that by modifying it. in newest glibc2.33, they added the time64 support, but compiler is less likely to say that `timeval32` looks like `timespec64` Currently testing this theory if i build 2.32 with no optimization. expected behaviour is a hang in nlcontrolc another way to test it is actually changing the input to the argument to the underlying syscall. expected behaviour is no hang in nlcontrol 89286 21 yf3yu 2021-02-17 17:38:44 +0000 Here is a working call to sleepers *without* the select update patch in 2.33 My previous assumption about compiler optimization is wrong, it used to directly passes timeout to the underlying syscall. ``` root@qemux86-64:/usr/lib/valgrind/ptest/gdbserver_tests# valgrind --trace-syscalls=yes ./sleepers 1000000000 1000000000 1000000000 BSBSBSBS 2>&1 | grep select SYSCALL[6021,2](23) sys_select ( 0, 0x0, 0x0, 0x0, 0x4041b0 ) --> [async] ... SYSCALL[6021,3](23) sys_select ( 0, 0x0, 0x0, 0x0, 0x4041c0 ) --> [async] ... SYSCALL[6021,1](23) sys_select ( 0, 0x0, 0x0, 0x0, 0x4041a0 ) --> [async] ... SYSCALL[6021,4](23) sys_select ( 0, 0x0, 0x0, 0x0, 0x4041d0 ) --> [async] ... ``` It used to pass timeout directly to `select`, now it calls `select6` ``` -__select (int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, - struct timeval *timeout) +__select64 (int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, + struct __timeval64 *timeout) { -#ifdef __NR_select - return SYSCALL_CANCEL (select, nfds, readfds, writefds, exceptfds, - timeout); ``` here is how it calls it now ``` + r = SYSCALL_CANCEL (pselect6, nfds, readfds, writefds, exceptfds, pts32, + NULL); ``` 89287 22 yf3yu 2021-02-17 21:04:15 +0000 TLDR: ## glibc patch that causes failure ``` commit 2433d39b69743f100f972e7886f91a2e21795ef0 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jul 6 16:06:51 2020 -0300 linux: Add time64 select support ``` ## what changed in this patch to cause it to fail? timeout argument the user passes no longer makes its way to the syscall. Glibc copies it over and converts into a different format to call a different syscall `pselect` the failing test tries to modify said timeout argument to make the syscall end faster. Unfortunately doesn't work. ## what is the fix for this? disable it now in YP ## what about actually fixing this bug though? talking to valgrind about the purpose of nlcontrolc test - can we use a different syscall to sleep for a duration? - what is the exact purpose if this test ## other questions that arise Is glibc setting a new standard? what is the expected libc implementation of select? Grey area... according to `man select` ``` On Linux, select() modifies timeout to reflect the amount of time not slept; most other implementations do not do this. (POSIX.1-2001 permits either behavior.) ``` Did glibc violate the "linux standard"? ## other mentions the same test has been disabled for arm64 for around 6months now. 89288 23 randy.macleod 2021-02-17 21:26:40 +0000 Added Khem to CC since he knows how to spell glibc. :) I thought Alistair was in the YP BZ but he's not. I'll forward this thread to him. Is this commit: commit 2433d39b69743f100f972e7886f91a2e21795ef0 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jul 6 16:06:51 2020 -0300 linux: Add time64 select support The syscall __NR_pselect6_time64 (32-bit) or __NR_pselect6 (64-bit) is used as default. For architectures with __ASSUME_TIME64_SYSCALLS the 32-bit fallback uses __NR_select/__NR__newselect or __NR_pselect6 (it should cover the microblaze case where older kernels do not provide __NR_pselect6). Checked on x86_64-linux-gnu and i686-linux-gnu (on 5.4 and on 4.15 kernel). Reviewed-by: Alistair Francis <alistair.francis@wdc.com> a problem for any other package that you all know of? See Yi's previous comment for a good short summary of what's going wrong. Yi please look at the musl select behaviour/implementation. 89289 24 yf3yu 2021-02-18 03:56:04 +0000 musl master source code src/select/select.c tv is timeoutvalue s and ns are based on tv->tv_sec and tv->tv_usec ``` r = __syscall_cp(SYS_pselect6_time64, n, rfds, wfds, efds, tv ? ((long long[]){s, ns}) : 0, `` a new array is created, so musl select implementation looks to align with the new gibc one current nlcontrolc should fail also with musl... checking 89290 25 raj.khem 2021-02-18 04:43:06 +0000 Please try to reproduce this with gdb master and if it works then we need to search for a relevant fix perhaps try applying https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=439706e6a907b54658714a358b220febc63a0182 89318 26 Jagadheesan_Duraisamy 2021-02-19 11:44:16 +0000 For record, happened in: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1821/steps/12/logs/stdio 89322 27 yf3yu 2021-02-19 15:10:36 +0000 temporary fix to skip test https://git.openembedded.org/openembedded-core/commit/?id=bb991a24952ebf02cf0092c194d2b3a54a8b2e5b 89328 28 Jagadheesan_Duraisamy 2021-02-19 18:47:15 +0000 (In reply to comment #26) > For record, happened in: > https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1821/steps/ > 12/logs/stdio Also in https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1813/steps/12/logs/stdio 89330 29 yf3yu 2021-02-19 23:28:55 +0000 khem: built newest gdb (had to remove some patches and depend on gmp) test still fails ``` timed out FAIL: gdbserver_tests/nlcontrolc PASS: gdbserver_tests/nlfork_chain PASS: gdbserver_tests/nlgone_abrt c^C root@qemux86-64:/usr/lib/valgrind/ptest# ^C --version86-64:/usr/lib/valgrind/ptest# gdb GNU gdb (GDB) 11.0.50.20210219-git Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` 89352 30 leonardo.sandoval 2021-02-23 15:47:01 +0000 Another failure event [1] WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['gdbserver_tests/nlcontrolc']} [1] https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1811/steps/12/logs/stdio 89407 31 yf3yu 2021-02-25 16:37:28 +0000 (In reply to comment #30) > AssertionError: Failed ptests: > {'valgrind': ['gdbserver_tests/nlcontrolc']} > > > [1] > https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1811/steps/ > 12/logs/stdio this failure 9 days ago, the patch was committed around 7 days ago. https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1842 a similar test with 'master-next' 1 day ago AssertionError: Failed ptests: {'python3': ['test_many_processes']} 89410 32 randy.macleod 2021-02-25 17:16:55 +0000 Lowering to M+ since we are skipping the test and working with upstream valgrind and/or gdb (glibc?) devs to get to the root cause and a fix. 89416 33 alexandre.belloni 2021-02-26 01:45:15 +0000 An other similar failure from before the patch: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1810 And other failures seen: 'valgrind': ['memcheck/tests/linux/timerfd-syscall'] https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1556/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1548/steps/12/logs/stdio 'valgrind': ['drd/tests/std_list'] https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1550/steps/12/logs/stdio 89420 34 alexandre.belloni 2021-02-26 02:00:22 +0000 An other valgrind ptest failing: {'valgrind': ['drd/tests/thread_name', 'drd/tests/thread_name_xml']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1549/steps/10/logs/step1c 89548 35 yf3yu 2021-03-09 15:56:02 +0000 https://git.openembedded.org/openembedded-core/commit/?id=aac00b1c8042e41cd6bb1aea8e3033a1c6dd2b05 should make the test pass more frequently, upstream has fixed nlcontrolc by redesigning the test https://sourceware.org/git/?p=valgrind.git;a=commit;h=c79180a3afcf65902e578646c3b716cc749db406 a patch to oe-core is in master-next 89571 36 alexandre.belloni 2021-03-11 14:40:13 +0000 So far failures happened: 3 times on opensuse151-ty-1 4 times on ubuntu1804-arm-1 2 times on tumbleweed-ty-1 2 times on centos8-ty-1 2 times on debian10-ty-2 2 times on opensuse152-ty-1 2 times on ubuntu1804-ty-2 1 time on ubuntu1804-ty-1 1 time on ubuntu2004-ty-1 1 time on fedora33-ty-1 89627 37 yf3yu 2021-03-15 19:43:44 +0000 resolving this... please open new bug for other failures this ticket was for nlcontrolc https://git.openembedded.org/openembedded-core/commit/?id=64f98a731adaf86c9185c90436db1a6eb8e16b42 100828 38 joaomarcos.costa 2025-02-03 16:38:22 +0000 https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/866/steps/12/logs/stdio nlcontrolc appeared again: {'valgrind': ['gdbserver_tests/mcinfcallWSRU', 'gdbserver_tests/nlcontrolc', 'gdbserver_tests/nlvgdbsigqueue', 'memcheck/tests/sendmsg', 'drd/tests/fork-parallel', 'drd/tests/fork-serial', 'drd/tests/threaded-fork-vcs', 'drd/tests/threaded-fork', 'none/tests/fdleak_ipv4', 'none/tests/file_dclose', 'none/tests/socket_close']} qemux86-64-ptest rocky9-vk-1 100832 39 joaomarcos.costa 2025-02-03 16:46:52 +0000 Reopening. 100834 40 joaomarcos.costa 2025-02-03 16:48:48 +0000 https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/864/steps/12/logs/stdio Failed ptests: {'valgrind': ['gdbserver_tests/mcinfcallWSRU', 'gdbserver_tests/nlcontrolc', 'gdbserver_tests/nlvgdbsigqueue', 'memcheck/tests/sendmsg', 'drd/tests/fork-parallel', 'drd/tests/fork-serial', 'drd/tests/threaded-fork-vcs', 'drd/tests/threaded-fork', 'none/tests/fdleak_ipv4', 'none/tests/file_dclose', 'none/tests/socket_close']} qemuarm64-ptest ubuntu2204-vk-arm1 100857 41 randy.macleod 2025-02-06 15:09:12 +0000 This was with glibc-2.41. 104027 42 randy.macleod 2026-01-22 16:04:49 +0000 Mingli is working with valgrind upstream to try to improve the test approach so that we don't have as big a patchset and hopefully one that has fewer if any intermittent problems. We have disabled valgrind ptests on master: https://git.openembedded.org/openembedded-core/commit/?id=8fc31fb235af12530475ec5055f6c00ccfafadfa