New intermittent ptest failure in test core-image-sato, seen in: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1590/steps/12/logs/stdio NOTE: Running task 901 of 901 (/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb:do_testimage) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Started Bitbake still alive (5000s) Bitbake still alive (10000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'valgrind': ['memcheck/tests/linux/timerfd-syscall']}
This was on ubuntu1804-arm-1
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1556/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1548/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1647
I can't reproduce this failure I think its due to the load/IO issue version: 3.16.1 === Test Summary === TOTAL: 726 PASSED: 686 FAILED: 0 SKIPPED: 40 DURATION: 1278 leaving it open for now...
Until now, this only failed on ubuntu1804-arm-1
Failed again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1662
this seems to happening frequently on arm and not that intermittent in nature
Another instance: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1664/steps/12/logs/stdio
Another one: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1671/steps/12/logs/stdio
This last onesA were on ubunt1804-arm-1. and the last one did contain the runqemu in tmpfs patch
fixed(more a workaround), see previous commit.
Sorry wrong bugzilla link.. reopening although I did submit a patch https://git.openembedded.org/openembedded-core/commit/?id=099313ef541920d4a84b801d9d8788a56ba7ec61 to print out the diff from expected result.
assigning it back to randy, if it happens again, we can get the test output details file at https://autobuilder.yocto.io/pub/non-release/ and make a judgement if the test needs to be disabled for timing/load reasons.
Seen additional failures: AssertionError: Failed ptests: {'valgrind': ['drd/tests/bar_bad']} on master (qemux86-64): woerker: ubuntu1604-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2005 On master-next: worker:fedora32-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2006
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1723/steps/12/logs/stdio qemuarm64 master on ubuntu1804-arm-1
'valgrind': ['memcheck/tests/leak_cpp_interior'] https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2022/steps/12/logs/stdio qemuarm64 master on ubuntu1804-arm-1
leak_cpp_interior is now disabled by 984ffe3ab4b7857b6201494cd77f2261bfc6d050
happen again qemuarm64-ptest worker : ubuntu1804-arm-1 branch_poky : master-next https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1777
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1799/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1
Reoccurence https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1818/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1840/steps/12/logs/stdio {'valgrind': ['memcheck/tests/linux/stack_changes']} ubuntu1804-arm-1 qemuarm64-ptest
{'valgrind': ['memcheck/tests/linux/stack_changes']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1843/steps/12/logs/stdio qemuarm64-ptest ubuntu2004-arm-1
Happend again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1853/steps/12/logs/stdio 'valgrind': ['memcheck/tests/linux/stack_changes'] qemuarm64-ptest
Happend again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1852/steps/12/logs/stdio
Happend again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1850/steps/12/logs/stdio
Happend again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1871/steps/12/logs/stdio
Happend again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1872/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2161/steps/12/logs/stdio
Hello, I want to update this thread with a few findings. First, there are several different tests failing in the autobuilder logs in this thread, I'm not sure if it's due to the same underlying cause between these yet. Furthermore, some of these are failures are on x86_64 and some are on arm64. I don't think it's the same SMP issue that was affecting the helgrind tests, though it's harder to tell without the diffs or logs from the autobuilder. I tried to recreate this error by running stack_changes, timerfd-syscall and fb_test_amd64 in Qemu x86_64 in parallel with SMP enabled for over 12 hours overnight and neither of the tests failed. Arm64 on the other hand has several issues. Testing on qemu_arm64 from an x86_64 host without SMP and with 1G of RAM allocated, stack_changes failed all 10/10 times I ran it on master, and several of the other tests fail randomly a lot more frequently. I am going to try on older versions as well to check if it is a recent change.
As discussed, the memcheck/tests/linux/stack_changes issue is not intermittent and I split it ou in bug 14430
This was catch-all bug with several failures grouped together. We've created more-specific targeted bugs. The issues covered here are either covered by those bugs or no longer occurring.