tcl ptest failure on qemux86-64 machine (master-next) AssertionError: Failed ptests: {'tcl': ['socket.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654
Grace, please work on this in the coming weeks.
I didn't reproduce the issue locally.
The autobuilder log doesn't seem to be of much help. Here's the log: -------------------------------------------------------------------------------------------------------------------------------------------------------- NOTE: Running task 910 of 910 (/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb:do_testimage) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Started Bitbake still alive (5000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'tcl': ['socket.test']} Cannot run ptests without @expectedFailure as ptests are expected to fail RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.07s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.23s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (9679.28s) SUMMARY: core-image-sato-sdk-ptest () - Ran 4 tests in 9679.593s core-image-sato-sdk-ptest - OK - All required tests passed (successes=2, skipped=1, failures=0, errors=0) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 910 tasks of which 909 didn't need to be rerun and all succeeded. Summary: There was 1 WARNING message shown. /usr/lib64/python3.8/subprocess.py:844: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stdout = io.open(c2pread, 'rb', bufsize) program finished with exit code 0 elapsedTime=9726.330243 -------------------------------------------------------------------------------------------------------------------------------------------------------- the socket.test is doing quite a bit, can you add some logging so that we can find out what part of the tests is failing should it happen again? We do have access to the autobuilder so we can run tests overnight and we can manually collect logs if needed. Did you try to do a separate build and/or use stress-ng to generate high cpu/io load while the tests were running?
Another event of the same problem https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1560/steps/12/logs/stdio
(In reply to comment #4) > Another event of the same problem > > https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1560/steps/ > 12/logs/stdio Another from the same AB collection https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1842/steps/12/logs/stdio
Hi Leonardo, The link you posted is not the failed log for tcl and it should be as expected at some point. BTW, I couldn't reproduce the tcl ptest failure locally.
@Mingli, you are right. Somehow, I through these were the same issues.
@Mingli, Please add some logging so that we can find out what part of the tests is failing should it happen again. Richard is happy to add such changes to the master-next branch to improve the test results on the autobuilders.
An other tcl ptest failed: AssertionError: Failed ptests: {'tcl': ['interp.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1531/steps/12/logs/stdio
'tcl': ['interp.test'] https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1550/steps/12/l ogs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1892/steps/12/logs/stdio
So far, happened: 3 times on ubuntu1804-arm-1 1 time on fedora-ty-1 1 time on ubuntu1804-ty-1 1 time on tumbleweed-ty-3
AssertionError: Failed ptests: {'tcl': ['timer.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1615/steps/12/logs/stdio
Grace, can you send a patch to either increase the timeout or remove the timer test?
@Randy, I'm still looking into this issue. Currently there are three cases failed as the autobuilder ever reported: {'tcl': ['socket.test']} {'tcl': ['interp.test']} {'tcl': ['timer.test']}
*** Bug 14312 has been marked as a duplicate of this bug. ***
qemuarm64-ptest ubuntu1804-arm-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1652 NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Started Bitbake still alive (5000s) Bitbake still alive (10000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'tcl': ['chanio.test', 'interp.test'], 'valgrind': ['callgrind/tests/simwork-branch']} Cannot run ptests without @expectedFailure as ptests are expected to fail RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.09s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.20s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (12885.39s) SUMMARY: core-image-sato-sdk-ptest () - Ran 4 tests in 12885.711s core-image-sato-sdk-ptest - OK - All required tests passed (successes=2, skipped=1, failures=0, errors=0) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 901 tasks of which 900 didn't need to be rerun and all succeeded. Summary: There was 1 WARNING message shown.
Failed again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1662
tcl ptest failed again (qemuarm64): AssertionError: Failed ptests: {'tcl': ['interp.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1704
Happened again on master-next (qemuarm64): AssertionError: Failed ptests: {'tcl': ['interp.test']} Worker: ubuntu1804-arm-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1711
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2088/steps/12/logs/stdio {'tcl': ['interp.test']} debian10-ty-1 qemux86-64-ptest
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1805/steps/12/logs/stdio {'tcl': ['interp.test']} qemuarm64-ptest ubuntu1804-arm-1
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1812/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1825/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1
Happend again:https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1668/steps/12/logs/stdio
Happend again: tcl ptest failure on qemux86-64-ptest https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2141/steps/13/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2193/steps/12/logs/stdio qemux86-64-ptest opensuse152-ty-1
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1946/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1
Got a patch for interp at least. Hopefully that helps that test case specifically, and if it does the others can have a similar approach taken. At least some of the test cases have changed since this issue was first opened so may not be as prone to timing.
Closing as the patches and landed and hopefully they fix the issues. Re-open this or new tickets for future tcl issues.