14177 2021-01-06 06:40:34 +0000 AB-INT PTEST: tcl ptest intermittent failure 2021-07-15 14:56:00 +0000 1 1 10 QA/Testing Package Testing (ptest) ptest 3.3 x86 Multiple RESOLVED FIXED AB-INT Medium+ normal 3.4 M2 1 naveen.kumar.saini ross.burton alexandre.belloni bluelightning chee.yang.lee flowergom jon.mason kory.maincent leonardo.sandoval oobitots randy.macleod richard.purdie tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 88870 0 naveen.kumar.saini 2021-01-06 06:40:34 +0000 tcl ptest failure on qemux86-64 machine (master-next) AssertionError: Failed ptests: {'tcl': ['socket.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654 88940 1 randy.macleod 2021-01-14 16:24:11 +0000 Grace, please work on this in the coming weeks. 89050 2 mingli.yu 2021-01-28 01:49:12 +0000 I didn't reproduce the issue locally. 89071 3 randy.macleod 2021-01-28 17:12:56 +0000 The autobuilder log doesn't seem to be of much help. Here's the log: -------------------------------------------------------------------------------------------------------------------------------------------------------- NOTE: Running task 910 of 910 (/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb:do_testimage) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Started Bitbake still alive (5000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemux86-64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'tcl': ['socket.test']} Cannot run ptests without @expectedFailure as ptests are expected to fail RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.07s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.23s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (9679.28s) SUMMARY: core-image-sato-sdk-ptest () - Ran 4 tests in 9679.593s core-image-sato-sdk-ptest - OK - All required tests passed (successes=2, skipped=1, failures=0, errors=0) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 910 tasks of which 909 didn't need to be rerun and all succeeded. Summary: There was 1 WARNING message shown. /usr/lib64/python3.8/subprocess.py:844: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stdout = io.open(c2pread, 'rb', bufsize) program finished with exit code 0 elapsedTime=9726.330243 -------------------------------------------------------------------------------------------------------------------------------------------------------- the socket.test is doing quite a bit, can you add some logging so that we can find out what part of the tests is failing should it happen again? We do have access to the autobuilder so we can run tests overnight and we can manually collect logs if needed. Did you try to do a separate build and/or use stress-ng to generate high cpu/io load while the tests were running? 89376 4 leonardo.sandoval 2021-02-24 20:40:16 +0000 Another event of the same problem https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1560/steps/12/logs/stdio 89377 5 leonardo.sandoval 2021-02-24 20:41:40 +0000 (In reply to comment #4) > Another event of the same problem > > https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1560/steps/ > 12/logs/stdio Another from the same AB collection https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1842/steps/12/logs/stdio 89384 6 mingli.yu 2021-02-25 08:45:16 +0000 Hi Leonardo, The link you posted is not the failed log for tcl and it should be as expected at some point. BTW, I couldn't reproduce the tcl ptest failure locally. 89388 7 leonardo.sandoval 2021-02-25 15:20:13 +0000 @Mingli, you are right. Somehow, I through these were the same issues. 89408 8 randy.macleod 2021-02-25 16:39:30 +0000 @Mingli, Please add some logging so that we can find out what part of the tests is failing should it happen again. Richard is happy to add such changes to the master-next branch to improve the test results on the autobuilders. 89415 9 alexandre.belloni 2021-02-26 01:38:30 +0000 An other tcl ptest failed: AssertionError: Failed ptests: {'tcl': ['interp.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1531/steps/12/logs/stdio 89418 10 alexandre.belloni 2021-02-26 01:55:31 +0000 'tcl': ['interp.test'] https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1550/steps/12/l ogs/stdio 89554 11 kory.maincent 2021-03-09 16:16:55 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1892/steps/12/logs/stdio 89568 12 alexandre.belloni 2021-03-11 14:28:36 +0000 So far, happened: 3 times on ubuntu1804-arm-1 1 time on fedora-ty-1 1 time on ubuntu1804-ty-1 1 time on tumbleweed-ty-3 89611 13 kory.maincent 2021-03-12 09:27:23 +0000 AssertionError: Failed ptests: {'tcl': ['timer.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1615/steps/12/logs/stdio 89619 14 randy.macleod 2021-03-12 16:54:07 +0000 Grace, can you send a patch to either increase the timeout or remove the timer test? 89622 15 mingli.yu 2021-03-15 06:22:01 +0000 @Randy, I'm still looking into this issue. Currently there are three cases failed as the autobuilder ever reported: {'tcl': ['socket.test']} {'tcl': ['interp.test']} {'tcl': ['timer.test']} 89701 16 richard.purdie 2021-03-23 12:04:40 +0000 *** Bug 14312 has been marked as a duplicate of this bug. *** 89742 17 oobitots 2021-03-25 10:44:56 +0000 qemuarm64-ptest ubuntu1804-arm-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1652 NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Started Bitbake still alive (5000s) Bitbake still alive (10000s) WARNING: core-image-sato-sdk-ptest-1.0-r0 do_testimage: There were failing ptests. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 25, in test_ptestrunner_expectfail self.do_ptestrunner() File "/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/lib/oeqa/runtime/cases/ptest.py", line 112, in do_ptestrunner self.fail(failmsg) AssertionError: Failed ptests: {'tcl': ['chanio.test', 'interp.test'], 'valgrind': ['callgrind/tests/simwork-branch']} Cannot run ptests without @expectedFailure as ptests are expected to fail RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.09s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.20s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: EXPECTEDFAIL (12885.39s) SUMMARY: core-image-sato-sdk-ptest () - Ran 4 tests in 12885.711s core-image-sato-sdk-ptest - OK - All required tests passed (successes=2, skipped=1, failures=0, errors=0) NOTE: recipe core-image-sato-sdk-ptest-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 901 tasks of which 900 didn't need to be rerun and all succeeded. Summary: There was 1 WARNING message shown. 89799 18 bluelightning 2021-03-29 03:01:58 +0000 Failed again: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1662 89991 19 naveen.kumar.saini 2021-04-19 07:03:38 +0000 tcl ptest failed again (qemuarm64): AssertionError: Failed ptests: {'tcl': ['interp.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1704 90089 20 naveen.kumar.saini 2021-04-23 02:37:57 +0000 Happened again on master-next (qemuarm64): AssertionError: Failed ptests: {'tcl': ['interp.test']} Worker: ubuntu1804-arm-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1711 90387 21 chee.yang.lee 2021-05-14 08:28:28 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2088/steps/12/logs/stdio {'tcl': ['interp.test']} debian10-ty-1 qemux86-64-ptest 90413 22 jon.mason 2021-05-19 01:08:40 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1805/steps/12/logs/stdio {'tcl': ['interp.test']} qemuarm64-ptest ubuntu1804-arm-1 90417 23 jon.mason 2021-05-19 15:10:27 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1812/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1 90487 24 alexandre.belloni 2021-05-27 13:05:14 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1825/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1 90545 25 flowergom 2021-06-02 01:27:25 +0000 Happend again:https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1668/steps/12/logs/stdio 90547 26 flowergom 2021-06-02 02:06:43 +0000 Happend again: tcl ptest failure on qemux86-64-ptest https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2141/steps/13/logs/stdio 90716 27 alexandre.belloni 2021-06-20 17:02:34 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2193/steps/12/logs/stdio qemux86-64-ptest opensuse152-ty-1 90843 28 alexandre.belloni 2021-07-04 13:26:52 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1946/steps/12/logs/stdio qemuarm64-ptest ubuntu1804-arm-1 90911 29 ross.burton 2021-07-12 13:53:33 +0000 Got a patch for interp at least. Hopefully that helps that test case specifically, and if it does the others can have a similar approach taken. At least some of the test cases have changed since this issue was first opened so may not be as prone to timing. 90942 30 ross.burton 2021-07-15 14:56:00 +0000 Closing as the patches and landed and hopefully they fix the issues. Re-open this or new tickets for future tcl issues.