Bug 14825 - AB-INT PTEST: tcl socket.test failure
Summary: AB-INT PTEST: tcl socket.test failure
Status: RESOLVED FIXED
Alias: None
Product: Package Testing (ptest)
Classification: QA/Testing
Component: ptest (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.0
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2022-06-03 12:03 UTC by Luca Ceresoli
Modified: 2023-12-13 16:48 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luca Ceresoli 2022-06-03 12:03:49 UTC
Seen this tcl failure once:

  AssertionError: Failed ptests:
  {'openssl': ['Dubious,_test_returned_1'], 'tcl': ['socket.test']}

(the openssl one is a separate issue, already solved)

qemux86-64-ptest on ubuntu1604-ty-1

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/3565/steps/13/logs/stdio

From https://autobuilder.yocto.io/pub/non-release/20220601-9/testresults/qemux86-64-ptest/tcl.log :

-------------------8<-------------------
FAIL: socket.test
Tests running in interp:  /usr/lib/tcl/ptest/tcltest
Tests located in:  /usr/lib/tcl/ptest/tests
Tests running in:  /usr/lib/tcl/ptest
Temporary files stored in /usr/lib/tcl/ptest
Test files run in separate interpreters
Running tests that match:  *
Skipping test files that match:  l.*.test
Only running test files that match:  socket.test
Tests began at Wed Jun 01 13:01:00 BST 2022
socket.test


==== socket-14.14 testing fileevent readable on failed async socket connect FAILED
==== Contents of test case:

    # Test for bug 581937ab1e
    set a1 [after $latency {set x timeout}]
    # This connect should fail
    set s [socket -async localhost [randport]]
    fileevent $s readable {set x readable}
    vwait x
    set x

---- Result was:
timeout
---- Result should have been (exact matching):
readable
==== socket-14.14 FAILED


Tests ended at Wed Jun 01 13:01:07 BST 2022
all.tcl:	Total	173	Passed	156	Skipped	16	Failed	1
Sourced 1 Test Files.
Files with failing tests: socket.test
Number of tests skipped for each constraint:
	1	nonPortable
	4	notRoot
	10	thread
	1	win
-------------------8<-------------------

Similar, but not equal, to: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14488
Comment 1 Randy MacLeod 2022-06-09 14:36:45 UTC
We should disable this test. Ross gets the bug as our resident TCL expert!
Comment 2 Alexandre Belloni 2022-06-16 14:07:44 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/3343/steps/12/logs/stdio

qemuarm64-ptest ubuntu1804-arm-1
Comment 3 Richard Purdie 2022-09-22 14:43:12 UTC
We probably want to disable this test so we can close the issue as it will likely come back.
Comment 4 Ross Burton 2022-09-22 15:08:32 UTC
From the little tcl I know, this shouldn't be load-sensitive so we should be able to keep it.  I'll ask the TCL people for advice but we're not seeing it often so this isn't a high priority fix.
Comment 5 Ross Burton 2022-10-13 14:29:30 UTC
Hasn't happened for a while.
Comment 6 Alexandre Belloni 2022-11-16 11:07:42 UTC
https://autobuilder.yoctoproject.org/typhoon/#builders/81/builds/4267/steps/12/logs/stdio

qemux86-64-ptest debian11-ty-3
Comment 7 Randy MacLeod 2023-02-09 15:48:53 UTC
Not seen in a while.
If it happens again, we'll just disable some of the socket tests case.
Comment 9 Alexandre Belloni 2023-09-07 16:36:13 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/5547/steps/12/logs/stdio

qemux86-64-ptest debian11-ty-1
Comment 10 Alexandre Belloni 2023-10-05 20:51:02 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5420/steps/13/logs/stdio

qemuarm64-ptest ubuntu1804-arm-1
Comment 11 Alexandre Belloni 2023-10-05 20:53:22 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5414/steps/13/logs/stdio

qemuarm64-ptest ubuntu1804-arm-1 1
Comment 12 Ross Burton 2023-12-07 22:20:32 UTC
Patch to add skips sent.
Comment 13 Alexandre Belloni 2023-12-09 23:55:04 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5700/steps/13/logs/stdio

qemuarm64-ptest ubuntu1804-arm-1
Comment 14 Chee Yang 2023-12-12 06:24:44 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6029/steps/13/logs/stdio
qemux86-64-ptest   fedora38-ty-4
this is also 3.1.30.rc2 build
Comment 15 Ross Burton 2023-12-13 16:48:54 UTC
Hopefully fixed in 68beb4f4b5a0bea5d431decddf7656f18ac7a04a.