Perform diff and found that python tests (test_compile_files, test_magic_number, test_mtime, and more) that passing in 2.4.2.rc2 are failing on 2.4.3.rc2 --- a/python (2.4.2.rc2) +++ b/python (2.4.3.rc2) -PASS: test_compile_files (test.test_compileall.CompileallTests) -PASS: test_magic_number (test.test_compileall.CompileallTests) -PASS: test_mtime (test.test_compileall.CompileallTests) +FAIL: test_compile_files (test.test_compileall.CompileallTests) +FAIL: test_magic_number (test.test_compileall.CompileallTests) +FAIL: test_mtime (test.test_compileall.CompileallTests) -PASS: test_http2time (test.test_cookielib.DateTimeTests) -PASS: test_http2time_formats (test.test_cookielib.DateTimeTests) +FAIL: test_http2time (test.test_cookielib.DateTimeTests) +FAIL: test_http2time_formats (test.test_cookielib.DateTimeTests) -PASS: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) -PASS: test_context (test.test_ftplib.TestTLS_FTPClass) -PASS: test_control_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_data_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_login (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_context (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_control_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_data_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_login (test.test_ftplib.TestTLS_FTPClass) -PASS: test_basics (test.test_import.PycRewritingTests) -PASS: test_foreign_code (test.test_import.PycRewritingTests) -PASS: test_incorrect_code_name (test.test_import.PycRewritingTests) +FAIL: test_basics (test.test_import.PycRewritingTests) +FAIL: test_foreign_code (test.test_import.PycRewritingTests) +FAIL: test_incorrect_code_name (test.test_import.PycRewritingTests) -PASS: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) +FAIL: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) PASS: A brutal shutdown of an SSL server should raise an OSError -PASS: test_server_accept (test.test_ssl.ThreadedTests) +FAIL: test_server_accept (test.test_ssl.ThreadedTests) -PASS: test_date (test.test_strptime.StrptimeTests) +FAIL: test_date (test.test_strptime.StrptimeTests) -PASS: test_year (test.test_strptime.StrptimeTests) +FAIL: test_year (test.test_strptime.StrptimeTests) -PASS: test_redirect_no_path (test.test_urllib2.HandlerTests) +FAIL: test_redirect_no_path (test.test_urllib2.HandlerTests) -PASS: test_basic_auth_success (test.test_urllib2_localnet.BasicAuthTests) +FAIL: test_basic_auth_success (test.test_urllib2_localnet.BasicAuthTests) -PASS: test_compile_files (test.test_compileall.CompileallTests) -PASS: test_magic_number (test.test_compileall.CompileallTests) -PASS: test_mtime (test.test_compileall.CompileallTests) +FAIL: test_compile_files (test.test_compileall.CompileallTests) +FAIL: test_magic_number (test.test_compileall.CompileallTests) +FAIL: test_mtime (test.test_compileall.CompileallTests) -PASS: test_http2time (test.test_cookielib.DateTimeTests) -PASS: test_http2time_formats (test.test_cookielib.DateTimeTests) +FAIL: test_http2time (test.test_cookielib.DateTimeTests) +FAIL: test_http2time_formats (test.test_cookielib.DateTimeTests) -PASS: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) -PASS: test_context (test.test_ftplib.TestTLS_FTPClass) -PASS: test_control_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_data_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_login (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_context (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_control_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_data_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_login (test.test_ftplib.TestTLS_FTPClass) -PASS: test_basics (test.test_import.PycRewritingTests) -PASS: test_foreign_code (test.test_import.PycRewritingTests) -PASS: test_incorrect_code_name (test.test_import.PycRewritingTests) +FAIL: test_basics (test.test_import.PycRewritingTests) +FAIL: test_foreign_code (test.test_import.PycRewritingTests) +FAIL: test_incorrect_code_name (test.test_import.PycRewritingTests) -PASS: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) +FAIL: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) PASS: A brutal shutdown of an SSL server should raise an OSError -PASS: test_server_accept (test.test_ssl.ThreadedTests) +FAIL: test_server_accept (test.test_ssl.ThreadedTests)
My hunch is that python-ptest doesn't depend on *all* of the Python library packages.
running test against latest.
currently seeing these failures: FAIL: test_getattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: Check same msg for exceptions with 0 args and overridden __str__ FAIL: Check same msg for Exceptions with 1 arg FAIL: Check same msg for exceptions with overridden __str__ and 1 arg FAIL: Check same msg for exceptions with overridden __str__ and many args FAIL: test_0_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_many_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_basic_script (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpXzFI_5/script.py': FAIL: test_directory (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3bo47W': FAIL: test_directory_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpCRBJAD': FAIL: test_directory_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3JEJ7X': FAIL: test_module_in_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpVSBSXY/launch.py': FAIL: test_module_in_package_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpqx0kZy/launch.py': FAIL: test_module_in_subpackage_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpc2YAYT/launch.py': FAIL: test_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpgVJEhI/launch.py': FAIL: test_package_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpmUaQro/launch.py': FAIL: test_package_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3BA0Nh/launch.py': FAIL: test_package_recursion (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpfrnWsV/launch.py': FAIL: test_script_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp2sWkG8/script.pyc': FAIL: test_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp6NffD_/test_zip.zip': FAIL: test_zipfile_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpoVHT1r/test_zip.zip': FAIL: test_zipfile_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpIHoD_v/test_zip.zip': FAIL: test_invalid_name (ctypes.test.test_structures.StructureTestCase) FAIL: test_getattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: Check same msg for exceptions with 0 args and overridden __str__ FAIL: Check same msg for Exceptions with 1 arg FAIL: Check same msg for exceptions with overridden __str__ and 1 arg FAIL: Check same msg for exceptions with overridden __str__ and many args FAIL: test_0_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_many_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_basic_script (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpTfONN6/script.py': FAIL: test_directory (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpgtbLth': FAIL: test_directory_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpemZyS6': FAIL: test_directory_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpMUTrNN': FAIL: test_module_in_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpSMinIW/launch.py': FAIL: test_module_in_package_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpZGGV90/launch.py': FAIL: test_module_in_subpackage_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpbiYMd3/launch.py': FAIL: test_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpN3s55C/launch.py': FAIL: test_package_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmptB125_/launch.py': FAIL: test_package_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpSfRzJf/launch.py': FAIL: test_package_recursion (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpRXoCvq/launch.py': FAIL: test_script_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpeYMwv4/script.pyc': FAIL: test_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpNohLD7/test_zip.zip': FAIL: test_zipfile_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpLaqYMv/test_zip.zip': FAIL: test_zipfile_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpnY8i8L/test_zip.zip': FAIL: test_invalid_name (ctypes.test.test_structures.StructureTestCase)
The focus here was not so much on how many tests pass or fail in current release, but rather the tests that were passing in previous release (eg. 2.4.2) but now failing on current release (eg. 2.4.3) showing that some changes in current release may caused the package functionality to be broken.
Performed diff and found that python tests that passed in 2.6 M2 rc1 are failing on 2.6 M3 rc1 ------------------------------------------------------------------------------ Package Name: python ------------------------------------------------------------------------------ -PASS: test_options (test.test_ssl.ContextTests) +FAIL: test_options (test.test_ssl.ContextTests) -PASS: test_options (test.test_ssl.ContextTests) +FAIL: test_options (test.test_ssl.ContextTests) ------------------------------------------------------------------------------ Total number of test cases passing in Previous release but failing in Current release: 2 ------------------------------------------------------------------------------
Looks like openssl 1.1 broke Python2 too. Anuj, you worked wonders with Python3, can you take py2 too?
I cherry-picked some patches and looks like it's fixed now. Will send to list.
Merged for ssl tests: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2b4dce4925627c81ee2f27a0eeb2422e15029c3e If there are any other python tests failing as compared to last runs (like the ones in first comment), please re-open or file a new bug.