12788 2018-06-14 07:17:54 +0000 [2.4.3.rc2] python ptest failed 2018-10-08 23:40:49 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.6 x86 Multiple RESOLVED FIXED Medium+ normal 2.6 M4 1 ee.peng.yeoh anuj.mittal manish.das meta.mr.watcher meta.watcher randy.macleod ross.burton --- Regression (Found by QA test) No (bug/feature does not impact docs) oldest_to_newest 80784 0 ee.peng.yeoh 2018-06-14 07:17:54 +0000 Perform diff and found that python tests (test_compile_files, test_magic_number, test_mtime, and more) that passing in 2.4.2.rc2 are failing on 2.4.3.rc2 --- a/python (2.4.2.rc2) +++ b/python (2.4.3.rc2) -PASS: test_compile_files (test.test_compileall.CompileallTests) -PASS: test_magic_number (test.test_compileall.CompileallTests) -PASS: test_mtime (test.test_compileall.CompileallTests) +FAIL: test_compile_files (test.test_compileall.CompileallTests) +FAIL: test_magic_number (test.test_compileall.CompileallTests) +FAIL: test_mtime (test.test_compileall.CompileallTests) -PASS: test_http2time (test.test_cookielib.DateTimeTests) -PASS: test_http2time_formats (test.test_cookielib.DateTimeTests) +FAIL: test_http2time (test.test_cookielib.DateTimeTests) +FAIL: test_http2time_formats (test.test_cookielib.DateTimeTests) -PASS: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) -PASS: test_context (test.test_ftplib.TestTLS_FTPClass) -PASS: test_control_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_data_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_login (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_context (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_control_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_data_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_login (test.test_ftplib.TestTLS_FTPClass) -PASS: test_basics (test.test_import.PycRewritingTests) -PASS: test_foreign_code (test.test_import.PycRewritingTests) -PASS: test_incorrect_code_name (test.test_import.PycRewritingTests) +FAIL: test_basics (test.test_import.PycRewritingTests) +FAIL: test_foreign_code (test.test_import.PycRewritingTests) +FAIL: test_incorrect_code_name (test.test_import.PycRewritingTests) -PASS: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) +FAIL: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) PASS: A brutal shutdown of an SSL server should raise an OSError -PASS: test_server_accept (test.test_ssl.ThreadedTests) +FAIL: test_server_accept (test.test_ssl.ThreadedTests) -PASS: test_date (test.test_strptime.StrptimeTests) +FAIL: test_date (test.test_strptime.StrptimeTests) -PASS: test_year (test.test_strptime.StrptimeTests) +FAIL: test_year (test.test_strptime.StrptimeTests) -PASS: test_redirect_no_path (test.test_urllib2.HandlerTests) +FAIL: test_redirect_no_path (test.test_urllib2.HandlerTests) -PASS: test_basic_auth_success (test.test_urllib2_localnet.BasicAuthTests) +FAIL: test_basic_auth_success (test.test_urllib2_localnet.BasicAuthTests) -PASS: test_compile_files (test.test_compileall.CompileallTests) -PASS: test_magic_number (test.test_compileall.CompileallTests) -PASS: test_mtime (test.test_compileall.CompileallTests) +FAIL: test_compile_files (test.test_compileall.CompileallTests) +FAIL: test_magic_number (test.test_compileall.CompileallTests) +FAIL: test_mtime (test.test_compileall.CompileallTests) -PASS: test_http2time (test.test_cookielib.DateTimeTests) -PASS: test_http2time_formats (test.test_cookielib.DateTimeTests) +FAIL: test_http2time (test.test_cookielib.DateTimeTests) +FAIL: test_http2time_formats (test.test_cookielib.DateTimeTests) -PASS: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) -PASS: test_context (test.test_ftplib.TestTLS_FTPClass) -PASS: test_control_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_data_connection (test.test_ftplib.TestTLS_FTPClass) -PASS: test_login (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_check_hostname (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_context (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_control_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_data_connection (test.test_ftplib.TestTLS_FTPClass) +FAIL: test_login (test.test_ftplib.TestTLS_FTPClass) -PASS: test_basics (test.test_import.PycRewritingTests) -PASS: test_foreign_code (test.test_import.PycRewritingTests) -PASS: test_incorrect_code_name (test.test_import.PycRewritingTests) +FAIL: test_basics (test.test_import.PycRewritingTests) +FAIL: test_foreign_code (test.test_import.PycRewritingTests) +FAIL: test_incorrect_code_name (test.test_import.PycRewritingTests) -PASS: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) +FAIL: test_read_write_after_close_raises_valuerror (test.test_ssl.ThreadedTests) PASS: A brutal shutdown of an SSL server should raise an OSError -PASS: test_server_accept (test.test_ssl.ThreadedTests) +FAIL: test_server_accept (test.test_ssl.ThreadedTests) 80789 1 ross.burton 2018-06-14 14:58:02 +0000 My hunch is that python-ptest doesn't depend on *all* of the Python library packages. 81364 2 akuster 2018-08-15 16:21:34 +0000 running test against latest. 81365 3 akuster 2018-08-15 16:56:56 +0000 currently seeing these failures: FAIL: test_getattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: Check same msg for exceptions with 0 args and overridden __str__ FAIL: Check same msg for Exceptions with 1 arg FAIL: Check same msg for exceptions with overridden __str__ and 1 arg FAIL: Check same msg for exceptions with overridden __str__ and many args FAIL: test_0_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_many_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_basic_script (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpXzFI_5/script.py': FAIL: test_directory (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3bo47W': FAIL: test_directory_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpCRBJAD': FAIL: test_directory_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3JEJ7X': FAIL: test_module_in_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpVSBSXY/launch.py': FAIL: test_module_in_package_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpqx0kZy/launch.py': FAIL: test_module_in_subpackage_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpc2YAYT/launch.py': FAIL: test_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpgVJEhI/launch.py': FAIL: test_package_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpmUaQro/launch.py': FAIL: test_package_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp3BA0Nh/launch.py': FAIL: test_package_recursion (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpfrnWsV/launch.py': FAIL: test_script_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp2sWkG8/script.pyc': FAIL: test_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmp6NffD_/test_zip.zip': FAIL: test_zipfile_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpoVHT1r/test_zip.zip': FAIL: test_zipfile_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpIHoD_v/test_zip.zip': FAIL: test_invalid_name (ctypes.test.test_structures.StructureTestCase) FAIL: test_getattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: test_hasattr (test.test_builtin.BuiltinTest) FAIL: Check same msg for exceptions with 0 args and overridden __str__ FAIL: Check same msg for Exceptions with 1 arg FAIL: Check same msg for exceptions with overridden __str__ and 1 arg FAIL: Check same msg for exceptions with overridden __str__ and many args FAIL: test_0_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_1_arg_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_many_args_with_overridden___str__ (test.test_exceptions.TestSameStrAndUnicodeMsg) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_unicode (test.test_StringIO.TestcStringIO) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_output_htmlcalendar (test.test_calendar.OutputTestCase) FAIL: test_basic_script (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpTfONN6/script.py': FAIL: test_directory (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpgtbLth': FAIL: test_directory_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpemZyS6': FAIL: test_directory_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpMUTrNN': FAIL: test_module_in_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpSMinIW/launch.py': FAIL: test_module_in_package_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpZGGV90/launch.py': FAIL: test_module_in_subpackage_in_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpbiYMd3/launch.py': FAIL: test_package (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpN3s55C/launch.py': FAIL: test_package_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmptB125_/launch.py': FAIL: test_package_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpSfRzJf/launch.py': FAIL: test_package_recursion (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpRXoCvq/launch.py': FAIL: test_script_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpeYMwv4/script.pyc': FAIL: test_zipfile (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpNohLD7/test_zip.zip': FAIL: test_zipfile_compiled (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpLaqYMv/test_zip.zip': FAIL: test_zipfile_error (test.test_cmd_line_script.CmdLineTest) ... Output from test script '/var/volatile/tmp/tmpnY8i8L/test_zip.zip': FAIL: test_invalid_name (ctypes.test.test_structures.StructureTestCase) 81370 4 ee.peng.yeoh 2018-08-16 01:22:33 +0000 The focus here was not so much on how many tests pass or fail in current release, but rather the tests that were passing in previous release (eg. 2.4.2) but now failing on current release (eg. 2.4.3) showing that some changes in current release may caused the package functionality to be broken. 81706 5 manish.das 2018-10-03 10:15:04 +0000 Performed diff and found that python tests that passed in 2.6 M2 rc1 are failing on 2.6 M3 rc1 ------------------------------------------------------------------------------ Package Name: python ------------------------------------------------------------------------------ -PASS: test_options (test.test_ssl.ContextTests) +FAIL: test_options (test.test_ssl.ContextTests) -PASS: test_options (test.test_ssl.ContextTests) +FAIL: test_options (test.test_ssl.ContextTests) ------------------------------------------------------------------------------ Total number of test cases passing in Previous release but failing in Current release: 2 ------------------------------------------------------------------------------ 81709 6 ross.burton 2018-10-03 11:08:29 +0000 Looks like openssl 1.1 broke Python2 too. Anuj, you worked wonders with Python3, can you take py2 too? 81717 7 anuj.mittal 2018-10-04 00:04:24 +0000 I cherry-picked some patches and looks like it's fixed now. Will send to list. 81782 8 anuj.mittal 2018-10-08 23:40:49 +0000 Merged for ssl tests: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2b4dce4925627c81ee2f27a0eeb2422e15029c3e If there are any other python tests failing as compared to last runs (like the ones in first comment), please re-open or file a new bug.