https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/489 [RP: Failure appears to occur occasionally, in this case within a selftest but happens in general] 2019-11-18 00:23:39,959 - oe-selftest - INFO - 11: 7/41 297/378 (510.69s) (incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline) 2019-11-18 00:23:39,960 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line 92, in test_core_image_full_cmdline bitbake('-c testimage core-image-full-cmdline') File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/utils/commands.py", line 221, in bitbake return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options) File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/utils/commands.py", line 199, in runCmd raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) AssertionError: Command 'bitbake -c testimage core-image-full-cmdline' returned non-zero exit status 1: Loading cache...done. Loaded 1336 entries from dependency cache. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.44.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "x86_64-poky-linux" MACHINE = "qemux86-64" DISTRO = "poky" DISTRO_VERSION = "3.0" TUNE_FEATURES = "m64 core2" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "master-next:c2b4cea558e60d47986255b7281b58ba7acc3fa4" meta-selftest = "master:b78b2c3215f79088b6ff0e401be52db9dbff9422" Initialising tasks...done. NOTE: Executing Tasks NOTE: Running setscene task 90 of 90 (virtual:native:/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/cpio/cpio_2.12.bb:do_populate_sysroot_setscene) NOTE: recipe cpio-native-2.12-r0: task do_populate_sysroot_setscene: Started NOTE: recipe cpio-native-2.12-r0: task do_populate_sysroot_setscene: Succeeded NOTE: Setscene tasks completed NOTE: Running task 731 of 731 (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage) NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started Not starting HTTPService for directory tmp/deploy/deb/all which doesn't exist Test requires apt to be installed Stopped HTTPService on 192.168.7.1:0 Test requires autoconf to be installed Test requires gtk+3 to be installed Test requires autoconf to be installed Test requires connman to be installed Test case connman.ConnmanTest.test_connmand_running depends on connman.ConnmanTest.test_connmand_help but it didn't pass/run. Test requires dnf to be installed Test case dnf.DnfBasicTest.test_dnf_history depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_info depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_search depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_version depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Started HTTPService on 192.168.7.1:44343 Test case dnf.DnfRepoTest.test_dnf_makecache depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_exclude depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_dependency depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_from_disk depends on dnf.DnfRepoTest.test_dnf_install_dependency but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_from_http depends on dnf.DnfRepoTest.test_dnf_install_from_disk but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_installroot depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test run when enable usrmege Test case dnf.DnfRepoTest.test_dnf_reinstall depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_repoinfo depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Stopped HTTPService on 192.168.7.1:44343 Test requires gcc to be installed Test requires g++ to be installed Test requires g++ to be installed Test requires make to be installed Test requires python3-pygobject to be installed Test requires gcc to be installed Test requires ldd to be installed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/runtime/cases/logrotate.py", line 49, in test_2_logrotate self.assertTrue(int(output)>=3, msg = msg) AssertionError: False is not true : new logfile could not be created. List of files within log directory: drwxr-xr-x 2 root root 1024 Nov 18 00:23 . drwx------ 3 root root 1024 Nov 18 00:23 .. Test requires busybox-syslog to be installed Test requires opkg to be installed Test requires pam to be in DISTRO_FEATURES Test requires perl to be installed Test requires ptest-runner to be installed Test requires python3-core to be installed Test requires rpm to be installed Test case rpm.RpmBasicTest.test_rpm_query depends on rpm.RpmBasicTest.test_rpm_help but it didn't pass/run. Test case rpm.RpmBasicTest.test_rpm_query_nonroot depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_rpm_install depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_rpm_remove depends on rpm.RpmInstallRemoveTest.test_rpm_install but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size depends on rpm.RpmInstallRemoveTest.test_rpm_remove but it didn't pass/run. Test requires systemtap to be installed systemd is not the init manager for this image Test case systemd.SystemdBasicTests.test_systemd_failed depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdBasicTests.test_systemd_list depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdJournalTests.test_systemd_boot_time depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdJournalTests.test_systemd_journal depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdServiceTests.test_systemd_status depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdServiceTests.test_systemd_disable_enable depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run. Test is only meant to run with read-only-rootfs in IMAGE_FEATURES Test case systemd.SystemdServiceTests.test_systemd_stop_start depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run. Test requires xserver-nodm-init to be installed RESULTS: RESULTS - date.DateTest.test_date: PASSED (0.44s) RESULTS - df.DfTest.test_df: PASSED (0.12s) RESULTS - logrotate.LogrotateTest.test_1_logrotate_setup: PASSED (0.23s) RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (0.13s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (1.23s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (2.38s) RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (2.79s) RESULTS - ping.PingTest.test_ping: PASSED (0.17s) RESULTS - scp.ScpTest.test_scp_file: PASSED (0.55s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.25s) RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.01s) RESULTS - buildcpio.BuildCpioTest.test_cpio: SKIPPED (0.00s) RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s) RESULTS - buildlzip.BuildLzipTest.test_lzip: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand_help: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand2019-11-18 00:23:39,959 - oe-selftest - INFO - 11: 7/41 297/378 (510.69s) (incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline) 2019-11-18 00:23:39,960 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line 92, in test_core_image_full_cmdline bitbake('-c testimage core-image-full-cmdline') File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/utils/commands.py", line 221, in bitbake return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options) File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/utils/commands.py", line 199, in runCmd raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) AssertionError: Command 'bitbake -c testimage core-image-full-cmdline' returned non-zero exit status 1: Loading cache...done. Loaded 1336 entries from dependency cache. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.44.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "x86_64-poky-linux" MACHINE = "qemux86-64" DISTRO = "poky" DISTRO_VERSION = "3.0" TUNE_FEATURES = "m64 core2" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "master-next:c2b4cea558e60d47986255b7281b58ba7acc3fa4" meta-selftest = "master:b78b2c3215f79088b6ff0e401be52db9dbff9422" Initialising tasks...done. NOTE: Executing Tasks NOTE: Running setscene task 90 of 90 (virtual:native:/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/cpio/cpio_2.12.bb:do_populate_sysroot_setscene) NOTE: recipe cpio-native-2.12-r0: task do_populate_sysroot_setscene: Started NOTE: recipe cpio-native-2.12-r0: task do_populate_sysroot_setscene: Succeeded NOTE: Setscene tasks completed NOTE: Running task 731 of 731 (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage) NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started Not starting HTTPService for directory tmp/deploy/deb/all which doesn't exist Test requires apt to be installed Stopped HTTPService on 192.168.7.1:0 Test requires autoconf to be installed Test requires gtk+3 to be installed Test requires autoconf to be installed Test requires connman to be installed Test case connman.ConnmanTest.test_connmand_running depends on connman.ConnmanTest.test_connmand_help but it didn't pass/run. Test requires dnf to be installed Test case dnf.DnfBasicTest.test_dnf_history depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_info depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_search depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfBasicTest.test_dnf_version depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Started HTTPService on 192.168.7.1:44343 Test case dnf.DnfRepoTest.test_dnf_makecache depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_exclude depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_dependency depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_from_disk depends on dnf.DnfRepoTest.test_dnf_install_dependency but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_install_from_http depends on dnf.DnfRepoTest.test_dnf_install_from_disk but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_installroot depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Test run when enable usrmege Test case dnf.DnfRepoTest.test_dnf_reinstall depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run. Test case dnf.DnfRepoTest.test_dnf_repoinfo depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run. Stopped HTTPService on 192.168.7.1:44343 Test requires gcc to be installed Test requires g++ to be installed Test requires g++ to be installed Test requires make to be installed Test requires python3-pygobject to be installed Test requires gcc to be installed Test requires ldd to be installed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/runtime/cases/logrotate.py", line 49, in test_2_logrotate self.assertTrue(int(output)>=3, msg = msg) AssertionError: False is not true : new logfile could not be created. List of files within log directory: drwxr-xr-x 2 root root 1024 Nov 18 00:23 . drwx------ 3 root root 1024 Nov 18 00:23 .. Test requires busybox-syslog to be installed Test requires opkg to be installed Test requires pam to be in DISTRO_FEATURES Test requires perl to be installed Test requires ptest-runner to be installed Test requires python3-core to be installed Test requires rpm to be installed Test case rpm.RpmBasicTest.test_rpm_query depends on rpm.RpmBasicTest.test_rpm_help but it didn't pass/run. Test case rpm.RpmBasicTest.test_rpm_query_nonroot depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_rpm_install depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_rpm_remove depends on rpm.RpmInstallRemoveTest.test_rpm_install but it didn't pass/run. Test case rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size depends on rpm.RpmInstallRemoveTest.test_rpm_remove but it didn't pass/run. Test requires systemtap to be installed systemd is not the init manager for this image Test case systemd.SystemdBasicTests.test_systemd_failed depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdBasicTests.test_systemd_list depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdJournalTests.test_systemd_boot_time depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdJournalTests.test_systemd_journal depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdServiceTests.test_systemd_status depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run. Test case systemd.SystemdServiceTests.test_systemd_disable_enable depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run. Test is only meant to run with read-only-rootfs in IMAGE_FEATURES Test case systemd.SystemdServiceTests.test_systemd_stop_start depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run. Test requires xserver-nodm-init to be installed RESULTS: RESULTS - date.DateTest.test_date: PASSED (0.44s) RESULTS - df.DfTest.test_df: PASSED (0.12s) RESULTS - logrotate.LogrotateTest.test_1_logrotate_setup: PASSED (0.23s) RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (0.13s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (1.23s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (2.38s) RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (2.79s) RESULTS - ping.PingTest.test_ping: PASSED (0.17s) RESULTS - scp.ScpTest.test_scp_file: PASSED (0.55s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.25s) RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.01s) RESULTS - buildcpio.BuildCpioTest.test_cpio: SKIPPED (0.00s) RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s) RESULTS - buildlzip.BuildLzipTest.test_lzip: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand_help: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand_running: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.01s) RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gcc_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp2_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_make: SKIPPED (0.00s) RESULTS - gi.GObjectIntrospectionTest.test_python: SKIPPED (0.00s) RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: SKIPPED (0.00s) RESULTS - ldd.LddTest.test_ldd: SKIPPED (0.01s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s) RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: SKIPPED (0.00s) RESULTS - pam.PamBasicTest.test_pam: SKIPPED (0.00s) RESULTS - perl.PerlTest.test_perl_works: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner: SKIPPED (0.01s) RESULTS - python.PythonTest.test_python3: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s) RESULTS - stap.StapTest.test_stap: SKIPPED (0.00s) RESULTS - systemd.SystemdBasicTests.test_systemd_basic: SKIPPED (0.00s) RESULTS - systemd.SystemdBasicTests.test_systemd_failed: SKIPPED (0.01s) RESULTS - systemd.SystemdBasicTests.test_systemd_list: SKIPPED (0.00s) RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: SKIPPED (0.00s) RESULTS - systemd.SystemdJournalTests.test_systemd_journal: SKIPPED (0.01s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_status: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: SKIPPED (0.00s) RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s) RESULTS - logrotate.LogrotateTest.test_2_logrotate: FAILED (0.73s) SUMMARY: core-image-full-cmdline () - Ran 62 tests in 9.934s core-image-full-cmdline - FAIL - Required tests failed (successes=10, skipped=51, failures=1, errors=0) ERROR: core-image-full-cmdline-1.0-r0 do_testimage: core-image-full-cmdline - FAILED - check the task log and the ssh log ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/work/qemux86_64-poky-linux/core-image-full-cmdline/1.0-r0/temp/log.do_testimage.8386 NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Failed ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage) failed with exit code '1' NOTE: Tasks Summary: Attempted 731 tasks of which 730 didn't need to be rerun and 1 failed. NOTE: The errors for this build are stored in /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/log/error-report/error_report_20191118002217.txt You can send the errors to a reports server by running: send-error-report /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/log/error-report/error_report_20191118002217.txt [-s server] NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending. Summary: 1 task failed: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage Summary: There was 1 ERROR message shown, returning a non-zero exit code. _running: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.01s) RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gcc_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp2_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_make: SKIPPED (0.00s) RESULTS - gi.GObjectIntrospectionTest.test_python: SKIPPED (0.00s) RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: SKIPPED (0.00s) RESULTS - ldd.LddTest.test_ldd: SKIPPED (0.01s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s) RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: SKIPPED (0.00s) RESULTS - pam.PamBasicTest.test_pam: SKIPPED (0.00s) RESULTS - perl.PerlTest.test_perl_works: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner: SKIPPED (0.01s) RESULTS - python.PythonTest.test_python3: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s) RESULTS - stap.StapTest.test_stap: SKIPPED (0.00s) RESULTS - systemd.SystemdBasicTests.test_systemd_basic: SKIPPED (0.00s) RESULTS - systemd.SystemdBasicTests.test_systemd_failed: SKIPPED (0.01s) RESULTS - systemd.SystemdBasicTests.test_systemd_list: SKIPPED (0.00s) RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: SKIPPED (0.00s) RESULTS - systemd.SystemdJournalTests.test_systemd_journal: SKIPPED (0.01s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_status: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: SKIPPED (0.00s) RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s) RESULTS - logrotate.LogrotateTest.test_2_logrotate: FAILED (0.73s) SUMMARY: core-image-full-cmdline () - Ran 62 tests in 9.934s core-image-full-cmdline - FAIL - Required tests failed (successes=10, skipped=51, failures=1, errors=0) ERROR: core-image-full-cmdline-1.0-r0 do_testimage: core-image-full-cmdline - FAILED - check the task log and the ssh log ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/work/qemux86_64-poky-linux/core-image-full-cmdline/1.0-r0/temp/log.do_testimage.8386 NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Failed ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage) failed with exit code '1' NOTE: Tasks Summary: Attempted 731 tasks of which 730 didn't need to be rerun and 1 failed. NOTE: The errors for this build are stored in /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/log/error-report/error_report_20191118002217.txt You can send the errors to a reports server by running: send-error-report /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-16822/tmp/log/error-report/error_report_20191118002217.txt [-s server] NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending. Summary: 1 task failed: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Maybe be a dup of 13602
Haven't been able to recreate this yet with bitbake -c testimage core-image-full-cmdline. Cutting down to just logrotate and the tests it requires to see if I can force it out by running the tests on loop.
Happened again on the autobuilder in master: https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/830
and another one https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/519
Another failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/524
I'm making some progress on this, but I don't have an actual patch for it yet. Here's a condensed test_logrotate, where lines 55 and 56 are added for debug: 37 @OETestDepends(['ssh.SSHTest.test_ssh']) 38 def test_logrotate(self): 39 status, output = self.target.run('mkdir $HOME/logrotate_dir') 40 msg = 'Could not create logrotate_dir. Output: %s' % output 41 self.assertEqual(status, 0, msg = msg) 42 43 status, output = self.target.run('echo "create \n include /etc/logrotate.d \n olddir $HOME/logrotate_dir" > /tmp/logrotate-test.conf') 44 status, output = self.target.run('logrotate -f /tmp/logrotate-test.conf') 45 46 msg = ('logrotate service could not be reloaded. Status and output: ' 47 '%s and %s' % (status, output)) 48 self.assertEqual(status, 0, msg = msg) 49 50 _, output = self.target.run('ls -la $HOME/logrotate_dir/ | wc -l') 51 msg = ('new logfile could not be created. List of files within log ' 52 'directory: %s' % ( 53 self.target.run('ls -la $HOME/logrotate_dir')[1])) 54 self.assertTrue(int(output)>=3, msg = msg) 55 testoutput = "output is {0} and the files are: \n {1}".format(output, self.target.run('ls -la $HOME/logrotate_dir/')[1]) 56 self.assertTrue(2 < 1, msg = testoutput) The second self.assertTrue is put there so that you can see the contents of the output variable and what's in the directory. When I run the image tests, it shows this: Traceback (most recent call last): File "/yow-lpggp31/tgamblin/oe-core.git/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/yow-lpggp31/tgamblin/oe-core.git/meta/lib/oeqa/runtime/cases/logrotate.py", line 57, in test_logrotate self.assertTrue(2 < 1, msg = testoutput) AssertionError: False is not true : output is 3 and the files are: total 2 drwxr-xr-x 2 root root 1024 Jan 14 19:30 . drwx------ 4 root root 1024 Jan 14 19:30 .. Note that that's three lines of output from that ls call, so in current usage the test is passing even though there aren't any files in the directory. Logging into the core-image-full-cmdline image confirms this (ls from coreutils): root@qemux86-64:~# ls -al /bin/ls lrwxrwxrwx 1 root root 17 Jan 14 19:48 /bin/ls -> /bin/ls.coreutils root@qemux86-64:~# ls root@qemux86-64:~# mkdir testdir root@qemux86-64:~# ls -al testdir/ total 2 drwxr-xr-x 2 root root 1024 Jan 14 19:50 . drwx------ 3 root root 1024 Jan 14 19:50 .. root@qemux86-64:~# Compare with a core-image-minimal build (ls from busybox): root@qemux86-64:~# ls -al /bin/ls lrwxrwxrwx 1 root root 19 Jan 14 19:53 /bin/ls -> /bin/busybox.nosuid root@qemux86-64:~# mkdir testdir root@qemux86-64:~# ls -al testdir/ drwxr-xr-x 2 root root 1024 Jan 14 19:54 . drwx------ 3 root root 1024 Jan 14 19:54 .. root@qemux86-64:~# So the wc -l in the logrotate test may returning 3 in some instances (passing the assert) even though there's nothing in logrotate_dir. Is it possible that the autobuilder is occasionally hitting a sequence of building and testing where it builds an image with busybox (e.g. core-image-minimal) in the same place as a previous image with coreutils (e.g. core-image-full-cmdline), but then tries to run e.g. bitbake -c testimage core-image-full-cmdline on those build artifacts? Bigger issue - maybe I'm misunderstanding how logrotate should be used, but it doesn't look like the call is ever actually rotating things into the path specified by olddir. Still debugging.
Patch submitted to clean up the test and make debugging easier (if it still occurs).
Created attachment 4615 [details] packages unique to both core-image-full-cmdline and core-image-sato-sdk that are NOT in core-image-sato Added a list of the packages that are in both of the images we've seen fail, but not in another image e.g. core-image-sato. Not sure yet if any of these could be part of the problem. Thinking I need a new patch with the verbose option added to the logrotate calls, to see what it thinks is happening when it's unable to rotate a log.
Was looking around and noticed this after running qemu on a fresh core-image-sato-sdk: hwclock: settimeofday() failed: Invalid argument touch: cannot touch '/var/log/wtmp': No such file or directory chown: cannot access '/var/log/wtmp': No such file or directory Failed to set owner -root- for -/var/log/wtmp-. chmod: cannot access '/var/log/wtmp': No such file or directory Failed to set mode -0664- for -/var/log/wtmp-. Still appears inconsistent. Based on what comes after that in the boot log, I'm inclined to think that it might be a problem with syslogd: Wed Jan 29 02:13:16 2020: hwclock: settimeofday() failed: Invalid argument Wed Jan 29 02:13:16 2020: Starting syslogd/klogd: done Wed Jan 29 02:13:16 2020: * Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon
I have a single build environment consistently reporting that issue with /var/log/wtmp (as shown above) at boot, but the test still passes (even with my modifications to the logrotate test) when I run "bitbake core-image-sato-sdk -c testimage".
I'm suspecting that this is an issue with meta/recipes-core/initscripts/initscripts-1.0/populate-volatile.sh. Doing a text search on the filesystem after booting with QEMU points to a file called /etc/volatile.cache, which contains the following lines (among others for other data in /var/log): touch "/var/log/wtmp"; chown root:root /var/log/wtmp || echo "Failed to set owner -root- for -/var/log/wtmp-." >/dev/tty0 2>&1; chmod 0664 /var/log/wtmp || echo "Failed to set mode -0664- for -/var/log/wtmp-." >/dev/tty0 2>&1 Which matches the error messages seen in some boot logs: Fri Jan 31 15:50:35 2020: touch: cannot touch '/var/log/wtmp': No such file or directory Fri Jan 31 15:50:35 2020: chown: cannot access '/var/log/wtmp': No such file or directory Fri Jan 31 15:50:35 2020: Failed to set owner -root- for -/var/log/wtmp-. Fri Jan 31 15:50:35 2020: chmod: cannot access '/var/log/wtmp': No such file or directory Fri Jan 31 15:50:35 2020: Failed to set mode -0664- for -/var/log/wtmp-. Possibly a problem with the filesystem not being available when this script starts up? Still digging...
Clearly, if the autobuilder is running into the issue where there's no wtmp log to rotate in the logrotate tests, then the issue lies with the initscripts and sysvinit. Looking at meta/recipes-core/initscripts/initscripts-1.0/volatiles, here are the files that populate-volatile.sh should create: 26 d root root 1777 /run/lock none 27 d root root 0755 /var/volatile/log none 28 d root root 1777 /var/volatile/tmp none 29 l root root 1777 /var/lock /run/lock 30 l root root 0755 /var/run /run 31 l root root 1777 /var/tmp /var/volatile/tmp 32 l root root 1777 /tmp /var/tmp 33 d root root 0755 /var/lock/subsys none 34 f root root 0664 /var/log/wtmp none 35 f root root 0664 /var/run/utmp none 36 l root root 0644 /etc/resolv.conf /var/run/resolv.conf 37 f root root 0644 /var/run/resolv.conf none I don't see any obvious patches in the commit log when searching for "populate-volatile" or "initscripts" that are close enough to the first report to be the smoking gun. I also still can't get testimage to actually fail for the builds where I see this error.
*** Bug 13602 has been marked as a duplicate of this bug. ***
This has been fixed with https://git.openembedded.org/openembedded-core/commit/?id=b9392403e96c6d880de844b44da2c7300fcf411c Bug 13784 has been opened to investigate the race condition that got this error occurring in the first place.