https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/3626 qemuppc ping test failure on debian10-ty-3 Note the "Configuring network interfaces... ip: RTNETLINK answers: File exists" below as that is likely related to the issue. pokybuild@debian10-ty-3:~/yocto-worker/qemuppc/build/build-renamed/tmp/work/qemuppc-poky-linux/core-image-minimal/1.0-r0/testimage$ cat qemu_boot_log.20210707012539 C>> annot manage 'undefined' PCI device type '<NULL>': >> 1af4 1005 (0 ff 0) >> ============================================================= >> OpenBIOS 1.1 [Mar 16 2021 08:16] >> Configuration device id QEMU version 1 machine id 1 >> CPUs: 1 >> Memory: 256M >> UUID: 00000000-0000-0000-0000-000000000000 >> CPU type PowerPC,G4 milliseconds isn't unique. >> [ppc] Kernel already loaded (0x01000000 + 0x012739c0) (initrd 0x00000000 + 0x00000000) >> [ppc] Kernel command line: root=/dev/vda rw mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 console=ttyS0 console=ttyS1 console=tty console=ttyS0 printk.time=1 >> switching to new context: hwclock: can't open '/dev/misc/rtc': No such file or directory chmod: /var/log/wtmp: No such file or directory Failed to set mode -0664- for -/var/log/wtmp-. INIT: Entering runlevel: 5 Configuring network interfaces... ip: RTNETLINK answers: File exists hwclock: can't open '/dev/misc/rtc': No such file or directory Starting syslogd/klogd: done Poky (Yocto Project Reference Distro) 3.3+snapshot-8a3f38f2908b9c431cb0577f98ac955a9100319f qemuppc /dev/ttyS0 qemuppc login:
Looking at other successful qemuppc logs, th RTNETLINK message appears normal but the "chmod: /var/log/wtmp: No such file or directory Failed to set mode -0664- for -/var/log/wtmp-" is not.
I reran the test manually in the same build directory with the same image. The most interesting thing was on this run: QMP connected to QEMU at 07/07/21 07:26:56 and took 3.456681966781616 seconds QMP released QEMU at 07/07/21 07:26:57 and took 1.1139729022979736 seconds from connect runqemu - INFO - Copy done in 0.14536833763122559 seconds whilst on the failed run: QMP connected to QEMU at 07/07/21 01:26:34 and took 24.899512767791748 seconds QMP released QEMU at 07/07/21 01:27:18 and took 43.95131969451904 seconds from connect runqemu - INFO - Copy done in 0.26656150817871094 seconds In some tests the wtmp error is present, in some it is not, it doesn't affect the ping success/failure. The RTNETLINK message is always there. In one test run it took a long time, core-image-minimal () - Ran 66 tests in 17.148s vs core-image-minimal () - Ran 66 tests in 2642.934s and in the latter, the time seemed to be spent getting single characters over serial very slowly with periodic timeouts.
dmesg for the failing case shows boot in 13.4s, the success case was 9.2s. dmesg is pretty much identical except the usb stack init is later compared to the rest of the system in the success case. top and ps output show the same processes running with no significant looking differences.
Created attachment 4810 [details] Failure dmesg
Created attachment 4811 [details] Success dmesg
Netconsole might be an option.
I think that Paul has found a worlflow that might help with qemuppc hangs...
Not seen in months.
Occurred again in the dunfell 3.1.32 release build: https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/8596/steps/14/logs/stdio Not reopening though since it happens less than once a year.