| Summary: | qemu images stuck at boot with build 20140520-2 | ||
|---|---|---|---|
| Product: | [Runtime] General Runtime | Reporter: | Ke Zou <ke.zou> |
| Component: | General Runtime | Assignee: | Saul Wold <sgw> |
| Status: | RESOLVED NOTABUG | QA Contact: | |
| Severity: | major | ||
| Priority: | Medium | CC: | alexandru.c.georgescu, jessica.zhang, laurentiu.palcu, yi.zhao |
| Version: | 1.7 | ||
| Target Milestone: | 1.7 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Ke Zou
2014-05-23 03:42:11 UTC
We really need to start doing proper bug reports. When did this work last ? Is the network alive when the boot is "hung". There have been nearly no kernel changes in a while, so chances are this is a userspace hang, and not the kernel. But we need some information to go on. Seems it is a toolchain issue. When I use an old toolchain ( 20140507 build, git rev: master/a671ced0ea7c5775d2f2699d3a720c6d0c11f743 ), the qemu can boot without problems. I also diff the runqemu args with 20140507 toolchain and 20140520 toolchian, but there is no different: 20140507 build toolchain: ################### $ runqemu qemuppc vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 Continuing with the following parameters: KERNEL: [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] ROOTFS: [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] FSTYPE: [ext3] Setting up tap interface under sudo Acquiring lockfile for tap0... Running qemu-system-ppc... /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc -kernel vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no -cpu G4 -M mac99 -hda /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 --append "root=/dev/hda rw console=ttyS0 console=tty ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " ##################### 20140520 toolchain: ##################### $ runqemu qemuppc vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 Continuing with the following parameters: KERNEL: [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] ROOTFS: [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] FSTYPE: [ext3] Setting up tap interface under sudo Acquiring lockfile for tap0... Running qemu-system-ppc... /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc -kernel vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no -cpu G4 -M mac99 -hda /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 --append "root=/dev/hda rw console=ttyS0 console=tty ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " VNC server running on `::1:5900' ################### I can confirm that I was able to boot a freshly built qemuppc (rt image no less) sato build today. One last confirmation. Is it the kernel, or userspace that is having issues with the new toolchain ? i.e. if you boot the new toolchain userspace with the old toolchain kernel, does it boot ? (and vice versa). (In reply to comment #2) > Seems it is a toolchain issue. > When I use an old toolchain ( 20140507 build, git rev: > master/a671ced0ea7c5775d2f2699d3a720c6d0c11f743 ), the qemu can boot without > problems. > > I also diff the runqemu args with 20140507 toolchain and 20140520 toolchian, > but there is no different: > > 20140507 build toolchain: > ################### > $ runqemu qemuppc > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > Continuing with the following parameters: > KERNEL: > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > ROOTFS: > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > FSTYPE: [ext3] > Setting up tap interface under sudo > Acquiring lockfile for tap0... > Running qemu-system-ppc... > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > -kernel > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > -cpu G4 -M mac99 -hda > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > --append "root=/dev/hda rw console=ttyS0 console=tty > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > ##################### > > 20140520 toolchain: > ##################### > $ runqemu qemuppc > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > Continuing with the following parameters: > KERNEL: > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > ROOTFS: > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > FSTYPE: [ext3] > Setting up tap interface under sudo > Acquiring lockfile for tap0... > Running qemu-system-ppc... > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > -kernel > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > -cpu G4 -M mac99 -hda > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > --append "root=/dev/hda rw console=ttyS0 console=tty > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > VNC server running on `::1:5900' > > ################### hi Bruce, It looks that it is not about the newly built qemuppc image itself. And I assume that if you use the toolchain in the 20140520 build which lies in: http://autobuilder.yoctoproject.org/pub/nightly/20140520-2/toolchain/ , your freshly built qemu image will "hung" during boot, too. The QA procedure is like this: Get toolchain setup files Setup toolchain in local /opt/poky/xxx/ Using runqemu in your new toolchain to startup qemu image Plus, I noticed that if we replace the file that "runqemu" finally invokes: (take qemuppc on my machine for example) /opt/poky/1.6/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc If we replace this file with older one in 0507 build, everything's fine. I can only dig this far since this is a binary file, and I'm not sure what you mean by "toolchain userspace" and "toolchain kernel", maybe we need to add toolchain provider in the loop. (In reply to comment #3) > I can confirm that I was able to boot a freshly built qemuppc (rt image no > less) sato > build today. > > One last confirmation. Is it the kernel, or userspace that is having issues > with the > new toolchain ? > > i.e. if you boot the new toolchain userspace with the old toolchain kernel, > does it boot ? > (and vice versa). > > > (In reply to comment #2) > > Seems it is a toolchain issue. > > When I use an old toolchain ( 20140507 build, git rev: > > master/a671ced0ea7c5775d2f2699d3a720c6d0c11f743 ), the qemu can boot without > > problems. > > > > I also diff the runqemu args with 20140507 toolchain and 20140520 toolchian, > > but there is no different: > > > > 20140507 build toolchain: > > ################### > > $ runqemu qemuppc > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > > Continuing with the following parameters: > > KERNEL: > > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > > ROOTFS: > > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > > FSTYPE: [ext3] > > Setting up tap interface under sudo > > Acquiring lockfile for tap0... > > Running qemu-system-ppc... > > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > > -kernel > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > > -cpu G4 -M mac99 -hda > > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > > --append "root=/dev/hda rw console=ttyS0 console=tty > > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > > ##################### > > > > 20140520 toolchain: > > ##################### > > $ runqemu qemuppc > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > > Continuing with the following parameters: > > KERNEL: > > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > > ROOTFS: > > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > > FSTYPE: [ext3] > > Setting up tap interface under sudo > > Acquiring lockfile for tap0... > > Running qemu-system-ppc... > > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > > -kernel > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > > -cpu G4 -M mac99 -hda > > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > > --append "root=/dev/hda rw console=ttyS0 console=tty > > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > > VNC server running on `::1:5900' > > > > ################### (In reply to comment #4) > hi Bruce, > > It looks that it is not about the newly built qemuppc image itself. > And I assume that if you use the toolchain in the 20140520 build which lies > in: > http://autobuilder.yoctoproject.org/pub/nightly/20140520-2/toolchain/ > , your freshly built qemu image will "hung" during boot, too. > > The QA procedure is like this: > Get toolchain setup files > Setup toolchain in local /opt/poky/xxx/ > Using runqemu in your new toolchain to startup qemu image > > > Plus, I noticed that if we replace the file that "runqemu" finally invokes: > (take qemuppc on my machine for example) > /opt/poky/1.6/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > If we replace this file with older one in 0507 build, everything's fine. This is just what I wanted to find out. It indicates that we have a qemu and the new toolchain causing an issue, and not the new toolchain and the kernel causing an issue. What I meant when mentioning the "toolchain kernel" and "toolchain userspace" is fundamentally what you did. I wanted you to take the old userspace and tools built with the old toolchain, and the kernel built using the new toolchain. If that worked, then we know to not look at the kernel for the hang. Your test did that same isolation. Can we make this bug new again, but assign it to qemu and not the kernel/BSPs ? > > I can only dig this far since this is a binary file, and I'm not sure what > you mean by "toolchain userspace" and "toolchain kernel", > maybe we need to add toolchain provider in the loop. > > > > (In reply to comment #3) > > I can confirm that I was able to boot a freshly built qemuppc (rt image no > > less) sato > > build today. > > > > One last confirmation. Is it the kernel, or userspace that is having issues > > with the > > new toolchain ? > > > > i.e. if you boot the new toolchain userspace with the old toolchain kernel, > > does it boot ? > > (and vice versa). > > > > > > (In reply to comment #2) > > > Seems it is a toolchain issue. > > > When I use an old toolchain ( 20140507 build, git rev: > > > master/a671ced0ea7c5775d2f2699d3a720c6d0c11f743 ), the qemu can boot without > > > problems. > > > > > > I also diff the runqemu args with 20140507 toolchain and 20140520 toolchian, > > > but there is no different: > > > > > > 20140507 build toolchain: > > > ################### > > > $ runqemu qemuppc > > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > > > > Continuing with the following parameters: > > > KERNEL: > > > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > > > ROOTFS: > > > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > > > FSTYPE: [ext3] > > > Setting up tap interface under sudo > > > Acquiring lockfile for tap0... > > > Running qemu-system-ppc... > > > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > > > -kernel > > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > > > -cpu G4 -M mac99 -hda > > > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > > > --append "root=/dev/hda rw console=ttyS0 console=tty > > > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > > > ##################### > > > > > > 20140520 toolchain: > > > ##################### > > > $ runqemu qemuppc > > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > > core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > > > > Continuing with the following parameters: > > > KERNEL: > > > [vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin] > > > ROOTFS: > > > [/home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3] > > > FSTYPE: [ext3] > > > Setting up tap interface under sudo > > > Acquiring lockfile for tap0... > > > Running qemu-system-ppc... > > > /opt/poky/1.6+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-ppc > > > -kernel > > > vmlinux--3.14.2+git0+4df1e2ed99_5ac4b714c6-r0-qemuppc-20140513195124.bin > > > -net nic,model=pcnet -net tap,vlan=0,ifname=tap0,script=no,downscript=no > > > -cpu G4 -M mac99 -hda > > > /home/zhaoyi/qemuppc/core-image-sato-sdk-qemuppc-20140520222950.rootfs.ext3 > > > -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -m 256 > > > --append "root=/dev/hda rw console=ttyS0 console=tty > > > ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=256M " > > > VNC server running on `::1:5900' > > > > > > ################### I think the issue is Qemu, may not even be runqemu. The indication is that there is a line: VNC server running on `127.0.0.1:5900' at the end of the runqemu exection on the newer version vs the older version, if you start up a vncviewer localhost on the system (or point a vncview to your remote system), you will see the video out put of that qemu session. I am looking into what changed to cause it to prefer VNC vs X11 display. Due to a change in the qemu, it starts up in VNC mode by default, there is a note about this in the Migration Guide about setting up SDL in local.conf, if you are building locally and are using an older local.conf you might need to set the following: # # Qemu configuration # # By default qemu will build with a builtin VNC server where graphical output can be # seen. The two lines below enable the SDL backend too. This assumes there is a # libsdl library available on your build system. PACKAGECONFIG_pn-qemu-native = "sdl" ASSUME_PROVIDED += "libsdl-native" |