| Summary: | QEMU Segfaults with nographic option | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] Meta-yocto | Reporter: | Mark Hatle <mark.hatle> |
| Component: | meta-yocto | Assignee: | Edwin Zhai <edwin.zhai> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | major | ||
| Priority: | High | CC: | jessica.zhang, julie.n.fleischer, kevin.tian, matias.sundman, poky.bs.watcher, poky.watcher, richard.purdie, scott.a.garman, sgw |
| Version: | 0.9 | ||
| Target Milestone: | 1.0 M4 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | --- | |
|
Description
Mark Hatle
2011-01-07 10:26:04 UTC
Mark, I can't reproduce this bug with commit 4b055ed in poky master on ubuntu. Generaly speaking, "-vga cirrus" should be removed when specifying "-nographic", but qemu should handle this well and treat "nographic" as higher priority, which is observed on my side. Can you boot without "nographic"? If this is specific issue with "nographic", and it's hard to reproduce on my side, is it possible to let me use your machine? Thanks, edwin I removed the GL pass-through patches: - file://qemu-git-qemugl-host.patch \ - file://fix-nogl.patch \ - file://qemugl-allow-glxcontext-release.patch \ - file://init-info.patch \ - file://enable-i386-linux-user.patch \ - file://vmware-vga-fifo-rewind.patch \ and compiled with --disable-sdl. After that the crash went away. My host system has the nvidia graphics drivers installed on it. So likely it was a combination of the nvidia graphics drivers, and QEMU still using graphics resources even when -nographics was enabled. I believe this points to a bug in QEMU, that even when nographics is selected it attempts to initialize the OpenGL resources -- which triggers the incompatibility with the Nvidia drivers. This may or may not be triggered b the -vga cirrus... We do need to make sure there is a way to run QEMU without graphics being used at all, to avoid problems like this in the future. (In reply to comment #2) > I removed the GL pass-through patches: > and compiled with --disable-sdl. After that the crash went away. My host > system has the nvidia graphics drivers installed on it. So likely it was a > combination of the nvidia graphics drivers, and QEMU still using graphics > resources even when -nographics was enabled. > We do need to make sure there is a way to run QEMU without graphics being used > at all, to avoid problems like this in the future. Mark, As I can't reproduce it on my side, could you pls. retest it after upgrading qemu to 0.13? If it still exist, could you pls. use gdb to located where the seg fault come from? Only possible reason is that GL patch is not clean enough and touch display screen. In theory, it has nothing to do with nvidia hardware. If qemu runs well in normal, it should also with "nographic". There is no difference regarding hardware resource access. But it's just my guess, need further investigation. Mark, Could you pls. help to provide more info after upgrading qemu? Thanks, With the latest version of Poky (master as of 2011-02-24), QEMU still segfaults.. poky-qemu qemuppc nographic ... hda: unknown partition table /home/mhatle/git/poky/scripts/poky-qemu-internal: line 449: 21531 Segmentation fault (core dumped) $QEMUBIN -kernel $KERNEL $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT --append "$KERNCMDLINE $SCRIPT_KERNEL_OPT" By applying the patch on the top of the poky-contrib mhatle/qemu branch -- and running with "nographic" as above, it does not crash. This might be related to the automatic use of "-enable-gl" in the qemux86 and qemux86-64 configurations. Mark,
I have tried latest nightly build, and made sure it works well with "nographic". Can you try it also too see if this issue is machine related or build related?
Sorry for bother you again, but I need reproduce first then fix it:)
HOWTO:
1. download disk image
http://autobuilder.pokylinux.org/nightly/20110305-4/machines/qemu/qemumips/poky-image-sato-qemumips-20110303092454.rootfs.ext3
2. download mips kernel
http://autobuilder.pokylinux.org/nightly/20110305-4/machines/qemu/qemumips/vmlinux-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+76f30ca447fd0f06e76d0cb4935b016a1440e23e-r16-qemumips-20110303091203.bin
3. download native environment from
http://git.pokylinux.org/cgit.cgi/poky-contrib/plain/poky-lsb.tar.bz2?h=gzhai/for_mark
4. unpack it under /var/,
cd poky-lsb/0.9+snapshot/
source environment-setup-mips-poky-linux
poky-qemu mips-kernel-name disk-image-name nographic
To cause the crash you need a computer with an NVIDIA video card, and the proprietary (binary) nvidia X and OpenGL drivers. If you are unable to get to one, let me know and I'll attempt to get you access to a machine so you can reproduce the problem. I have not yet tested the exact build you are mentioning below, however I did try the HEAD of bernard on Tuesday an experienced exactly the same problem as before. Even with "nographic" being set (and also making sure that -enable-gl is not passed to qemu), it still crashes as QEMU starts. The only way I could work around this was remove the custom QEMU OpenGL patches and disable the SDL library when building qemu. (This is what the patch does at the top of mhatle/qemu.) Somehow we need to completely avoid even initializing OpenGL if -enable-gl is not passed to QEMU. I have tried the steps you outlined, it still crashes: hda: 556982 sectors (285 MB) w/256KiB Cache, CHS=552/255/63 hda: cache flushes supported /opt/poky-lsb/0.9+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/poky-qemu-internal: line 442: 22879 Segmentation fault (core dumped) $QEMUBIN -kernel $KERNEL $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT --append "$KERNCMDLINE $SCRIPT_KERNEL_OPT" Set 'tap0' nonpersistent Releasing lockfile of preconfigured tap device 'tap0' unfortunately the core that was dumped is bad and all it tells me is that again, it's a segmentation fault. The host system is a Fedora 13 system, with the binary Nvidia drivers installed: NVIDIA-Linux-x86_64-260.19.21.run The video card is: 03:00.0 VGA compatible controller: nVidia Corporation G98 [Quadro NVS 295] (rev a1) Mark, I can reproduce it. debug is on going... See also 698 Finally I found this bug is duplicated bug from 698, caused by nVidia GL lib. As long as qemu is linked with nVidia GL lib, even upstream qemu, there must be segfault regardless of "-nographic" setting. Possible root cause is: 1. nVidia GL lib provides some symbol override generic one, and cause segfauls when calling these symbol. 2. The .init section in nVidia GL lib dirty qemu's data structure and lead the final segfault. Kevin and I went through the related symbols, and excluded 1. So we doubt .init section need some fix. Before it got fixed in nVidia GL lib, we can have 3 options: 1. Add some checks in qemu's configure, and remove "-lGL" if detecting nVidia's GL lib installed. It cause trouble when building and running machine are different. 2. Use dlopen rather than "-lGL", so we can avoid touching bogus GL lib if specifying "-nographic" or no "-enable-gl". By this way, we can turn of gl when running. But need change source code to access the lib symbol, which is a little bit tricky. 3. Document it as known issue, and ask end user don't use nVidia driver. option 1 is not accepted since the build system could be different from the maching running Qemu. option 2 has some risk. dlopen/dlsyms are not a very stable approach, and we need to wrap all the opengl calls which are not trivial. Some NVidia driver version even overwrites dlsyms with its own version to cause trouble (http://www.nvnews.net/vbulletin/showthread.php?p=2070826). So it also needs careful test to ensure it not blocking other things when making slightly better in this nv case. option 3 is possibly the choice we can do now for 1.0, given the limited time and the risk from option 2. User could fall back to default xorg-x11 libgl driver to use Qemu, and we may address it in next point release Edwin will continue post his progress of course to see whether we can have another less risky option Edwin and I got another alternative. We could use LD_PRELOAD=/usr/lib/libGL.so to make sure xorg libGL is used instead of nVidia version. dynamic loader (ld.so) binds the symbol in reverse order, while LD_PRELOAD adds specificed library in the head. This ensures xorg libGL taking effect, while user can still use proprietary nVidia driver for his other desktop activity. We have tested that this approach basically works. Edwin will do more test and then add conditional check in poky-qemu script to make sure LD_PRELOADED prefixed only when nVidia proporietary driver is detected. Then it should have no impact to other build environment Mark, Can you have a look at where is the nVidia's libGL on your fedora system? /usr/lib/nvidia-current/libGL.so.xxx, or /usr/lib64/nvidia-current/libGL.so.xxx? I'll cook up a patch to skip nvidia's libGL if finding it. Edwin, In case this is helpful, on my 64-bit Ubuntu 10.10 system, NVIDIA's libGL is in /usr/lib64/nvidia-current/libGL.so Mark & Scott, I have pushed 2 patches @ the branch http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=gzhai/for_mark Can you have a try to see if it can fix this issue? Thanks, edwin Edwin, This appears to fix the problem on my system. Nice work! I have verified on Fedora 13 that the nvidia software gets installed into simply /usr/lib64. lrwxrwxrwx. 1 root root 10 Jan 21 10:54 /usr/lib64/libGL.so -> libGL.so.1* lrwxrwxrwx. 1 root root 18 Jan 21 10:54 /usr/lib64/libGL.so.1 -> libGL.so.260.19.21* -rwxr-xr-x. 1 root root 982536 Jan 21 10:54 /usr/lib64/libGL.so.260.19.21* The system version is removed, it would have been "/usr/lib64/libGL.so.1.2". You might be able to detect it though by looking for: libnvidia-cfg.so libnvidia-cfg.so.260.19.21 libnvidia-glcore.so.260.19.21 libnvidia-cfg.so.1 libnvidia-compiler.so.260.19.21 libnvidia-tls.so.260.19.21 this libGL.so.260.19.21 has internal dependencies on: 0x0000000000000001 (NEEDED) Shared library: [libnvidia-tls.so.260.19.21] 0x0000000000000001 (NEEDED) Shared library: [libnvidia-glcore.so.260.19.21] That is likely a clue as well. My suggestion is simply looking at the needed values on the libGL.so installed on the system, then looking for "libnvidia" contained within. If that is found then trigger a warning/error that this is known to conflict with QEMU. I attempted to run the poky-contrib gzhai/for_mark branch. It unfortunately resulted in the same core dump as before. As mentioned in my previous comment. Likely a good alternative is to check the libraries actually being loaded. One method would be: ldd tmp/sysroots/x86_64-linux/usr/bin/qemu | grep nvidia if the return code is "0", then qemu was linked against an nvidia driver and may fail to load. If it's != 0, then the nvidia libraries are not present and things are likely to work. yes, this way is more sane though we can expect all nvidia drivers falling into this pattern (In reply to comment #20) > yes, this way is more sane though we can expect all nvidia drivers falling into > this pattern 'can'->'can't' Mark, Can you test if following fix works on your fedora system? thanks, http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=gzhai/fix2&id=f596757000465a4b8350e16f21553a23b8bbedfa After discussion with Mark, we agree we just give warnings to end user and document this as known issue, as different distribution need different solution. New fix is @ http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=gzhai/fix3&id=1c56783be10e6d51a801ccbee9e9ec5ae2c5db32 This patch looks good to me. Mark, I have created a new bug requesting for the doc: bug 902. Could you pls. provide fix for fedora? Thanks, Got fixed in master @ c0298557bad2e431f6dc87b6fda061cab561e14c *** Bug 698 has been marked as a duplicate of this bug. *** |