Bug 782

(In reply to comment #0)
> Testing Environment
> ===============================
> Host: qemumips-sato-sdk
> Kernel: Linux version 2.6.37.2-yocto-standard+
> Image location:
> http://autobuilder.pokylinux.org/nightly/20110226-1/qemu/qemumips/
> commit info: e08dc5aaaeba61e332caab4a3df5750df6b49ac1
> Steps
> ===============================
> 1. Run command 'x11vnc -display :0.0' on target.
> 2. On a client, run connmand 'vncviewer 192.168.7.2:0' (TargetIp: 192.168.7.2)
> Observed Result
> ===============================
> On target after running command 'x11vnc -display :0.0'
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> root@qemumips:~# x11vnc -display :0.0
> ###############################################################
> #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@#
> #@                                                           @#
> #@  **  WARNING  **  WARNING  **  WARNING  **  WARNING  **   @#
> #@                                                           @#
> #@        YOU ARE RUNNING X11VNC WITHOUT A PASSWORD!!        @#
> #@                                                           @#
> #@  This means anyone with network access to this computer   @#
> #@  may be able to view and control your desktop.            @#
> #@                                                           @#
> #@ >>> If you did not mean to do this Press CTRL-C now!! <<< @#
> #@                                                           @#
> #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@#
> #@                                                           @#
> #@  You can create an x11vnc password file by running:       @#
> #@                                                           @#
> #@       x11vnc -storepasswd password /path/to/passfile      @#
> #@  or   x11vnc -storepasswd /path/to/passfile               @#
> #@  or   x11vnc -storepasswd                                 @#
> #@                                                           @#
> #@  (the last one will use ~/.vnc/passwd)                    @#
> #@                                                           @#
> #@  and then starting x11vnc via:                            @#
> #@                                                           @#
> #@      x11vnc -rfbauth /path/to/passfile                    @#
> #@                                                           @#
> #@  an existing ~/.vnc/passwd file from another VNC          @#
> #@  application will work fine too.                          @#
> #@                                                           @#
> #@  You can also use the -passwdfile or -passwd options.     @#
> #@  (note -passwd is unsafe if local users are not trusted)  @#
> #@                                                           @#
> #@  Make sure any -rfbauth and -passwdfile password files    @#
> #@  cannot be read by untrusted users.                       @#
> #@                                                           @#
> #@  Use x11vnc -usepw to automatically use your              @#
> #@  ~/.vnc/passwd or ~/.vnc/passwdfile password files.       @#
> #@  (and prompt you to create ~/.vnc/passwd if neither       @#
> #@  file exists.)  Under -usepw, x11vnc will exit if it      @#
> #@  cannot find a password to use.                           @#
> #@                                                           @#
> #@                                                           @#
> #@  Even with a password, the subsequent VNC traffic is      @#
> #@  sent in the clear.  Consider tunnelling via ssh(1):      @#
> #@                                                           @#
> #@    http://www.karlrunge.com/x11vnc/#tunnelling            @#
> #@                                                           @#
> #@  Or using the x11vnc SSL options: -ssl and -stunnel       @#
> #@                                                           @#
> #@  Please Read the documention for more info about          @#
> #@  passwords, security, and encryption.                     @#
> #@                                                           @#
> #@    http://www.karlrunge.com/x11vnc/faq.html#faq-passwd    @#
> #@                                                           @#
> #@  To disable this warning use the -nopw option, or put     @#
> #@  'nopw' on a line in your ~/.x11vncrc file.               @#
> #@                                                           @#
> #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@#
> ###############################################################
> 28/02/2011 09:45:05 x11vnc version: 0.9.12 lastmod: 2010-09-09  pid: 349
> 28/02/2011 09:45:05 Using X display :0.0
> 28/02/2011 09:45:05 rootwin: 0x43 reswin: 0x1200001 dpy: 0x7f6e60
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 ------------------ USEFUL INFORMATION ------------------
> 28/02/2011 09:45:05 X DAMAGE available on display, using it for polling hints.
> 28/02/2011 09:45:05   To disable this behavior use: '-noxdamage'
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05   Most compositing window managers like 'compiz' or 'beryl'
> 28/02/2011 09:45:05   cause X DAMAGE to fail, and so you may not see any screen
> 28/02/2011 09:45:05   updates via VNC.  Either disable 'compiz' (recommended)
> or
> 28/02/2011 09:45:05   supply the x11vnc '-noxdamage' command line option.
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 Wireframing: -wireframe mode is in effect for window moves.
> 28/02/2011 09:45:05   If this yields undesired behavior (poor response,
> painting
> 28/02/2011 09:45:05   errors, etc) it may be disabled:
> 28/02/2011 09:45:05    - use '-nowf' to disable wireframing completely.
> 28/02/2011 09:45:05    - use '-nowcr' to disable the Copy Rectangle after the
> 28/02/2011 09:45:05      moved window is released in the new position.
> 28/02/2011 09:45:05   Also see the -help entry for tuning parameters.
> 28/02/2011 09:45:05   You can press 3 Alt_L's (Left "Alt" key) in a row to 
> 28/02/2011 09:45:05   repaint the screen, also see the -fixscreen option for
> 28/02/2011 09:45:05   periodic repaints.
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 XFIXES available on display, resetting cursor mode
> 28/02/2011 09:45:05   to: '-cursor most'.
> 28/02/2011 09:45:05   to disable this behavior use: '-cursor arrow'
> 28/02/2011 09:45:05   or '-noxfixes'.
> 28/02/2011 09:45:05 using XFIXES for cursor drawing.
> 28/02/2011 09:45:05 GrabServer control via XTEST.
> Xlib:  extension "RECORD" missing on display ":0.0".
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 The RECORD X extension was not found on the display.
> 28/02/2011 09:45:05 If your system has disabled it by default, you can
> 28/02/2011 09:45:05 enable it to get a nice x11vnc performance speedup
> 28/02/2011 09:45:05 for scrolling by putting this into the "Module" section
> 28/02/2011 09:45:05 of /etc/X11/xorg.conf or /etc/X11/XF86Config:
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05   Section "Module"
> 28/02/2011 09:45:05   ...
> 28/02/2011 09:45:05       Load    "record"
> 28/02/2011 09:45:05   ...
> 28/02/2011 09:45:05   EndSection
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 XKEYBOARD: number of keysyms per keycode 6 is greater
> 28/02/2011 09:45:05   than 4 and 2 keysyms are mapped above 4.
> 28/02/2011 09:45:05   Automatically switching to -xkb mode.
> 28/02/2011 09:45:05   If this makes the key mapping worse you can
> 28/02/2011 09:45:05   disable it with the "-noxkb" option.
> 28/02/2011 09:45:05   Also, remember "-remap DEAD" for accenting characters.
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 X FBPM extension not supported.
> 28/02/2011 09:45:05 X display is not capable of DPMS.
> 28/02/2011 09:45:05 --------------------------------------------------------
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 Default visual ID: 0x21
> 28/02/2011 09:45:05 Read initial data from X display into framebuffer.
> 28/02/2011 09:45:05 initialize_screen: fb_depth/fb_bpp/fb_Bpl 16/16/1280
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 X display :0.0 is 16bpp depth=16 true color
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 Autoprobing TCP port 
> 28/02/2011 09:45:05 Autoprobing selected port 5900
> 28/02/2011 09:45:05 Listening also on IPv6 port 5900 (socket 7)
> 28/02/2011 09:45:05 fb read rate: 33 MB/sec
> 28/02/2011 09:45:05 The X server says there are 10 mouse buttons.
> 28/02/2011 09:45:05 screen setup finished.
> 28/02/2011 09:45:05 
> 28/02/2011 09:45:05 WARNING: You are running x11vnc WITHOUT a password.  See
> 28/02/2011 09:45:05 WARNING: the warning message printed above for more info.
> 28/02/2011 09:45:05 
> The VNC desktop is:      qemumips:0
> PORT=5900
> ******************************************************************************
> Have you tried the x11vnc '-ncache' VNC client-side pixel caching feature yet?
> The scheme stores pixel data offscreen on the VNC viewer side for faster
> retrieval.  It should work with any VNC viewer.  Try it by running:
>     x11vnc -ncache 10 ...
> One can also add -ncache_cr for smooth 'copyrect' window motion.
> More info: http://www.karlrunge.com/x11vnc/faq.html#faq-client-caching
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> On client after running command 'vncviewer 192.168.7.2:0'
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> charles@PEK-LWANG2-D1:~$ vncviewer 192.168.7.2:0
> Connected to RFB server, using protocol version 3.8
> No authentication needed
> Authentication successful
> On *TARGET* display:
> 28/02/2011 09:49:02 Got connection from client 192.168.7.1
> 28/02/2011 09:49:02   other clients:
> 28/02/2011 09:49:02 Disabled X server key autorepeat.
> 28/02/2011 09:49:02   to force back on run: 'xset r on' (3 times)
> 28/02/2011 09:49:02 incr accepted_client=1 for 192.168.7.1:57164  sock=8
> 28/02/2011 09:49:02 Client Protocol Version 3.8
> 28/02/2011 09:49:02 Protocol version sent 3.8, using 3.8
> 28/02/2011 09:49:02 rfbProcessClientSecurityType: executing handler for type 1
> 28/02/2011 09:49:02 rfbProcessClientSecurityType: returning securityResult for
> client rfb version >= 3.8
> 28/02/2011 09:49:07 client_set_net: 192.168.7.1  0.1479
> 28/02/2011 09:49:07 created   xdamage object: 0x1200018
> 28/02/2011 09:49:07 copy_tiles: allocating first_line at size 21
> 28/02/2011 09:49:11 created selwin: 0x1200019
> 28/02/2011 09:49:11 called initialize_xfixes()
> And there is *not* have an virtual X desktop of target pop-up on the client.

Just double confirm, does the issue exists with other images, like qemuarm, on the same host?

1. On the same host, this issue does *not* occur with qemuarm-sato-sdk image.
2. On the other host that qemuarm can pop-up the virtual X desktop, the same issue also occurs. 

So i think this issue is *not* impact by my host.

After tracking the message between vnc server/client, it is finally root caused as word endian issue.

mips  linux is bigendian, while x86 linux is little endian, so there should be endian convention between the vnc client/server.  the code does have the endian convertion logic and the logic works well in most case. the only issue is that in vncserver mips compiling, there is missing flag "-DLIBVNCSERVER_WORDS_BIGENDIAN", thus the vncserver in mips has wrong endian.

Adding the flag can fix this issue. patch is ongoing.

Fixed by http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=87290a8497e892a65bde4755c4fee8b8ca93424b

This issue does not occur on current nightly build (20110313-2).

Image location: http://autobuilder.yoctoproject.org/nightly/20110313-2/machines/qemu/qemumips/

commit a33a2cc024f9c9147c92c5da70d8fbb631b968f9