randomly broken input despite qemu protocol extensions

[vthriller]

How to reproduce

$ grep xkb .config/sway/config
input * xkb_layout "us,ru"
input * xkb_variant ",winkeys"
input * xkb_options "grp:caps_toggle,grp_led:scroll,compose:menu"

Then:

• connect to wayvnc
• type in: a, shift-a, caps, a, shift-a, caps, menu, o, c (that is, "aA", change layout, "фФ", change layout, compose key, "©")
• observe unexpected text
• while staying connected, issue swaymsg reload
• type in the same thing
• observe different unexpected text

Actual behaviour

Varies greatly:

client	output before swaymsg reload	output after swaymsg reload
tigervnc (clean xkbmap)	aAaa (shows context menu)	aФAф©
tigervnc (grp:caps_toggle,compose:menu)	aAaa (shows context menu)	aФAф©
turbovnc (clean xkbmap)	aAaA (shows context menu)	aФAФ©
turbovnc (grp:caps_toggle,compose:menu)	aAoc	aФoc

Also: when reconnected, one would again get pre-reload output (until the next reload, that is).

Expected behaviour

The aforementioned sequence should produce "aAфФ©". This is exactly what happens when sway receives keystrokes through qemu -vnc and emulated input device instead of wayvnc, regardless of client's software and xkbmap settings.

Versions

wayvnc: v0.8.0-15d09b0 (HEAD)
neatvnc: v0.8.0-46432ce (HEAD)
aml: v0.3.0-0-gb83f357 (HEAD)
sway version 1.8.1
TigerVNC Viewer v1.13.1
TurboVNC Viewer v3.1.1 (build 20240311) [JVM: amd64]

wayvnc was built from the git tag, same for neatvnc/aml. The 0.7.2/0.7.0/0.3.0 combination is also affected.

Logs

I'm only posting logs for clients with non-standard xkb settings. They only differ in keysym anyway, and the whole point of QEMU protocol extension is in avoiding exactly that.

Note that both wayvnc and tigervnc/turbovnc signal support for the QEMU extension.

TurboVNC

vncviewer -loglevel 100

CConn: Enabling QEMU Extended Key Event
...
CConn: key PRESS, code A (65), loc STANDARD, char 'a' => 0x0061
CConn: key release, code A (65), loc STANDARD, char 'a' => 0x0061
CConn: key PRESS, code Shift (16), loc RIGHT => 0xffe2
CConn: key PRESS, code A (65), loc STANDARD, char 'A' RShift => 0x0041
CConn: key release, code A (65), loc STANDARD, char 'A' RShift => 0x0041
CConn: key release, code Shift (16), loc RIGHT RShift => 0xffe2
CConn: key PRESS, code Unknown keyCode: 0x0 (0), loc UNKNOWN IGNORED
CConn: key release, code Unknown keyCode: 0x0 (0), loc UNKNOWN IGNORED
CConn: key PRESS, code A (65), loc UNKNOWN, char 1092 => 0x06c6
CConn: key release, code A (65), loc UNKNOWN, char 1092 => 0x06c6
CConn: key PRESS, code Shift (16), loc RIGHT => 0xffe2
CConn: key PRESS, code A (65), loc UNKNOWN, char 1060 RShift => 0x06e6
CConn: key release, code A (65), loc UNKNOWN, char 1060 RShift => 0x06e6
CConn: key release, code Shift (16), loc RIGHT RShift => 0xffe2
CConn: key PRESS, code Unknown keyCode: 0x0 (0), loc UNKNOWN IGNORED
CConn: key release, code Unknown keyCode: 0x0 (0), loc UNKNOWN IGNORED
CConn: key PRESS, code Compose (65312), loc STANDARD => 0x100ffff
CConn: key release, code Compose (65312), loc STANDARD => 0x100ffff
CConn: key PRESS, code O (79), loc STANDARD, char 'o' => 0x006f
CConn: key PRESS, code C (67), loc STANDARD, char 'c' => 0x0063
CConn: key release, code O (79), loc STANDARD, char 'o' => 0x006f
CConn: key release, code C (67), loc STANDARD, char 'c' => 0x0063

wayvnc -L trace

DEBUG: ../subprojects/neatvnc/src/server.c: 1048: Client 0x5599322f6880 set encodings: cursor,desktop-size,extended-desktop-size,qemu-extended-key-event,tight,copyrect,zrle,hextile,raw
...
DEBUG: ../src/keyboard.c: 146: symbol=a level=0 code=AC01 released
DEBUG: ../src/keyboard.c: 146: symbol=a level=0 code=AC01 pressed
DEBUG: ../src/keyboard.c: 146: symbol=Shift_R level=0 code=RTSH released
DEBUG: ../src/keyboard.c: 146: symbol=A level=1 code=AC01 released
DEBUG: ../src/keyboard.c: 146: symbol=A level=1 code=AC01 pressed
DEBUG: ../src/keyboard.c: 146: symbol=Shift_R level=0 code=RTSH pressed
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: Cyrillic_ef
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: Cyrillic_ef
DEBUG: ../src/keyboard.c: 146: symbol=Shift_R level=0 code=RTSH released
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: Cyrillic_EF
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: Cyrillic_EF
DEBUG: ../src/keyboard.c: 146: symbol=Shift_R level=0 code=RTSH pressed
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: UFFFF
ERROR: ../src/keyboard.c: 400: Failed to look up keyboard symbol: UFFFF
DEBUG: ../src/keyboard.c: 146: symbol=o level=0 code=AD09 released
DEBUG: ../src/keyboard.c: 146: symbol=c level=0 code=AB03 released
DEBUG: ../src/keyboard.c: 146: symbol=o level=0 code=AD09 pressed
DEBUG: ../src/keyboard.c: 146: symbol=c level=0 code=AB03 pressed

TigerVNC

vncviewer -Log='*:stderr:500'

 CMsgHandler: supportsQEMUKeyEvent
...
 Viewport:    Key pressed: 0x001e => XK_a (0x0061)
 Viewport:    Key released: 0x001e => XK_a (0x0061)
 Viewport:    Key pressed: 0x0036 => XK_Shift_R (0xffe2)
 Viewport:    Key pressed: 0x001e => XK_A (0x0041)
 Viewport:    Key released: 0x001e => XK_A (0x0041)
 Viewport:    Key released: 0x0036 => XK_Shift_R (0xffe2)
 Viewport:    Key pressed: 0x003a => XK_ISO_Next_Group (0xfe08)
 Viewport:    Key released: 0x003a => XK_ISO_Next_Group (0xfe08)
 Viewport:    Key pressed: 0x001e => XK_Cyrillic_ef (0x06c6)
 Viewport:    Key released: 0x001e => XK_Cyrillic_ef (0x06c6)
 Viewport:    Key pressed: 0x0036 => XK_Shift_R (0xffe2)
 Viewport:    Key pressed: 0x001e => XK_Cyrillic_EF (0x06e6)
 Viewport:    Key released: 0x001e => XK_Cyrillic_EF (0x06e6)
 Viewport:    Key released: 0x0036 => XK_Shift_R (0xffe2)
 Viewport:    Key pressed: 0x003a => XK_ISO_Next_Group (0xfe08)
 Viewport:    Key released: 0x003a => XK_ISO_Next_Group (0xfe08)
 Viewport:    Key pressed: 0x00dd => XK_Multi_key (0xff20)
 Viewport:    Key released: 0x00dd => XK_Multi_key (0xff20)
 Viewport:    Key pressed: 0x0018 => XK_o (0x006f)
 Viewport:    Key pressed: 0x002e => XK_c (0x0063)
 Viewport:    Key released: 0x0018 => XK_o (0x006f)
 Viewport:    Key released: 0x002e => XK_c (0x0063)

The first line requires a patch:

diff --git a/common/rfb/CMsgHandler.cxx b/common/rfb/CMsgHandler.cxx
index 8cdfc451..0a0bb7e7 100644
--- a/common/rfb/CMsgHandler.cxx
+++ b/common/rfb/CMsgHandler.cxx
@@ -80,6 +80,7 @@ void CMsgHandler::endOfContinuousUpdates()
 
 void CMsgHandler::supportsQEMUKeyEvent()
 {
+  vlog.debug("supportsQEMUKeyEvent");
   server.supportsQEMUKeyEvent = true;
 }

wayvnc -L trace

DEBUG: ../subprojects/neatvnc/src/server.c: 1048: Client 0x5599322f6880 set encodings: cursor,desktop-size,extended-desktop-size,qemu-extended-key-event,tight,copyrect,zrle,hextile,rre,copyrect,raw

That's it! Not a single line from src/keyboard.c was ever emitted.

Other observations

TurboVNC behaves like QEMU extension is not even supported, despite logging otherwise. First, it produces no text for caps, a if client-side server also uses caps_toggle; secondly, compare its log from above to the one emitted after connecting to QEMU (note the RFB keycode part, and also lack of IGNORED):

vncviewer -loglevel 100

CConn: Enabling QEMU Extended Key Event
CConn: Disabling automatic desktop resizing because the server doesn't support it.
CConn: Enabling LED State
CConn: Server's LED state: 0x2
...
CConn: key PRESS, code A (65), loc STANDARD, char 'a', RFB keycode 0x1e => 0x0061
CConn: key release, code A (65), loc STANDARD, char 'a', RFB keycode 0x1e => 0x0061
CConn: key PRESS, code Shift (16), loc RIGHT, RFB keycode 0x36 => 0xffe2
CConn: key PRESS, code A (65), loc STANDARD, char 'A', RFB keycode 0x1e RShift => 0x0041
CConn: key release, code A (65), loc STANDARD, char 'A', RFB keycode 0x1e RShift => 0x0041
CConn: key release, code Shift (16), loc RIGHT, RFB keycode 0x36 RShift => 0xffe2
CConn: key PRESS, code Unknown keyCode: 0x0 (0), loc UNKNOWN, RFB keycode 0x3a => 0x100ffff
CConn: Server's LED state: 0x6
CConn: key release, code Unknown keyCode: 0x0 (0), loc UNKNOWN, RFB keycode 0x3a => 0x100ffff
CConn: key PRESS, code A (65), loc UNKNOWN, char 1092, RFB keycode 0x1e => 0x06c6
CConn: key release, code A (65), loc UNKNOWN, char 1092, RFB keycode 0x1e => 0x06c6
CConn: key PRESS, code Shift (16), loc RIGHT, RFB keycode 0x36 => 0xffe2
CConn: key PRESS, code A (65), loc UNKNOWN, char 1060, RFB keycode 0x1e RShift => 0x06e6
CConn: key release, code A (65), loc UNKNOWN, char 1060, RFB keycode 0x1e RShift => 0x06e6
CConn: key release, code Shift (16), loc RIGHT, RFB keycode 0x36 RShift => 0xffe2
CConn: key PRESS, code Unknown keyCode: 0x0 (0), loc UNKNOWN, RFB keycode 0x3a => 0x100ffff
CConn: Server's LED state: 0x2
CConn: key release, code Unknown keyCode: 0x0 (0), loc UNKNOWN, RFB keycode 0x3a => 0x100ffff
CConn: key PRESS, code Compose (65312), loc STANDARD, RFB keycode 0xdd => 0x100ffff
CConn: key release, code Compose (65312), loc STANDARD, RFB keycode 0xdd => 0x100ffff
CConn: key PRESS, code O (79), loc STANDARD, char 'o', RFB keycode 0x18 => 0x006f
CConn: key release, code O (79), loc STANDARD, char 'o', RFB keycode 0x18 => 0x006f
CConn: key PRESS, code C (67), loc STANDARD, char 'c', RFB keycode 0x2e => 0x0063
CConn: key release, code C (67), loc STANDARD, char 'c', RFB keycode 0x2e => 0x0063

[any1]

This might be related to #271

Edit: Also, this is an exemplary bug report. Thank you!

[vthriller]

#271

Well, that might've explained the last observation, but my TurboVNC is already v3.1.1, which, as GitHub indicates, already contains TurboVNC/turbovnc@b4fd6ae.