Skip to content

Netlink uevent hotplug monitor (mechanism + headless diagnostic) - #8

Merged
mariotaku merged 3 commits into
webOS-2.30.xfrom
feature/netlink-uevent-monitor
Aug 17, 2026
Merged

Netlink uevent hotplug monitor (mechanism + headless diagnostic)#8
mariotaku merged 3 commits into
webOS-2.30.xfrom
feature/netlink-uevent-monitor

Conversation

@mariotaku

@mariotaku mariotaku commented Aug 16, 2026

Copy link
Copy Markdown
Member

Replaces the 3-second joystick and hidraw presence polls on webOS with kernel uevents. Both backends are wired up.

Background: https://gist.github.com/mariotaku/131ed7ba5a9bbdfc3a1dbb1261feb174 — context is moonlight-tv#614, which carries the reconnect workaround as a build-time patch. This should let that PR drop the patch file, the apply step, and the revision pin.

Problem

No libudev in the app jail, so enumeration_method falls back to ENUMERATION_POLLING. Every 3s LINUX_FallbackJoystickDetect rescans /dev/input and diffs a 32-bit presence bitmask. A controller that disconnects and reconnects on the same index between two polls leaves the bitmask unchanged, so SDL never notices.

Two things beyond the gist:

  • The webOS poll branch only ever called MaybeAddDevice for present indices — never MaybeRemoveDevice. Plain disconnects weren't reported by this path either.
  • There was a second, independent 3s poller in src/hidapi/SDL_hidapi.c doing its own hidraw bitmask diff. A DualShock 4 goes through HIDAPI, so fixing only the joystick side would have left half the latency in place. Both are gone now.

Using it

The monitor is self-contained — no harder than the poll it replaces:

monitor = SDL_webOSUeventMonitorOpen(SDL_WEBOS_DEVICE_PRESENCE_CHECK_EVDEV);

while (SDL_webOSUeventMonitorPoll(monitor, &event)) {
    if (event.action == SDL_WEBOS_UEVENT_ACTION_ADD) {
        MaybeAddDevice(event.devnode);
    } else {
        MaybeRemoveDevice(event.devnode);
    }
}

Open() binds the socket and only then enumerates what's already attached, queueing those as ordinary add events — so callers have one code path for existing and hotplugged devices, and the bind-before-scan ordering is handled internally rather than left to each integration to remember. Poll() filters to the requested node class, collapses events that wouldn't change what the caller believes, and returns a devnode ready for MaybeAddDevice().

Overflow recovery is internal for the same reason. Netlink is lossy: the kernel discards broadcasts under pressure and reports ENOBUFS once. An earlier revision exposed this as LostEvents(), which only worked if the caller remembered to ask — and forgetting left the device list permanently stale, silently, which is worse than the polling it replaces. The monitor now re-enumerates on ENOBUFS and reports the difference against what it last told the caller. This matters on webOS because an app can be backgrounded or suspended, and a process not draining while devices come and go is exactly how the buffer fills.

Other design points:

  • Each subsystem opens its own monitor. Netlink delivers a copy to every bound socket; sharing one fd would mean whichever side drained first ate the other's events.
  • DEVNAME preferred over DEVPATH, falling back to DEVPATH's trailing component since a remove on an older kernel can omit DEVNAME.
  • Sender validation. Any user process can bind netlink and unicast at us, so messages not from portid 0 / not on a multicast group are dropped — a malformed path can't be injected into MaybeAddDevice.
  • Node tracking is by name, so it has none of the bitmask's 32-index ceiling.

Polling stays for exactly two cases: a socket that won't open, and the re-enumeration behind overflow recovery.

before after
joystick scandir + stat sweep every 3 s non-blocking recv, usually EAGAIN
hidraw separate 3 s bitmask diff same monitor, own socket
latency up to 3 s sub-100 ms
same-index reconnect invisible explicit remove + add

Hardware results

Measured with testwebosuevent, running the monitor and the presence poll side by side over the same hotplug activity. webOS 4 and 3.4 ran inside a real native_devmode jail via jailer, as the app's unprivileged uid.

kernel webOS context poll changes netlink transitions missed by netlink poll lost mean lead
5.4.268 10 (OLED C5) devmode user 30 36 0 5 1722 ms
4.4.84 4 (LK5900) jail uid 6927 14 14 0 0 2701 ms
4.4.84 4 (LK5900) jail uid 5235 12 18 0 6 1813 ms
4.4.84 4, 2 controllers jail uid 5235 21 21 0 0 1631 ms
3.10.19 3.4 (UH6100) jail uid 6339 34 46 0 12 1912 ms

111 poll-detected changes across three kernel generations. Netlink missed none. The poll lost 23 transitions netlink caught.

  • Kernel 3.10 works. The gist listed unprivileged bind on 3.x as expected-but-unverified, and it was the only reason the fallback was load-bearing. Verified on a UH6100 (webOS 3.4, glibc 2.19): binds and delivers inside the jail.
  • The blind spot, captured live. On the C5: remove hidraw/0 at 62.575, add hidraw/0 at 62.756 — 181 ms apart, both between two polls, bitmask byte-identical across them.
  • Whole connect/disconnect cycles vanish. On 3.10, a controller appeared and disappeared inside 1.77 s between two scans: netlink saw 2 transition(s), the bitmask could express 0.
  • Burst load is fine. A Logitech Unifying receiver enumerating 5 tracked nodes within 21 ms, and two controllers connecting together, both with 0 dropped.
  • Lead time is a distribution, not a constant — 82 ms to 3.0 s. The poll's latency is a coin flip between "fine" and "three seconds" on every event.

End to end, through SDL's own API

testwebosuevent --sdl reports SDL_JOYDEVICEADDED/REMOVED instead, which checks the backend wiring rather than the stream underneath it. On the UH6100 (kernel 3.10), inside the jail:

[  0.540] SDL joystick subsystem up, 2 joystick(s) already present
[  1.390] SDL_JOYDEVICEADDED    device index 0 (PS4 Controller)
[  1.390] SDL_JOYDEVICEADDED    device index 1 (Google Stadia Controller)
[ 48.791] SDL_JOYDEVICEREMOVED  instance id 2
[ 48.826] SDL_JOYDEVICEREMOVED  instance id 1     <- 35 ms later
[ 51.247] SDL_JOYDEVICEADDED    device index 0 (PS4 Controller)
[ 51.949] SDL_JOYDEVICEADDED    device index 1 (Google Stadia Controller)
[ 52.384] SDL_JOYDEVICEREMOVED  instance id 3     <- 435 ms later
[ 53.778] SDL_JOYDEVICEADDED    device index 1 (PS4 Controller)
[ 54.395] SDL_JOYDEVICEREMOVED  instance id 4     <- 617 ms later
[120.013] 2 joystick(s) present at exit

Five distinct transitions inside two 3-second windows, each surfaced as its own SDL event — the old code could emit at most one bitmask diff per window. The two controllers already attached at startup arrive as ordinary SDL_JOYDEVICEADDED at 1.390 s, which is the monitor's internal enumeration.

Over the run: 7 adds, 5 removes, 2 present at exit. The accounting balances exactly, so 12 transitions of aggressive hotplugging left no leaked or phantom entries in SDL's device list.

Tests

Both follow the testevdev pattern: internal sources are compiled in rather than linked (they're unexported), and SDL_internal.h comes first so the dynapi renaming applies to the public declarations as well as our own calls. On webOS builds the test targets also need __WEBOS__SDL_dynapi.h keys SDL_DYNAMIC_API off it, and the library gets it from sdl-build-options while tests don't, so without it the test references SDL_*_REAL while the library defines plain names and fails to link.

  • test/testwebosuevent_parse (non-interactive, ctest): nine synthetic uevents — evdev add, remove without DEVNAME, hidraw, change/bind rejection, libudev-format rejection, missing ACTION, trailing-slash path, index past 31.
  • test/testwebosuevent (needs hardware): headless, meant to run over SSH on a TV. Cross-checks netlink against the poll and reports NETLINK USABLE / FALLBACK REQUIRED / INCONCLUSIVE, the latter because an idle system produces no uevents whether or not the socket works. --sdl switches to reporting SDL's own device events.

Not gated on WEBOS — the mechanism is plain Linux netlink, so it runs on a desktop with a USB controller too.

On the fallback trigger

The gist suggests falling back to polling if no uevent arrives within a settle window after the initial scan. I'd argue against that specific rule: absence of events isn't evidence of breakage, since an idle TV legitimately produces none. The runs bear this out — netlink was silent for 45 s at a stretch while working perfectly. Open() failing plus ENOBUFS covers what's actually detectable.

Open question

On the C5, netlink reported hidraw/1 add and remove that the presence poll never saw in either direction, while hidraw/0 tracked correctly. Most likely that jail's /dev didn't contain the node. It didn't reproduce on the other two TVs. So netlink can report devices the jail can't open — MaybeAddDevice opens the path and fails gracefully, so this should be safe, but it's the one behaviour I'd want a second opinion on.

🤖 Generated with Claude Code

There's no libudev in the webOS app jail, so the joystick and hidapi
backends fall back to rescanning /dev every 3 seconds and diffing a
presence bitmask. That's slow to react, and it cannot see a controller
that disconnects and reconnects on the same index between two scans,
because the bitmask comes out unchanged.

Bind a NETLINK_KOBJECT_UEVENT socket instead. The kernel broadcasts
add/remove to group 1 with no privilege and no libudev, events are
explicit and ordered rather than inferred from a bitmask diff, and they
arrive in well under 100ms.

This adds the mechanism and a way to measure it. Neither backend is
wired up to it yet.

Each subsystem is expected to open its own monitor: netlink delivers a
copy to every bound socket, whereas sharing one fd would mean whichever
side drained it first consumed the other's events.

testwebosuevent runs the monitor and the presence poll side by side over
the same hotplug activity and reports whether netlink can replace the
poll. Unprivileged bind is verified on webOS 10 (kernel 5.4) but not on
the 3.10 kernels older versions ship, and that is the open question
standing between this and dropping the poll. It reports INCONCLUSIVE
rather than a pass when no device came or went, since an idle system
produces no uevents whether or not the socket is delivering.

That one needs hardware to conclude anything, so the message parsing is
covered separately by testwebosuevent_parse, which runs non-interactively
under ctest against synthetic uevents. Both follow testevdev: internal
sources are compiled in rather than linked, since they're unexported, and
SDL_internal.h comes first so the dynapi renaming applies to the public
declarations as well as to our calls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mariotaku
mariotaku force-pushed the feature/netlink-uevent-monitor branch from d1b318b to 469fb00 Compare August 16, 2026 07:55
Mariotaku and others added 2 commits August 16, 2026 18:37
…ting

Netlink is lossy under pressure: rather than blocking the sender, the
kernel discards broadcasts and reports ENOBUFS once. Poll() treated that
exactly like EAGAIN, so a caller could not tell "socket drained" from
"the kernel threw events away", and would carry on believing its device
list was in sync. Since a webOS app can be backgrounded or suspended --
which is precisely how the buffer fills while controllers come and go --
that would leave the list permanently stale, strictly worse than the
polling this replaces, which self-heals within one interval.

Report it instead, via SDL_webOSUeventMonitorLostEvents(), so a backend
can respond with the same full scan it does at init. Draining stops at
the overflow: a rescan supersedes whatever is still queued, and
returning avoids spinning if the condition repeats.

This also fixes how testwebosuevent counts what the poll misses. It only
flagged an index when the bitmask did not change at all, which catches a
reconnect (2 transitions, 0 expressible) but not a whole connect/
disconnect cycle landing between two scans (3 transitions, 1
expressible). It now compares transitions seen against transitions the
diff could express, per index, so both collapse the same way. On
hardware the corrected count matches the raw event difference exactly:
46 netlink transitions against 34 poll changes, 12 reported lost.

The seeding scan is quiet now as well. It ran through the reporting
path, so every already-attached device printed as an arrival netlink had
failed to report -- 21 alarming lines before the run even started.

Tests need __WEBOS__ on webOS builds. SDL_dynapi.h keys SDL_DYNAMIC_API
off it, and the library gets it from sdl-build-options while test
targets do not, so without it the test compiles against SDL_*_REAL while
the library defines the plain names and fails to link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both backends now take hotplug from the netlink stream instead of
rescanning /dev on a timer. LINUX_JoystickDetect drains a monitor rather
than diffing a presence bitmask, and the hidapi discovery loses its
separate 3-second hidraw scan.

The monitor is self-contained, so using it is no harder than the poll it
replaces: open one for a node class and drain it wherever you used to
poll. Open() binds the socket and only then enumerates what's already
attached, queueing those as ordinary add events -- callers get one code
path for existing and hotplugged devices, and the bind-before-scan
ordering is handled here rather than left to each integration to
remember. Poll() filters to the requested class, collapses events that
would not change what the caller believes, and hands back a devnode ready
for MaybeAddDevice().

Overflow recovery moved inside for the same reason. It was exposed as
LostEvents(), which only worked if the caller remembered to ask; forget
it and the device list goes permanently stale, silently. The monitor now
re-enumerates on ENOBUFS and reports the difference against what it last
told the caller, so the caller sees a correct add/remove stream either
way.

Polling stays for the cases that need it: a socket that will not open,
and the re-enumeration behind the overflow recovery.

Measured inside the app jail across three kernel generations -- webOS
3.4 (3.10.19), 4 (4.4.84) and 10 (5.4.268). Over 111 poll-detected
changes netlink missed none, while the poll lost 23 transitions netlink
caught, including whole connect/disconnect cycles that landed between
two scans and left the bitmask unchanged.

testwebosuevent grows a --sdl mode that reports SDL's own joystick
device events, which checks the backend wiring rather than the stream
underneath it. On a webOS 3.4 TV it shows two removes 35ms apart and
several add/remove pairs under a second, all resolved individually --
none of which the 3-second poll could express.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mariotaku
mariotaku marked this pull request as ready for review August 17, 2026 04:08
@mariotaku
mariotaku merged commit 263629d into webOS-2.30.x Aug 17, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant