Skip to content

fix(supervision): fingerprint lock identity from /proc start_ticks, not ps lstart#621

Open
noahjobse wants to merge 1 commit into
kunchenguid:mainfrom
noahjobse:contrib/watcher-lock-drift-fix
Open

fix(supervision): fingerprint lock identity from /proc start_ticks, not ps lstart#621
noahjobse wants to merge 1 commit into
kunchenguid:mainfrom
noahjobse:contrib/watcher-lock-drift-fix

Conversation

@noahjobse

Copy link
Copy Markdown

Problem

The supervision lock-identity fingerprint (fm_pid_identity in bin/fm-wake-lib.sh) is derived from ps -p <pid> -o lstart=. lstart is not a stable primitive: the kernel derives it from btime (boot time) plus the process's starttime ticks. starttime never moves, but btime is recomputed and jitters (observed ~1s, roughly every 30s, on a WSL2 host).

Every btime jump shifts the derived lstart of every live process. So a live lock holder's fingerprint drifts against its own process with no second writer. The health predicate fm_watcher_lock_matches_pid then misreads a healthy, beating, lock-holding watcher as "not ours" and raises false "supervision off" alarms. A forced re-arm off that misread could then act on the wrong process.

This is a self-inflicted flap: nothing external changes, yet supervision reports itself down.

Fix

Fingerprint from /proc/<pid>/stat field 22 (start ticks) instead. That value is boot-relative, carries no btime term, and is constant for the process lifetime, so it cannot drift.

  • New fm_pid_start_ticks reads field 22, counting fields after the last ) so a comm containing spaces or ) can't shift the count.
  • fm_pid_identity now emits <start_ticks> <command>. The old lstart read is kept only as a fallback where /proc does not exist (macOS). LC_ALL=C is still pinned so the command half stays locale-invariant.

Compatibility

The identity string is only written and equality-compared by every consumer (fm-watch, fm-watch-arm, fm-afk-launch, fm-afk-start, fm-supervise-daemon) — none parse its fields — so this is drop-in. The one transition effect: a watcher/daemon already running at upgrade whose recorded identity is an lstart string re-reads as not-matching once, is treated as stale, and re-records its ticks identity. A single, self-healing event.

Verification

Two focused cases added to tests/fm-watcher-lock.test.sh:

  1. byte-stability — the same live pid fingerprints byte-identically across 20 repeated reads.
  2. start-ticks, not lstart — asserts the fingerprint's start half is /proc/<pid>/stat field 22, and that it stays fixed across a simulated btime shift under which the equivalent lstart derivation (btime + starttime/HZ) provably would move. This pins the cure so a future refactor can't quietly regress it back to lstart. Skips cleanly where /proc is absent.

The existing test_pid_identity_is_locale_invariant still passes. Full tests/fm-watcher-lock.test.sh suite is green (24 cases, exit 0) on Linux.

Verified on the original WSL2 host where the drift was observed: the new fingerprint held byte-identical across a real btime jitter that shifted lstart.

Scope

Deliberately narrow: the root-cause primitive plus its regression tests. This is extracted from a larger torn-watcher-lock hardening set on the contributing fork; this PR is the focused, reviewable core of it.

…ot ps lstart

The supervision lock-identity fingerprint (fm_pid_identity) was derived from
`ps -p <pid> -o lstart=`. lstart is not a stable primitive: it is derived from
the kernel's btime plus the process's starttime ticks. starttime never moves,
but btime is recomputed and jitters (observed ~1s, roughly every 30s, on a WSL2
host). Every btime jump shifts the derived lstart of every live process, so a
live lock holder's fingerprint drifts against its own process with no second
writer. The health predicate (fm_watcher_lock_matches_pid) then misreads a
healthy, beating, lock-holding watcher as not-ours and raises false
"supervision off" alarms, and a forced re-arm could act on the wrong process.

Fingerprint instead from /proc/<pid>/stat field 22 (start ticks), which is
boot-relative, carries no btime term, and is constant for the process lifetime.
A new fm_pid_start_ticks reads it (parsing fields after the last ')' so a comm
with spaces or ')' cannot shift the count). fm_pid_identity uses it, with the
old lstart read kept only as a fallback where /proc does not exist (macOS);
LC_ALL=C is still pinned so the command half stays locale-invariant.

The identity string is only written and equality-compared by every consumer
(fm-watch, fm-watch-arm, fm-afk-launch, fm-afk-start, fm-supervise-daemon), so
this is drop-in. The one transition effect: a watcher/daemon already running at
upgrade whose recorded identity is an lstart string re-reads as not-matching
once, is treated as stale, and re-records its ticks identity - a single,
self-healing event.

Tests: two focused cases added to tests/fm-watcher-lock.test.sh - one asserting
byte-stability across repeated reads of one live pid, and one asserting the
fingerprint is /proc field 22 and stays fixed across a simulated btime shift
that the equivalent lstart derivation would move under, so the cure cannot be
quietly regressed back to lstart. The existing locale-invariance test still
passes.

Claude-Session: https://claude.ai/code/session_01PKVpT6RbwBg74REPzHMG4n
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant