Summary
Every state/runtime.json writer does an unlocked read-modify-write of the whole object. Two writers overlapping can silently drop a field written by the other. The field that matters most here is last_context_reset_at, because losing it re-opens the stale-cost-entry bug the hygiene guards exist to prevent.
Writers
None of these coordinate with each other:
hermit-watchdog.ts — long-running daemon, writes on its own tickcost-tracker.ts (writeRuntimeFields) — Stop hook, roughly once per assistant turn, the hottest writerstop-pipeline.ts → applyContextReset — Stop hookprecompact-stamp.ts → stampContextReset — PreCompact hook
(hermit-start.ts / hermit-stop.ts also write, but hold LIFECYCLE_LOCK, so they are out of this race.)
Failure mode
- Writer A reads
runtime.json.
- Writer B reads, sets
last_context_reset_at, writes.
- Writer A applies its own fields to the pre-B snapshot and writes.
B's stamp is gone. The hygiene tiers then cannot tell that a cost entry observed before the reset describes a context that no longer exists, so a /compact or /clear can fire against a freshly reset context — the exact behavior poisonedEntrySkip was added to stop.
Likelihood
Low. The realistic sequences are ordered rather than concurrent: PreCompact fires, then the turn ends and the Stop hook reads post-stamp. Hitting the lossy interleave requires cost-tracker's read to land before the stamp and its write after, inside a single hook invocation.
Not the same as the temp-file bug
The related hazard — all writers sharing one .runtime.json.tmp, letting one publish a zero-length runtime.json mid-write — is fixed separately by giving each process its own temp file (runtimeTmpPath() in scripts/lib/runtime.ts). That makes the rename atomic so readers never see a torn file. It does nothing for the lost-update race described here, which needs mutual exclusion.
Options
- Take
LIFECYCLE_LOCK (or a lighter state/.runtime.lock) around read-modify-write. Adds contention on a per-turn hook path, so the lock needs a short timeout and a fail-open path.
- Narrow the writes: have each writer own a disjoint set of fields and merge at write time by re-reading immediately before the write, shrinking (but not closing) the window.
- Move reset timestamps out of
runtime.json into a single-writer sidecar file.
Worth deciding deliberately rather than bolting a lock onto one call site.
Summary
Every
state/runtime.jsonwriter does an unlocked read-modify-write of the whole object. Two writers overlapping can silently drop a field written by the other. The field that matters most here islast_context_reset_at, because losing it re-opens the stale-cost-entry bug the hygiene guards exist to prevent.Writers
None of these coordinate with each other:
hermit-watchdog.ts— long-running daemon, writes on its own tickcost-tracker.ts(writeRuntimeFields) — Stop hook, roughly once per assistant turn, the hottest writerstop-pipeline.ts→applyContextReset— Stop hookprecompact-stamp.ts→stampContextReset— PreCompact hook(
hermit-start.ts/hermit-stop.tsalso write, but holdLIFECYCLE_LOCK, so they are out of this race.)Failure mode
runtime.json.last_context_reset_at, writes.B's stamp is gone. The hygiene tiers then cannot tell that a cost entry observed before the reset describes a context that no longer exists, so a
/compactor/clearcan fire against a freshly reset context — the exact behaviorpoisonedEntrySkipwas added to stop.Likelihood
Low. The realistic sequences are ordered rather than concurrent: PreCompact fires, then the turn ends and the Stop hook reads post-stamp. Hitting the lossy interleave requires cost-tracker's read to land before the stamp and its write after, inside a single hook invocation.
Not the same as the temp-file bug
The related hazard — all writers sharing one
.runtime.json.tmp, letting one publish a zero-lengthruntime.jsonmid-write — is fixed separately by giving each process its own temp file (runtimeTmpPath()inscripts/lib/runtime.ts). That makes the rename atomic so readers never see a torn file. It does nothing for the lost-update race described here, which needs mutual exclusion.Options
LIFECYCLE_LOCK(or a lighterstate/.runtime.lock) around read-modify-write. Adds contention on a per-turn hook path, so the lock needs a short timeout and a fail-open path.runtime.jsoninto a single-writer sidecar file.Worth deciding deliberately rather than bolting a lock onto one call site.