Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions context/knowledge/gotchas/daemon-rpc.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@
- **A re-attached LIVE worker stranded in InReview is RESTORED to InProgress on reattach — `reattachSupervised` revives the live set, not just orphans (BUG-B).** `ReconcileStaleSessionsExcept` only handles the orphan direction (InProgress→InReview for tasks NOT alive); a live worker the supervisor confirms alive that is already parked in InReview (from a prior BUG-050 roll or an earlier reconcile) would otherwise stay mislabeled forever — across repeated bounces EVERY live worker drifts into InReview (0 in_progress on a busy daemon). So after the re-attach `Get` loop, `reattachSupervised` calls `db.ReviveHeraWorkerToInProgress(id)` for every task in `liveSet`. The same helper backs the TUI's in-place revive (`reviveHeraWorker` success branch → `App.reviveRestoreInProgress`, local `*db.DB` only; `--remote` defers to the local daemon's reattach). **`ReviveHeraWorkerToInProgress` is the exact inverse of `RollHeraWorkerToReview` and MUST refuse to un-roll a genuinely-finished worker** — it no-ops unless the task is worker-bound AND currently InReview AND NOT awaiting close-out, where "awaiting close-out" = `meta:hera.ready_to_close=true` (the done/clean-exit stamp) OR a terminal role-status (`done`/`failed`). That guard is what keeps #707 / BUG-050 intact: a done/failed worker with a still-idle-alive session stays InReview for coordinator close-out; only a non-terminal live worker flips back. DB status only, never touches the session, idempotent.
- **`hera_revive` (add-hera-revive) is a THIRD caller of `ReviveHeraWorkerToInProgress`, giving a coordinator (not just a human at the TUI) a way to PULL-revive a bound role.** The gating sequence — dead session (any role kind) restarts unconditionally; a LIVE coordinator is never auto-restarted; a live worker/freelance session only gets kicked when idle AND not blocked on a prompt AND no restart already pending — lives ONCE in `internal/hera.ReviveRole` (mirrors `RecycleCoord`'s architecture: pure function over narrow `ReviveStore`/`ReviveRunner` interfaces), wired daemon-side by `daemon.HeraReviveRunner`. **Deliberately NOT unified with the TUI's `Enter`-key revive** (`internal/tui/heraactions.go`'s `heraReattach`/`reviveHeraWorker`) — see `openspec/changes/archive/*-add-hera-revive/design.md` D3: the TUI's kick additionally resizes to the CURRENT PANE's dimensions (no such surface exists for a headless MCP caller, which instead preserves the session's existing PTY size) and is threaded through tview's `QueueUpdateDraw` model. Every individual check the TUI's inline version performs stays single-sourced regardless (`agent.BlockedOnPrompt`, `ReviveHeraWorkerToInProgress` itself, `SessionRunner.KickRerender`/`StartOrReattach`) — only the ~10-line ORDERING of those checks is expressed twice, a known and accepted residual overlap, not silent duplication.

- **KNOWN, UNFIXED: `Daemon.SessionStatus` can transiently report a session the supervisor is STILL RUNNING as `Alive=false` right around a daemon bounce, causing a client-side false "confirmed exit" that fires `RollHeraWorkerToReview` on a task that never actually died (found during narrow-needs-input-sustained-active's ground-truth investigation, deliberately NOT fixed there — flagged as its own follow-up).** Live repro (task `1785216680765732000`): `ux.log` showed `stream: ended ... err=EOF` → `stream: SessionStatus ... alive=false pid=16443` → `client.removeSession` (the CONFIRMED-exit path, not `removeSessionStreamLost`) at `2026-08-02T10:51:08.933`, and `task_meta` shows `ready_to_close` stamped at `.948` — 15ms later, via `App.handleSessionExitUI`'s `RollHeraWorkerToReview` call (`internal/tui/app.go`, "hera worker close-out" log line, NOT the daemon's own `transitionTaskOnExit`). But `daemon.log` proves pid 16443 never actually died: no `rpc.StartSession` appears again until the NEXT bounce, and `reattach: re-attached live supervisor sessions reattached=10 live=10` logs 3 seconds later — the supervisor held the session alive the entire time. `internal/daemon/client/stream.go`'s `isSessionAlive()` calls `Daemon.SessionStatus`, whose handler (`sessionCore.SessionStatus`, `internal/daemon/sessioncore.go`) does `sess := c.runner.Get(req.TaskID); if sess == nil { ...Alive stays false unless HasPendingRestart... }` — in supervisor mode `c.runner` is the supervisor-CLIENT, and exactly HOW `Get` can transiently miss a session the supervisor's OWN `Running()`/reattach pass (above) reports live moments later is not yet root-caused (candidates: a per-task lazy-attach gap in the supervisor-client's local session map vs. `reattachSupervised`'s bulk `Get` loop; a race between the OLD daemon process's own teardown — `cleanup()` closes `d.supClient` AFTER `Shutdown()` already closed `ln`, per the code above — and a client-side reconnect dial landing on a narrow window). Distinct from the `Running()==nil` guard above (that guards the BULK reattach's own live-set query; this is a PER-TASK `SessionStatus` RPC racing independently) and from the #707 cache-vs-EOF race (that's a same-process exit-info-cache-write race, not a cross-process daemon-bounce timing race). Would strand ANY hera-worker-bound task in `in_review`/`ready_to_close` on an unlucky daemon restart, single-hat or dual-hat — a dual-bound sub-coordinator (see `gotchas/hera-view.md`'s narrow-needs-input-sustained-active entry) just makes the resulting symptom more visible (task `in_review` while a coordinator hat visibly keeps working), since `ReviveHeraWorkerToInProgress`'s `heraWorkerAwaitingCloseout` guard then correctly (per its OWN invariant) refuses to un-roll a task once `ready_to_close` is stamped, treating the false exit identically to a genuine one. NOT reproduced with a minimal unit test yet — needs its own dedicated investigation before a fix is attempted.

## Session-supervisor (P4 — default ON + rollback)

- **`cfg.Supervisor.Enabled` now defaults to TRUE (absent key ⇒ supervisor mode), mirroring `hera.enabled` exactly.** The base value moved to `true` in `config.DefaultConfig` (NOT in `db/config.go`, whose parse logic is unchanged — absent key inherits the now-true default, explicit `"true"`/`"false"` still win). The OFF (in-process) path is the same code as P2/P3 and is **retained one release as the supported rollback**, NOT deleted: an explicit `supervisor.enabled = false` in the DB or `config.toml` restores the daemon's in-process runner (byte-identical to pre-P2). config.toml still wins over the DB, so a power user's `[supervisor] enabled = false` is a valid rollback even with the DB unset (`TestDB_Config_SupervisorTOMLRollback`).
Expand Down
8 changes: 8 additions & 0 deletions context/knowledge/gotchas/events.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,11 @@ Both callers compute the per-session streak the same way the BUG-029 escalation
**Both callers thread a new per-tick counter map exactly like the other maps in this file** (`only store if != 0`, rebuilt fresh from the currently-running set): `internal/tui/app.go`'s `detectNeedsInputSticky` (`App.needsInputSettle`, computed only for sessions in `idleIDs` — a busy session never even gets a tail read for this pass) and `internal/api/push.go`'s `computeNeedsInput` (`idleWatcherState.needsInputSettle`; `computeNeedsInput`'s signature grew a `prevSettle map[string]int` parameter and a 6th `newSettle` return value — every existing call site, including tests, needed the extra threaded argument). One PRE-EXISTING daemon test (`TestDetectNeedsInputTick`'s BUG-061 subtest) had its fixture corrected: it modeled the flooding scenario as `idle=["a","b"]` with the tail switched to plain content for TWO consecutive ticks — which is now, correctly, the exact scenario BUG-072 resolves, not the true BUG-061 case (a session that NEVER goes idle). Fixed by dropping "a" from the `idle` list for those ticks (genuinely busy/flooding, `settledOf` never engages) while keeping the original "must not clear from tail-decay alone" assertion intact.

See `TestSettleTick` (`internal/agent/needsinput_test.go`) for the step-function pin (threshold-exact settlement, no-grace immediate reset on either a not-idle or still-signal-present tick, still-blocked-idle never accumulates), the `NeedsInputClear` "settled activity" subtests for the pure-function clear-path pin (including its own BUG-063 interaction), and `TestComputeNeedsInput_SettledActivityClears` / `TestDetectNeedsInputSticky_SettledActivityClears` (with their `..._StillBlockedIdleDoesNotSettle` and `TestComputeNeedsInput_NotIdleNeverSettles` regression-guard siblings) for the end-to-end repro through the real callers.

## `ResumeActivityTick`'s zero-grace design never lets a genuinely-but-burstily-active session cross its own threshold — a NEW sibling step function was needed for the Hera rail's display gate (narrow-needs-input-sustained-active)

Ground-truth repro (hera role `contrib-classifier`, demonstrably active 30k+ tokens over 7+ minutes, still showing `(?)`): `ux.log` showed this exact task's rerender/revive logic alternating "busy" / "blocked on user prompt" within seconds, repeatedly — its actual PTY content periodically LOOKS like a parked selection prompt to the content classifier even while genuinely mid-flow. `TestResumeActivityTick`'s own "mostly-working output with occasional single-tick misses never converges either" subtest PROVES why: `ResumeActivityTick`'s deliberate zero-grace-period reset (a single non-working tick resets the streak to zero outright, by design — see BUG-065 above) means a session whose classification misses more often than once every `NeedsInputResumeTicks` (5) ticks can run for HOURS of genuine, substantial activity and never once cross the threshold — indistinguishable, by this counter alone, from a session that is still genuinely parked.

**Fix: a NEW, separately-named pure step function, `agent.SustainedActivityTick(prevTicks int, workingNow bool) (newTicks int, sustained bool)`, used ONLY by the Hera rail's `RoleView.SustainedActive` display signal — `ResumeActivityTick` itself is UNCHANGED and still exclusively backs `NeedsInputClear`'s `resumedOf` (BUG-065) and `autoClearBlockedHeraRoles` (BUG-066).** `SustainedActivityTick` mirrors `EscalateParkedSelection`'s BUG-060 one-tick-grace encoding exactly (a negative-sentinel `prevTicks` holds a streak through a single isolated miss rather than discarding it; a SECOND consecutive miss is a genuine break and resets for real) but reuses `NeedsInputResumeTicks` as its threshold — no new dial. This is safe for its own narrower purpose (suppressing a DISPLAY glyph, not clearing a stored flag): under-suppressing (staying flagged a tick or two longer) is the accepted failure mode everywhere in this file, and `TestSustainedActivityTick`'s own "sparse still-genuinely-blocked pattern" subtest confirms the one-tick grace is NOT generous enough to let a genuinely-still-parked session (the same sparse pattern `ResumeActivityTick`'s anti-false-clear test uses) falsely converge.

Threaded as a genuinely NEW per-tick counter (`App.sustainedActiveTicks`, independent of `App.needsInputResume` — never sharing state with BUG-065's stricter counter) feeding `App.sustainedActiveIDs` → `HeraPage.SetSustainedActive` → `BuildModel`'s new 5th parameter → `buildRoleView` → `RoleView.SustainedActive`, computed PER ARGUS TASK (not per hera role) so two roles sharing one live binding's task ID (a dual-bound sub-coordinator's parent-orchestrator worker hat and its own child-orchestrator coordinator hat, per `MaterializeHeraSubCoordinator`) automatically read the identical value. `RoleView.needsInputOwn()` checks `SustainedActive` FIRST and returns `false` unconditionally when true — suppressing BOTH the content-scan `NeedsInput` flag AND a self-reported `blocked` hera status, regardless of which hat carries the stale value. See `gotchas/hera-view.md`'s own entry for the rail-side wiring and `TestSustainedActivityTick` (`internal/agent/needsinput_test.go`) + `TestBuildModel_SustainedActive*` (`internal/tui/hera/model_sustainedactive_test.go`) for the pins.
Loading
Loading