Fix TUI lag: debounce Hera pane kick + gate tick's Hera rebuild on change-detection - #924
Merged
Merged
Conversation
…tection Fixes two distinct causes of TUI lag/memory pressure Aaron hit dogfooding (~59GB RSS after ~14-15h, daemon/supervisor both normal — client-side only): 1. Ordinary rail navigation between a coordinator row and a worker row alone swings a bound pane's width past agent.ShouldKickRerender's margin (no resize/fullscreen toggle needed), so a fast Cmd+Arrow traversal kicked (Session.Stop()+restart+full replay) every transiently-bound task in quick succession. maybeKickPaneRerender now debounces the kick behind a 300ms wall-clock dwell, arm-then-fire per pane, goroutine-free. 2. The 1s UI tick's Hera rail rebuild (BuildModel + Rail.buildRows()) ran unconditionally every tick regardless of activity, doing O(total historical role/orchestrator/binding count) work — archived history included — even while idle. doRefresh now gates the rebuild on cheap change-detection: a SQLite PRAGMA data_version fingerprint (catches daemon-driven changes) combined with equality checks on the four per-tick runtime maps (catches agent-activity-driven changes the DB alone wouldn't show). Measured at ~900-role scale: 34ms/30MB/133k allocs per rebuild before, ~850ns/400B/12 allocs per tick when idle after — a ~35,000x steady-state reduction. Also de-dupes a redundant second Tasks() fetch BuildModel was doing per tick. This reduces per-tick cost dramatically and is strong evidence toward the sustained RSS growth, but does not by itself prove it accounts for the full 59GB (allocation-rate reduction isn't the same as retained-memory proof) — flagged as a named follow-up requiring an actual dogfood RSS comparison. Both fixes are scoped and documented in openspec/changes/archive/2026-08-04-fix-hera-tick-and-kick-perf/ (proposal, design with decisions/risks/open-questions, delta specs, tasks). Explicitly out of scope, named follow-ups: widening the gate to the plain task list's own base reads, and lazy-loading archived rail data structurally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Merging this branch changes the coverage (1 decrease, 2 increase)
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two distinct, confirmed causes of severe Argus TUI lag/memory pressure (Aaron's dogfooded TUI observed at ~59GB RSS / 46% RAM / 36% CPU after ~14-15h uptime, while the daemon (280MB) and session-supervisor (24MB) stayed small and normal — a client-side TUI problem, not daemon/supervisor):
Kick storm on ordinary rail navigation. Hera's own layout swings a bound pane's width between full-terminal-width (fullscreen/Details mode) and roughly half-width (split mode) purely from moving the rail cursor between a coordinator row and a worker row — no resize or fullscreen toggle needed. This alone crosses
agent.ShouldKickRerender'sRerenderMargin(15 cols), so a fast Cmd+Arrow traversal across several rows binds and kicks (Session.Stop()+--session-idrestart + full conversation replay) every transiently-bound task in quick succession.Fix:
HeraPage.maybeKickPaneRerendernow debounces the kick behind a 300ms wall-clock dwell (hera.KickDebounce). The firstDrawthat finds a newly-bound task past the margin arms a pending kick instead of firing immediately; only a laterDraw, once the dwell elapses AND the same task is still bound, actually kicks. A rebind to a different task before the dwell elapses discards the old pending kick un-fired. No new goroutine — checked on the same Draw cadence, mirroringhera.Refresher's existing goroutine-free style. The anti-corruption kick mechanism itself (agent.ShouldKickRerender,RerenderMargin, idle/prompt/pending gates, exit-handler resume) is completely unchanged — only when it fires moved by ~300ms.Unconditional O(total-history) work on the 1s UI tick.
App.onTick→refreshTasksWithIDsfetches every task row every second; while the Hera tab is active it also drivesHeraPage.doRefresh→BuildModel(fetches ALL orchestrators including fully-archived history, all live+latest bindings, re-fetches the full task list a second time) →Rail.buildRows()(runscanonicalParents()/structuralReach()graph algorithms over the WHOLE model, archived included) — every single tick, regardless of whether anything changed. This scales with total historical role/orchestrator/binding count (~900+ for Aaron), not active-agent count, matching the reported "stalling while sitting still with only 5-6 active agents."Fix:
doRefreshnow gates theBuildModel+SetModelrebuild behind cheap change-detection (HeraPage.shouldRebuild/markRebuilt): a SQLitePRAGMA data_versionfingerprint (near-O(1), catches any daemon/other-process-driven DB write) combined withmaps.Equalchecks on the four per-tick runtime maps also fed into the model (needsInput/sessionIdle/sessionRunning/sustainedActive— bounded by active session count, so cheap even with 900+ historical roles). Either changing triggers a rebuild, so the rail's spinner/needs-input glyphs never freeze while agents are active even when the DB itself is quiet.PRAGMA data_versionhas a documented same-connection blind spot (a write through the SAME connection doesn't bump what that connection reads back); closed byHeraPage.Refresh()itself (not just the App'sheraRefreshwrapper — see design.md Decision 5 for why the narrower first attempt broke an existing test) invalidating the gate before flushing, so its own "forces an immediate rebuild" contract holds regardless of caller. Also de-dupes a redundant secondTasks()fetchBuildModelwas performing every tick via a small reader-wrapper (tasksReader), avoiding aBuildModelsignature change that would've touched ~42 unrelated test call sites.Measurement (honest, not assumed)
internal/tui/hera/doRefresh_bench_test.go(go test -bench, permanent regression-guard benchmarks, never run during plaingo test ./...) seeded ~900 roles/~900 bindings across 450 archived orchestrators + 1 active one, matching Aaron's reported scale:~35,000x reduction in per-tick allocation/CPU cost while idle — at 1 tick/sec this eliminates ~30MB/s of allocation churn, strong evidence of a major contributor to sustained GC/heap pressure.
What this does NOT prove: allocation-rate reduction is not the same as retained-memory proof — Go's GC reclaims transient garbage, so this benchmark alone cannot confirm what fraction (if any) of the reported 59GB RSS was genuinely retained-forever vs. GC-pressure-inflated heap targets vs. a separate, still-unidentified leak. Confirming that requires an actual longitudinal RSS comparison of the real TUI dogfooded for a comparable multi-hour period — not performed in this PR (infeasible within a single PR's time budget) — flagged as a named follow-up in
design.md's Open Questions.Explicitly out of scope (named follow-ups, not silently dropped)
refreshTasksWithIDs's own base reads (db.Tasks(), bothListMetaByNamespacecalls,ManagedTaskIDs()) — more un-audited local-write call sites than the Hera-mutation chokepoint this gate relies on.Rail.buildRows()'s graph algorithms so an actually-needed rebuild is also structurally cheaper, not just less frequent — deferred given the risk torail.go's heavily invariant-laden fold/nesting logic (long bug history documented ingotchas/hera-view.md).Full design rationale (7 numbered decisions, risks/trade-offs, migration plan, open questions) in
openspec/changes/archive/2026-08-04-fix-hera-tick-and-kick-perf/design.md, archived in this same PR per repo convention.Test plan
internal/tui/hera/panes_test.go): fast multi-row traversal never kicks, genuine dwell-and-stay still kicks once, unbind-mid-dwell-then-rebind-same-task re-arms correctly (caught a real bug via TDD — see design.md Decision 2).internal/tui/hera/changegate_test.go): first-call-always-true, quiescent-skips, DB-fingerprint-change triggers, each of the 4 runtime maps individually triggers, unsupported-fingerprint/remote-mode always rebuilds,InvalidateChangeGate,Refresh()forces rebuild despite the same-connection blind spot,doRefreshintegration, tasks-dedup via a call-counting reader wrapper.DB.DataVersiontest (internal/db/db_test.go): cross-connection change visible, same-connection blind spot confirmed, stable with no writes — using two real file-backed connections (t.TempDir(), never touches real~/.argus/).make pre-pr(build/vet/fmt-check/lint-pr) clean.vulnfails only on 3 pre-existing stdlib CVEs (CIcontinue-on-error, confirmed unrelated to this diff).test-cover-gategreen at 88.8% (floor 88%) once two pre-existing, documented environmental issues are excluded: this hera-worker sandbox's ownARGUS_*env vars leaking into 2internal/agentprofile-env tests (passes with them unset; CI has none), andTestSmoke_NewTaskFormPaste's documented pre-existing-raceflake (confirmed flaky even in isolation, unrelated to any file this PR touches). A full clean run with the sandbox env excluded passed fully green.internal/tui,internal/tui/hera,internal/dbsuites pass with no regressions.🤖 Generated with Claude Code