feat(claude-code-error-hermit): production-error watcher for Sentry/GlitchTip (PROP-013) - #504
Draft
gtapps wants to merge 6 commits into
Draft
feat(claude-code-error-hermit): production-error watcher for Sentry/GlitchTip (PROP-013)#504gtapps wants to merge 6 commits into
gtapps wants to merge 6 commits into
Conversation
…13 Phase 1) New standalone domain plugin: a production-error watcher for Sentry/GlitchTip. Phase 1 of 4 — the scaffold, setup wizard, and zero-dependency API client. Each phase ships as an independently mergeable PR. - scripts/error-api-lib.ts + error-api.ts: Bun stdlib fetch client over the Sentry /api/0/ shape (GlitchTip-compatible). Subcommands check, issues, issue, latest-event, resolve/mute. Token never printed (redact()). - Write gating in two layers: in-CLI --confirm refusal (authoritative, sends no request without it) + write-confirm-gate.ts PreToolUse hook (fail-open). - skills/hatch: verifies credentials with a live check, injects the Error Watch block, extends the knowledge schema, stamps config.json. - Offline test suite: lib parsers, a Bun.serve fixture server driving the CLI, the hook matrix, skill-structure checks. No live backend needed. - Root: 7th marketplace entry, test-error.yml CI (path-filtered), CLAUDE.md plugin count 6 to 7. Watch loop (Phase 2) is wired via the core routines mechanism, not the non-existent "heartbeat precheck convention" the proposal assumed. Plan: ~/.claude/plans/propose-me-a-plan-immutable-swing.md Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
…(Phase 2) - scripts/error-precheck.ts: zero-token gate. Reads state/error-cursor.json (never writes it), queries issues since the cursor, prints one verdict: SKIP | EVALUATE|<n> new groups | EVALUATE|no cursor — bootstrap | ERROR. - skills/error-triage: precheck-gated. Classifies new vs regression vs known-noise against compiled/error-noise-ledger.md, correlates with recent releases, severity-gates to DM-now vs the digest queue, advances the cursor. resolve/mute are only ever proposed, never executed. - state-templates/compiled/error-noise-ledger.md: the classification memory. - hatch: step 5 drops the ledger; step 8 registers the error-triage routine (hourly, run_during_waiting, no model — the precheck is the cost gate; no prompt_file — core doesn't consume it). - tests/precheck.test.ts: SKIP/EVALUATE/bootstrap/ERROR verdicts + cursor immutability, driven by a Bun.serve fixture + a temp cursor dir. Fix: projectRoot() used readFileSync to test path existence, which throws EISDIR on the .claude-code-hermit dir — switched to existsSync so a set CLAUDE_PROJECT_DIR resolves correctly. Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
…e 3) - skills/error-reproduce: pull the event, map release→SHA, check out a throwaway worktree at the offending SHA, write a failing test from the stack, and git-bisect to the introducing commit when unclear. Read-only w.r.t. shared history; guards that it runs in the watched app's repo. - skills/error-draft-fix: branch error-fix/<shortId>, commit the failing test first then the fix, run the suite. Approval-gated hand-off — delegates to /claude-code-dev-hermit:dev-pr when installed, else stops at a local branch and DMs. Never git push from agent context. Tracker resolve stays --confirm-gated and post-merge only. Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
- skills/error-incident-summary: post-incident writeup to compiled/incident-<date>-<slug>.md; updates the ledger row to fixed-in <release> so a recurrence classifies as a regression. - skills/error-digest: drains state/error-digest-queue.json into one channel-friendly summary (new/regression/noise counts, draft branches awaiting a PR, resolve/mute approvals queued as micro-proposals). Ships as its own skill — core has no morning-brief extension point (HA precedent). - hatch step 8c: offers the optional error-digest routine via AskUserQuestion. Consolidate docs to 0.0.1 as the complete initial release (all four phases in one version, nothing shipped yet): CHANGELOG, README, plugin CLAUDE.md, and knowledge-schema (incident-summary now live). Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
Parallel-reviewer cleanup, all behavior-preserving: - error-api-lib: extract buildIssuesUrl — the CLI and precheck built the same issues-search URL; centralize in the lib that exists to be shared. - error-api: extract errHttp helper for the HTTP-error string repeated 4×. - test-utils: centralize loadFixture + jsonResponse used by both test files. - error-api.test: token-leak read commands now run via Promise.all. - loadFixture typed <T = any> so call-site indexing stays tsc-clean. tsc clean; 42 structure + 35 unit tests green. Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
…loop can reach SKIP The precheck queried firstSeen:>=cursor (inclusive), so the boundary group re-matched every run and EVALUATE fired forever — the "zero-cost when quiet" gate never went quiet. Track processed issue ids in the cursor and count only genuinely-new ones. A 6h lookback widens the window to catch backdated firstSeen late-arrivals, with a raised page size to keep reappearing overlap groups from crowding a real new group off the single page the gate fetches. Claude-Session: https://claude.ai/code/session_014ofkRhSQNaZpnESM2uGZUZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New standalone domain plugin
claude-code-error-hermit— a production-error watcher for Sentry/GlitchTip. It watches a project's error stream, classifies new groups against a known-noise ledger, correlates regressions with releases, reproduces failures locally, bisects them, and drafts a fix on a branch with the failing case as a test. Implements PROP-013 across its four phases in one plugin.Scope is deliberately bounded: reacting to a single error event is conceded to GitHub Actions; the plugin's value is the stateful work no cloud pipeline holds at once — the noise ledger, cursor-based watch, local repro/bisect, and draft-fix-with-failing-test.
Changes
scripts/error-api-lib.ts+error-api.ts) — zero-dependency Bunfetchover the Sentry/api/0/shape (GlitchTip-compatible).check/issues/issue/latest-event/ approval-gatedresolve/mute. Token never printed (redact()).--confirmrefusal (authoritative, sends no request without it) +write-confirm-gate.tsPreToolUse hook (fail-open).error-precheck.tszero-token gate (reads the cursor, never writes it) +error-triageskill (new vs regression vs known-noise, release correlation, DM-vs-digest severity gate). Hourlyerror-triageroutine.error-reproduce(throwaway worktree at the offending SHA, failing test from the stack,git bisect) anderror-draft-fix(fix on anerror-fix/<shortId>branch; push/PR delegated to/claude-code-dev-hermit:dev-pr, never improvised).error-incident-summary(writeup linked from the ledger) +error-digest(overnight summary; approvals as micro-proposals).check, drops the noise-ledger template, injects the Error Watch block, registers routines.marketplace.jsonentry, path-filteredtest-error.ymlCI, root CLAUDE.md plugin count 6 → 7.Built in four commits (one per phase) plus a
/simplifycleanup pass (sharedbuildIssuesUrl,errHttphelper, centralized test helpers).Test plan
bunx tscclean.bash plugins/claude-code-error-hermit/tests/run-all.sh). Fully offline — CLI/precheck driven against aBun.servefixture server; no live backend.bun test: 1333 tests, 0 fail.plugin-validatoragent: PASS (9/9, incl. nativeclaude plugin validate .).test-error.ymlruns the plugin suite on this PR.Live hatch verification (whoami + issues probe against a real Sentry/GlitchTip) is deferred until a backend is provisioned; the README flags GlitchTip's
/api/0/subset andcheckprobes the issues query path to surface any gap at setup.