test: pin the fleet hook stdin contract with a cross-plugin corpus - #716
Merged
Conversation
Every fleet hook keeps its own sealed stdin copy (plugins can't import core at runtime), so the drain/cap/fail-direction invariants were re-decided per file with nothing pinning them. The corpus feeds all 11 gates the same adversarial payloads on real stdin and asserts each hook's declared fail direction, plus that the writer completed — a mid-stream exit and a clean drain both report 0 from the hook itself, so the drain is only observable via the writer's SIGPIPE. It immediately caught two live defects in dev-hermit, both fixed here: the three guards abandoned the pipe half-read past their 1MB cap (worktree-boundary-guard even claimed otherwise in a comment), and record-test-result crashed with exit 1 on a null payload because its property access sat outside the parse try/catch.
…raining stdin worktree-boundary-guard returned on WORKTREE_GUARD=off before reading stdin, so the documented escape hatch still reproduced the SIGPIPE the drain loop two lines below was added to prevent: any Write whose content exceeds the pipe buffer left the writer to die on a half-read pipe. The drain now runs first and the switch is honored at EOF, matching git-push-guard, which already drains before its own AGENT_HOOK_PROFILE check. When the guard is off the payload is consumed without buffering, since it is discarded immediately after. All three dev hooks called main() bare and, unlike core's lib/hook-input.ts, never attach a stdin error listener, so a stream error became an unhandled rejection and exit 1 rather than failing open. They now use the main().catch(() => process.exit(0)) form core's cache-edit-guard already uses. The stdin corpus discovered scripts only via a hook's args array, so a PreToolUse gate registered with an inline command string (a form core's hooks.json already uses) produced no entry and passed the coverage assertion vacuously. Discovery now scans both forms, and a new case pins the WORKTREE_GUARD=off drain.
…drain # Conflicts: # .github/workflows/test-cross-plugin.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every fleet hook keeps its own sealed copy of stdin handling — fleet plugins can't import core at runtime, so the four invariants core states in
scripts/lib/hook-input.ts(drain fully past the cap, size cap, fail direction, decision shape) were re-decided per file with nothing pinning them. Five distinct stdin idioms and four decision idioms ship across the fleet today.This adds a repo-root corpus test that feeds all 11 gate hooks the same adversarial payloads on real stdin and asserts each hook's declared fail direction. Writing it immediately surfaced two live defects in
claude-code-dev-hermit, both fixed here.Changes
The corpus (
tests/cross-plugin/hook-stdin-contract.test.ts)mcp-safety-gateis the fleet's only fail-closed gate (exit 2); the other ten fail open.[1,2]/null/"str"), a well-formed benign payload, and a generated 2MB unparseable payload (generated, not committed — HA'sgate-corpusprecedent).PIPESTATUS. This matters: a hook that exits mid-stream and one that drains cleanly both report 0 themselves, so the hook's own exit code cannot detect the bug — only the writer'sSIGPIPE(141) can.plugins/*/hooks/hooks.jsonand checks both directions: a stale/renamed spec fails, and a newly registered PreToolUse gate missing from the corpus fails too.Two defects it caught (
claude-code-dev-hermit)git-push-guard,worktree-boundary-guard,record-test-resultall calledprocess.exit(0)inside the read loop on oversize input, abandoning the pipe half-read — the exact failure core's header exists to prevent.worktree-boundary-guardeven carried a comment claiming it drained to completion. They now stop buffering but keep consuming to EOF.record-test-resultcrashed with exit 1 on anullpayload: its property access sat outside the parsetry/catch, unlike its two siblings. Claude Code treats exit 1 as non-blocking, so this was an uncaught crash masquerading as fail-open.Wiring
test-cross-plugin.ymlpath filter now includesplugins/*/hooks/**and the three dev guard scripts — none of them triggered this workflow before.scripts/test-all.shnow runs the cross-plugin suite, which it never did. Deliberately serial after the parallel phase: these spawn a subprocess per corpus case, and the parallel phase is already saturated enough that HA's CPU-boundgate-corpustests sit near their 5s per-test timeout.Deliberately not done
hookSpecificOutputemitters are duplicated on purpose and already byte-pinned bygate-corpus.test.tsagainst the retired Python gate. Re-asserting them here would duplicate a deliberate pin.drainStdin()helper. A review pass proposed extracting the three now-identical loops into dev-hermit'sscripts/lib/. Rejected: the natural helper hidesprocess.exit(0)inside something that reads as "return a string", hardcoding fail-open into the helper and removing each caller's declared fail direction — the exact property this PR makes explicit. Core's ownreadHookInputreturns anOVERSIZEsentinel precisely so the caller decides; doing it that way is a redesign of three hooks.readFileSync(0)andBun.stdin.text()already read to EOF, so they satisfy the drain invariant; the corpus recordscap: nonerather than forcing a change.Test plan
bun test tests/cross-plugin/→ 141 pass, 0 fail.bunx tsc --noEmit→ exit 0.bash plugins/claude-code-dev-hermit/tests/run-all.sh→ exit 0.process.exit(0)ingit-push-guard.tsfails exactly one test, naming the hook and the oversize case, on the drain assertion (expected 0, received 141) while the hook's own exit code stays 0 — confirming the check is not vacuous.Known unrelated failure
bash scripts/test-all.shexits 1 on HA's twogate-corpusoversize tests (~6.1–6.6s vs a 5s per-test timeout). This is pre-existing and not caused by this branch — verified by stashing every change on this branch and re-running the untouched runner, which fails identically. HA's suite passes on its own (32s). Left alone as out of scope.