Skip to content

test: pin the fleet hook stdin contract with a cross-plugin corpus - #716

Merged
gtapps merged 3 commits into
mainfrom
fix/fleet-hook-stdin-drain
Aug 14, 2026
Merged

test: pin the fleet hook stdin contract with a cross-plugin corpus#716
gtapps merged 3 commits into
mainfrom
fix/fleet-hook-stdin-drain

Conversation

@gtapps

@gtapps gtapps commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Summary

Every fleet hook keeps its own sealed copy of stdin handling — fleet plugins can't import core at runtime, so the four invariants core states in scripts/lib/hook-input.ts (drain fully past the cap, size cap, fail direction, decision shape) were re-decided per file with nothing pinning them. Five distinct stdin idioms and four decision idioms ship across the fleet today.

This adds a repo-root corpus test that feeds all 11 gate hooks the same adversarial payloads on real stdin and asserts each hook's declared fail direction. Writing it immediately surfaced two live defects in claude-code-dev-hermit, both fixed here.

Changes

The corpus (tests/cross-plugin/hook-stdin-contract.test.ts)

  • One spec row per gate declaring its fail direction. HA's mcp-safety-gate is the fleet's only fail-closed gate (exit 2); the other ten fail open.
  • Cases: empty stdin, non-JSON garbage, non-object JSON ([1,2] / null / "str"), a well-formed benign payload, and a generated 2MB unparseable payload (generated, not committed — HA's gate-corpus precedent).
  • Drain is asserted via the writer's exit code, piping through bash and reading PIPESTATUS. This matters: a hook that exits mid-stream and one that drains cleanly both report 0 themselves, so the hook's own exit code cannot detect the bug — only the writer's SIGPIPE (141) can.
  • An auto-discovery test walks plugins/*/hooks/hooks.json and checks both directions: a stale/renamed spec fails, and a newly registered PreToolUse gate missing from the corpus fails too.

Two defects it caught (claude-code-dev-hermit)

  • git-push-guard, worktree-boundary-guard, record-test-result all called process.exit(0) inside the read loop on oversize input, abandoning the pipe half-read — the exact failure core's header exists to prevent. worktree-boundary-guard even carried a comment claiming it drained to completion. They now stop buffering but keep consuming to EOF.
  • record-test-result crashed with exit 1 on a null payload: its property access sat outside the parse try/catch, unlike its two siblings. Claude Code treats exit 1 as non-blocking, so this was an uncaught crash masquerading as fail-open.

Wiring

  • test-cross-plugin.yml path filter now includes plugins/*/hooks/** and the three dev guard scripts — none of them triggered this workflow before.
  • scripts/test-all.sh now runs the cross-plugin suite, which it never did. Deliberately serial after the parallel phase: these spawn a subprocess per corpus case, and the parallel phase is already saturated enough that HA's CPU-bound gate-corpus tests sit near their 5s per-test timeout.

Deliberately not done

  • The envelope bytes stay out of scope. HA's two hookSpecificOutput emitters are duplicated on purpose and already byte-pinned by gate-corpus.test.ts against the retired Python gate. Re-asserting them here would duplicate a deliberate pin.
  • No shared drainStdin() helper. A review pass proposed extracting the three now-identical loops into dev-hermit's scripts/lib/. Rejected: the natural helper hides process.exit(0) inside something that reads as "return a string", hardcoding fail-open into the helper and removing each caller's declared fail direction — the exact property this PR makes explicit. Core's own readHookInput returns an OVERSIZE sentinel precisely so the caller decides; doing it that way is a redesign of three hooks.
  • No caps added to HA/forge/feed. readFileSync(0) and Bun.stdin.text() already read to EOF, so they satisfy the drain invariant; the corpus records cap: none rather than forcing a change.

Test plan

  • bun test tests/cross-plugin/ → 141 pass, 0 fail.
  • bunx tsc --noEmit → exit 0.
  • bash plugins/claude-code-dev-hermit/tests/run-all.sh → exit 0.
  • Mutation check: restoring the mid-loop process.exit(0) in git-push-guard.ts fails exactly one test, naming the hook and the oversize case, on the drain assertion (expected 0, received 141) while the hook's own exit code stays 0 — confirming the check is not vacuous.

Known unrelated failure

bash scripts/test-all.sh exits 1 on HA's two gate-corpus oversize tests (~6.1–6.6s vs a 5s per-test timeout). This is pre-existing and not caused by this branch — verified by stashing every change on this branch and re-running the untouched runner, which fails identically. HA's suite passes on its own (32s). Left alone as out of scope.

gtapps added 3 commits August 14, 2026 10:13
Every fleet hook keeps its own sealed stdin copy (plugins can't import core at
runtime), so the drain/cap/fail-direction invariants were re-decided per file
with nothing pinning them. The corpus feeds all 11 gates the same adversarial
payloads on real stdin and asserts each hook's declared fail direction, plus
that the writer completed — a mid-stream exit and a clean drain both report 0
from the hook itself, so the drain is only observable via the writer's SIGPIPE.

It immediately caught two live defects in dev-hermit, both fixed here: the three
guards abandoned the pipe half-read past their 1MB cap (worktree-boundary-guard
even claimed otherwise in a comment), and record-test-result crashed with exit 1
on a null payload because its property access sat outside the parse try/catch.
…raining stdin

worktree-boundary-guard returned on WORKTREE_GUARD=off before reading stdin, so
the documented escape hatch still reproduced the SIGPIPE the drain loop two lines
below was added to prevent: any Write whose content exceeds the pipe buffer left
the writer to die on a half-read pipe. The drain now runs first and the switch is
honored at EOF, matching git-push-guard, which already drains before its own
AGENT_HOOK_PROFILE check. When the guard is off the payload is consumed without
buffering, since it is discarded immediately after.

All three dev hooks called main() bare and, unlike core's lib/hook-input.ts,
never attach a stdin error listener, so a stream error became an unhandled
rejection and exit 1 rather than failing open. They now use the
main().catch(() => process.exit(0)) form core's cache-edit-guard already uses.

The stdin corpus discovered scripts only via a hook's args array, so a PreToolUse
gate registered with an inline command string (a form core's hooks.json already
uses) produced no entry and passed the coverage assertion vacuously. Discovery
now scans both forms, and a new case pins the WORKTREE_GUARD=off drain.
…drain

# Conflicts:
#	.github/workflows/test-cross-plugin.yml
@gtapps
gtapps merged commit df07d11 into main Aug 14, 2026
4 checks passed
@gtapps
gtapps deleted the fix/fleet-hook-stdin-drain branch August 14, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant