fix: stop reviewer-receipt invalidation loop with deterministic review freeze (2.5.39) - #702
Conversation
leandrodamascena
left a comment
There was a problem hiding this comment.
Thanks for the thorough investigation and for addressing the terminal-receipt ordering in both prose and enforcement. The focused official suites and package parity pass, but I found three behavioral gaps plus a release-version conflict that should block merge.
1. P1: review-freeze is unavailable through the compiled dispatcher
core/hooks/aidlc-review-freeze.ts implements all runtime behavior inside if (import.meta.main) and does not export run(input). The compiled adapter path invokes core hooks through aidlc hook <name>, whose dispatcher requires exactly that export.
Reproduction:
$ bun core/tools/aidlc.ts hook review-freeze <<<"{}"
aidlc hook review-freeze: hook does not export run(input)
Codex and Kiro adapters use AIDLC_COMPILED_EXECUTABLE when running under the compiled distribution. The dispatcher exits 1, while both adapter arms block only on exit 2, so the failure becomes a silent allow and review-freeze is disabled on that supported path.
References: core/hooks/aidlc-review-freeze.ts:164, core/tools/aidlc.ts:976-986, harness/codex/hooks/aidlc-codex-adapter.ts:194-197, and harness/kiro/hooks/aidlc-kiro-adapter.ts:589-610.
Please move the hook body into an exported run(input): Promise<number> and keep the direct entrypoint as process.exit(await run(...)). Add a binary/dispatcher test, not only a source-registration assertion.
2. P1: Bash writes preserve a stale READY receipt as fresh
The hook deliberately excludes Bash, and the PostToolUse audit logger is also registered only for Write/Edit. After a READY receipt, a conductor can therefore mutate reviewed bytes with a normal shell command such as:
printf "change" >> aidlc/.../requirements-analysis/requirements.mdThat write is neither blocked nor recorded as ARTIFACT_UPDATED. freshReviewReceipts() therefore continues returning the old READY receipt, and the gate can approve content that was changed after review. This is worse than merely reopening the original loop: the stale receipt can certify different bytes.
References: core/hooks/aidlc-review-freeze.ts:43-48 and :76-85; harness/claude/settings.json:95-103.
The stated symmetry keeps both mechanisms blind, but it does not preserve the invariant that the receipt covers final artifact bytes. Please either inspect mutation-capable shell commands, make freshness content-based, or otherwise ensure shell writes invalidate the receipt.
3. P1: Kiro delegated writes are outside both freeze and invalidation
Kiro registers review-freeze only on the conductor config. Delegated agents retain fs_write, but their configs do not register review-freeze or the audit feed. This is reachable on reviewer-bearing dispatched stages: code-generation is both mode: subagent and reviewer-bearing.
If the conductor incorrectly re-invokes the lead after READY to apply a suggestion, the delegated writer can change code-generation-plan.md or code-summary.md; the change is neither blocked nor audited, so the old receipt remains fresh.
References: harness/kiro/agents/aidlc.json:74-89, harness/kiro/hooks/aidlc-kiro-adapter.ts:569-579, and core/aidlc-common/stages/construction/code-generation.md:8-15.
Please register the freeze for mutation-capable delegated agents as well, or provide an equivalent deterministic boundary. The current t264 Kiro assertion only verifies the conductor registration and therefore pins the gap rather than detecting it.
4. Merge blocker: version 2.5.35 is already occupied
This branch publishes 2.5.35, but current v2 is 2.5.37 and already contains an unrelated 2.5.35 release. The PR is consequently CONFLICTING / DIRTY. Per the repository conflict-trap policy, it needs to be rebased and re-bumped to the next available version, currently likely 2.5.38, with README and CHANGELOG kept in sync.
References: core/tools/aidlc-version.ts:4 and CHANGELOG.md:4.
Verification
- Official focused runner for t263/t264: 21/21 passed.
- Existing Codex/Kiro/OpenCode adapter tests: 34/34 passed.
bun scripts/package.ts --check: all five branch harnesses in sync.- Remote CI is green.
These results show that the added tests cover the direct hook and registration text, but not the compiled dispatch path or the mutation surfaces above. I recommend keeping this at CHANGES_REQUESTED until the three enforcement gaps are closed and the branch is rebased/re-versioned.
774e19c to
2b01032
Compare
|
Addressed all four blockers from the latest review in
Follow-up review also hardened shell operand handling for target-directory commands, Verification:
|
leandrodamascena
left a comment
There was a problem hiding this comment.
The four blockers from my previous review are resolved:
- Compiled dispatcher support now exports and tests
run(input). - Shell and unobserved writes are covered by pre-write detection plus artifact fingerprints.
- Kiro delegated writers now receive freeze and invalidation hooks.
- The branch is rebased and versioned as
2.5.38.
Focused tests, swarm integration, package parity, and remote CI pass. No remaining blocking findings.
Approved.
….5.35) A conductor that applied reviewer recommendations AFTER recording the terminal review receipt invalidated its own receipt (a later produces[] write voids it - deliberate fail-closed behavior in verifyReviewerPrecondition), re-reviewed, re-edited, and oscillated until the live session wedged at the gate (the t-tui-t73 standing red). - stage-protocol §12a READY branch: the recorded receipt is TERMINAL - no produces[] writes between the receipt and gate approval; suggestions riding on a READY verdict are quoted at the gate, never applied. - all five harness SKILL.md reviewer steps carry the same ordering. - aidlc-state.ts reviewerPreconditionError now teaches the terminal ordering instead of only asking for a fresh receipt (the old message re-triggered the loop). - t263 pins the prose in core + dist + all harness skills and drives the write-after-receipt refusal through the real CLI (mutation-checked both directions). - t-tui-t73 per-gate budget 200s -> 420s: the 200s figure predates the intent-capture reviewer (2.5.10); a legitimate single-pass review takes ~2.5-5 min between menu repaints, so the old budget starved it. Live verified: t-tui-t73 FAIL at 200s (reviewer mid-flight at timeout), PASS at 420s with exactly one reviewer pass and the conductor quoting the new terminal-ordering prose.
…g as a PreToolUse control The prose fix teaches the ordering; this makes it self-enforcing per the framework layering (determinism -> tools/hooks, knowledge -> agents, judgement -> humans). The engine already invalidates a REVIEW_COMPLETED receipt on a later produces[] write; nothing refused the write itself, so a conductor could still void its own receipt and loop. - freshReviewReceipts + producesArtifactFile/Unit + KNOWN_CODEKB_STAGES extracted from aidlc-state.ts into aidlc-lib.ts: ONE receipt scan shared by the completion precondition and the new hook, so the freeze window and the refusal window cannot diverge. verifyReviewerPrecondition now consumes the shared scan (behavior unchanged; t115/t205 green). - core/hooks/aidlc-review-freeze.ts (15th hook): refuses Write/Edit/ MultiEdit/NotebookEdit to a reviewer-bearing, not-yet-completed stage's declared produces[] artifact while a fresh READY receipt covers it (per-unit scoped on for_each stages). NOT-READY never freezes; GATE_REJECTED/jump/restart lift the freeze via the shared floor. Bash deliberately not inspected - symmetric with the audit-logger's Write/Edit-only invalidation feed. Fail-open everywhere; off-switch AIDLC_DISABLE_REVIEW_FREEZE_HOOK=1; emits REVIEW_FREEZE_BLOCKED (75th audit event). - Registered: Claude settings.json (shared PreToolUse group), Codex emit.ts + adapter (apply_patch fan-out incl Delete/Move), Kiro CLI conductor fs_write, opencode plugin write/edit/apply_patch. Kiro IDE stays prose-governed (no stable PreToolUse tool inputs) - absence pinned. - t264: pure decision table + real-ledger lifecycle (allow -> READY block -> reject release -> re-block; audit row; fail-open; off-switch; completed-stage exclusion) + per-harness registration pins. Mutation-checked: freeze arm removed from dist -> 3 cases red. - Count pins bumped deliberately: t01 (71 paths, 15 hooks), t02, t28/t81/ t111/t239 (75 events), t132 (fifteen), t148 (kiro fs_write), t150 (codex trust pre_tool_use:3), t219 (17 project-dir refs); docs swept (14->15 hooks, 74->75 events, five flow-altering / four PreToolUse). - Live verified: t-tui-t73 PASS with the hook active - one reviewer pass, zero freeze blocks on the healthy path (stamp 2026-08-02T04-29-57Z). Integration tier: t47 fixed (comment bracket trap), t193 + t-journey-workspace re-ran green alone (live-agent variance, reds also seen on base).
… budget in t73 Two findings from the full sliced pre-merge gate: - t139's clean-approve control run showed a conductor rendering "Request Changes (apply the reviewer's fix)" as the FIRST gate option after a READY-with-suggestion review - steering recommended-option drivers (and habituated humans) into rejecting a stage the reviewer passed. §12a now states that riding suggestions never reorder the gate: the §1 approval question keeps its standard option order (Approve first). Pinned in t263. - t73's 420s per-gate budget could not fit the protocol's own two-pass cycle (reviewer_max_iterations: 2): a live run recorded iteration 2's terminal receipt 5.8s before the cap fired, with a single pass alone consuming 350-400s. Raised to 900s; the overall timeout stays the hard ceiling. Live verified green: two review passes (NOT-READY -> fix -> READY), terminal receipt, gate answered, zero freeze blocks.
3e0fd41 to
4764fc2
Compare
Union-merges with the two reviewer-adjacent PRs that merged mid-gate: - #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH the terminal-receipt discipline and the advisory/adversarial class branch; the freeze prose applies to any receipt no further pass follows. - #703 (summary confirmation): schema/graph/lib field rosters carry review_class AND summary_confirmation. Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count pins re-swept; coverage registry regenerated; export golden regenerated.
Union-merges with the two reviewer-adjacent PRs that merged mid-gate: - #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH the terminal-receipt discipline and the advisory/adversarial class branch; the freeze prose applies to any receipt no further pass follows. - #703 (summary confirmation): schema/graph/lib field rosters carry review_class AND summary_confirmation. Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count pins re-swept; coverage registry regenerated; export golden regenerated.
Union-merges with the two reviewer-adjacent PRs that merged mid-gate: - #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH the terminal-receipt discipline and the advisory/adversarial class branch; the freeze prose applies to any receipt no further pass follows. - #703 (summary confirmation): schema/graph/lib field rosters carry review_class AND summary_confirmation. Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count pins re-swept; coverage registry regenerated; export golden regenerated.
Union-merges with the two reviewer-adjacent PRs that merged mid-gate: - #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH the terminal-receipt discipline and the advisory/adversarial class branch; the freeze prose applies to any receipt no further pass follows. - #703 (summary confirmation): schema/graph/lib field rosters carry review_class AND summary_confirmation. Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count pins re-swept; coverage registry regenerated; export golden regenerated.
… dial (2.5.54) (#718) * feat: reviewer class - adversarial | advisory | none, the review-cost dial (2.5.40) Stage reviews now run under a review class resolved by the engine from three inputs (low-wins): the stage's declared class, the active scope's review_cap, and a per-run --review override. The 7 human-gated ideation/inception prose stages default to a single advisory pass whose findings are quoted verbatim at the approval gate; the 5 Construction design/build stages keep the full adversarial refute-fix-re-review loop. Live A/B runs against the real harness measured the adversarial loop at 12+ minutes of review choreography per inception stage (2.4x the no-reviewer stage baseline) with the two-pass cycle occurring even on clean artifacts; advisory single-pass surfaced the same finding classes at roughly a quarter of the wall-clock. - review_class stage frontmatter (schema-validated, requires reviewer; compile defaults adversarial) rides through stage-graph.json onto the run-stage directive; a none resolution omits the reviewer block. - Scope review_cap (bugfix/poc/workshop ship advisory) + per-run /aidlc --review via config-change; Review Override state field, REVIEW_CLASS_CHANGED audit event (75-event taxonomy), config get/list. - Engine-enforced iteration ceiling: aidlc-log review refuses a REVIEW_REQUESTED beyond the effective budget (advisory 1, adversarial reviewer_max_iterations, none 0); refusal text teaches the terminal path; REVIEW_COMPLETED receipts never refused; fail-open on resolution errors; per-unit (--unit) requests use the declared class (swarm exemption - inside a Bolt the reviewer is the only pre-merge check). - stage-protocol 12a + all 5 harness SKILL.md reviewer steps carry the class branch; reviewer personas gain the Advisory Dispatch stance. - balanced tier (the reviewer tier) pins medium effort on Claude, Codex, and opencode: live A/B showed medium reviews at ~half the xhigh wall-clock with no finding-quality loss. - t265 (schema/graph/resolution/prose pins) + t266 (iteration ceiling via the real CLI); count pins bumped (t28/t81/t216/t220/t231/t239, coverage registry); docs swept (guide 06/10/12, reference 04/12/15, event-count mentions, onboarding fills). * fix: emit review_class in emitStageFrontmatter FIELD_ORDER + regen designer-export golden t65's parse->emit->parse round-trip dropped review_class (the emitter's FIELD_ORDER did not know the new key), and t66's designer-export golden predates the field. Verified the golden diff is exactly the 12 review_class rows (7 advisory + 5 adversarial), nothing else. * fix: post-rebase reconciliation onto v2 @ ebfa3ce (702/703 landed) Union-merges with the two reviewer-adjacent PRs that merged mid-gate: - #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH the terminal-receipt discipline and the advisory/adversarial class branch; the freeze prose applies to any receipt no further pass follows. - #703 (summary confirmation): schema/graph/lib field rosters carry review_class AND summary_confirmation. Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count pins re-swept; coverage registry regenerated; export golden regenerated. * fix: close reviewer-class enforcement gaps * test: pair reviewer receipt fixtures * fix: close reviewer-class recovery gaps * fix: address reviewer-class approval follow-ups
Fixes #698.
The bug
Reviewer-bearing stages could wedge at approval. If a conductor applied a suggestion after recording the terminal
REVIEW_COMPLETEDreceipt, the later artifact write invalidated that receipt, causing another review/edit cycle instead of reaching the gate.The fix
aidlc-review-freeze.tsPreToolUse enforcement blocks file-tool, apply-patch, and mutation-capable shell writes to declaredproduces[]artifacts while a fresh READY receipt covers them. It exports the compiledrun(input)contract and is registered on Claude, Codex, every writable Kiro CLI conductor/delegate surface, and opencode. Kiro IDE remains prose-governed.AIDLC_DISABLE_REVIEW_FREEZE_HOOK=1disables enforcement.Rebase reconciliation
PR #703 landed first as
2.5.38with the mandatory summary-confirmation receipt. This branch is rebased onto that head and published as2.5.39; both receipt systems coexist, andREVIEW_FREEZE_BLOCKEDraises the audit taxonomy to 76 events.The current TUI driver uses the overall deadline as its hang backstop, superseding this branch's earlier fixed 900-second per-gate workaround.
Verification
bun run check: package parity, all TypeScript configs, and Biome pass.t135): 12 assertions, 0 failures.Upgrade: re-copy your selected
dist/<harness>/shell into the project so the updated protocol, skills, hooks, tools, and agent configs are installed.