Skip to content

fix: stop reviewer-receipt invalidation loop with deterministic review freeze (2.5.39) - #702

Merged
apackeer merged 6 commits into
v2from
fix/issue-698-reviewer-receipt-loop
Aug 5, 2026
Merged

fix: stop reviewer-receipt invalidation loop with deterministic review freeze (2.5.39)#702
apackeer merged 6 commits into
v2from
fix/issue-698-reviewer-receipt-loop

Conversation

@apackeer

@apackeer apackeer commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #698.

The bug

Reviewer-bearing stages could wedge at approval. If a conductor applied a suggestion after recording the terminal REVIEW_COMPLETED receipt, the later artifact write invalidated that receipt, causing another review/edit cycle instead of reaching the gate.

The fix

  • Stage protocol section 12a makes a READY receipt terminal: apply fixes inside the reviewer loop, stop artifact writes after the terminal receipt, quote optional suggestions at the gate, and keep Approve as the first option.
  • New aidlc-review-freeze.ts PreToolUse enforcement blocks file-tool, apply-patch, and mutation-capable shell writes to declared produces[] artifacts while a fresh READY receipt covers them. It exports the compiled run(input) contract and is registered on Claude, Codex, every writable Kiro CLI conductor/delegate surface, and opencode. Kiro IDE remains prose-governed.
  • Every completed review records a SHA-256 fingerprint of its declared artifact set. Approval and swarm finalization require that fingerprint to remain current, so unobserved shell, interpreter, or delegated writes cannot preserve a stale receipt.
  • A gate rejection, jump, or workflow restart reopens artifact work. NOT-READY never freezes. AIDLC_DISABLE_REVIEW_FREEZE_HOOK=1 disables enforcement.

Rebase reconciliation

PR #703 landed first as 2.5.38 with the mandatory summary-confirmation receipt. This branch is rebased onto that head and published as 2.5.39; both receipt systems coexist, and REVIEW_FREEZE_BLOCKED raises the audit taxonomy to 76 events.

The current TUI driver uses the overall deadline as its hang backstop, superseding this branch's earlier fixed 900-second per-gate workaround.

Verification

  • bun run check: package parity, all TypeScript configs, and Biome pass.
  • Full smoke + unit: 190 files, 4,684 assertions, 0 failures.
  • Focused swarm integration (t135): 12 assertions, 0 failures.
  • Focused terminal-ordering, review-freeze, dispatcher, packaging, version, and taxonomy tests pass.

Upgrade: re-copy your selected dist/<harness>/ shell into the project so the updated protocol, skills, hooks, tools, and agent configs are installed.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough investigation and for addressing the terminal-receipt ordering in both prose and enforcement. The focused official suites and package parity pass, but I found three behavioral gaps plus a release-version conflict that should block merge.

1. P1: review-freeze is unavailable through the compiled dispatcher

core/hooks/aidlc-review-freeze.ts implements all runtime behavior inside if (import.meta.main) and does not export run(input). The compiled adapter path invokes core hooks through aidlc hook <name>, whose dispatcher requires exactly that export.

Reproduction:

$ bun core/tools/aidlc.ts hook review-freeze <<<"{}"
aidlc hook review-freeze: hook does not export run(input)

Codex and Kiro adapters use AIDLC_COMPILED_EXECUTABLE when running under the compiled distribution. The dispatcher exits 1, while both adapter arms block only on exit 2, so the failure becomes a silent allow and review-freeze is disabled on that supported path.

References: core/hooks/aidlc-review-freeze.ts:164, core/tools/aidlc.ts:976-986, harness/codex/hooks/aidlc-codex-adapter.ts:194-197, and harness/kiro/hooks/aidlc-kiro-adapter.ts:589-610.

Please move the hook body into an exported run(input): Promise<number> and keep the direct entrypoint as process.exit(await run(...)). Add a binary/dispatcher test, not only a source-registration assertion.

2. P1: Bash writes preserve a stale READY receipt as fresh

The hook deliberately excludes Bash, and the PostToolUse audit logger is also registered only for Write/Edit. After a READY receipt, a conductor can therefore mutate reviewed bytes with a normal shell command such as:

printf "change" >> aidlc/.../requirements-analysis/requirements.md

That write is neither blocked nor recorded as ARTIFACT_UPDATED. freshReviewReceipts() therefore continues returning the old READY receipt, and the gate can approve content that was changed after review. This is worse than merely reopening the original loop: the stale receipt can certify different bytes.

References: core/hooks/aidlc-review-freeze.ts:43-48 and :76-85; harness/claude/settings.json:95-103.

The stated symmetry keeps both mechanisms blind, but it does not preserve the invariant that the receipt covers final artifact bytes. Please either inspect mutation-capable shell commands, make freshness content-based, or otherwise ensure shell writes invalidate the receipt.

3. P1: Kiro delegated writes are outside both freeze and invalidation

Kiro registers review-freeze only on the conductor config. Delegated agents retain fs_write, but their configs do not register review-freeze or the audit feed. This is reachable on reviewer-bearing dispatched stages: code-generation is both mode: subagent and reviewer-bearing.

If the conductor incorrectly re-invokes the lead after READY to apply a suggestion, the delegated writer can change code-generation-plan.md or code-summary.md; the change is neither blocked nor audited, so the old receipt remains fresh.

References: harness/kiro/agents/aidlc.json:74-89, harness/kiro/hooks/aidlc-kiro-adapter.ts:569-579, and core/aidlc-common/stages/construction/code-generation.md:8-15.

Please register the freeze for mutation-capable delegated agents as well, or provide an equivalent deterministic boundary. The current t264 Kiro assertion only verifies the conductor registration and therefore pins the gap rather than detecting it.

4. Merge blocker: version 2.5.35 is already occupied

This branch publishes 2.5.35, but current v2 is 2.5.37 and already contains an unrelated 2.5.35 release. The PR is consequently CONFLICTING / DIRTY. Per the repository conflict-trap policy, it needs to be rebased and re-bumped to the next available version, currently likely 2.5.38, with README and CHANGELOG kept in sync.

References: core/tools/aidlc-version.ts:4 and CHANGELOG.md:4.

Verification

  • Official focused runner for t263/t264: 21/21 passed.
  • Existing Codex/Kiro/OpenCode adapter tests: 34/34 passed.
  • bun scripts/package.ts --check: all five branch harnesses in sync.
  • Remote CI is green.

These results show that the added tests cover the direct hook and registration text, but not the compiled dispatch path or the mutation surfaces above. I recommend keeping this at CHANGES_REQUESTED until the three enforcement gaps are closed and the branch is rebased/re-versioned.

@apackeer
apackeer force-pushed the fix/issue-698-reviewer-receipt-loop branch from 774e19c to 2b01032 Compare August 5, 2026 08:00
@apackeer

apackeer commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Addressed all four blockers from the latest review in 2b010324:

  • Exported run(input) from aidlc-review-freeze.ts and added dispatcher/compiled-binary coverage.
  • Added mutation-capable shell targeting plus SHA-256 artifact fingerprints, so approval and swarm finalization reject changed or missing reviewed artifacts regardless of write mechanism.
  • Registered review freeze and post-write invalidation across writable Kiro CLI conductor/delegate surfaces.
  • Rebased onto current v2 and synchronized the release version, README badge, and changelog at 2.5.38.

Follow-up review also hardened shell operand handling for target-directory commands, mv sources, multi-file sed/perl, truncate options, and read-only source/reference operands.

Verification:

  • Focused smoke/unit/integration/e2e slice: 222 assertions passed, 0 failed.
  • Final review-freeze slice: 19 passed, 0 failed.
  • bun run check: package parity, all TypeScript projects, and Biome passed.
  • Coverage registry freshness/ratchet check passed.
  • Independent final re-review found no remaining issue in the follow-up parser changes.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The four blockers from my previous review are resolved:

  • Compiled dispatcher support now exports and tests run(input).
  • Shell and unobserved writes are covered by pre-write detection plus artifact fingerprints.
  • Kiro delegated writers now receive freeze and invalidation hooks.
  • The branch is rebased and versioned as 2.5.38.

Focused tests, swarm integration, package parity, and remote CI pass. No remaining blocking findings.

Approved.

….5.35)

A conductor that applied reviewer recommendations AFTER recording the
terminal review receipt invalidated its own receipt (a later produces[]
write voids it - deliberate fail-closed behavior in
verifyReviewerPrecondition), re-reviewed, re-edited, and oscillated until
the live session wedged at the gate (the t-tui-t73 standing red).

- stage-protocol §12a READY branch: the recorded receipt is TERMINAL - no
  produces[] writes between the receipt and gate approval; suggestions
  riding on a READY verdict are quoted at the gate, never applied.
- all five harness SKILL.md reviewer steps carry the same ordering.
- aidlc-state.ts reviewerPreconditionError now teaches the terminal
  ordering instead of only asking for a fresh receipt (the old message
  re-triggered the loop).
- t263 pins the prose in core + dist + all harness skills and drives the
  write-after-receipt refusal through the real CLI (mutation-checked both
  directions).
- t-tui-t73 per-gate budget 200s -> 420s: the 200s figure predates the
  intent-capture reviewer (2.5.10); a legitimate single-pass review takes
  ~2.5-5 min between menu repaints, so the old budget starved it. Live
  verified: t-tui-t73 FAIL at 200s (reviewer mid-flight at timeout),
  PASS at 420s with exactly one reviewer pass and the conductor quoting
  the new terminal-ordering prose.
…g as a PreToolUse control

The prose fix teaches the ordering; this makes it self-enforcing per the
framework layering (determinism -> tools/hooks, knowledge -> agents,
judgement -> humans). The engine already invalidates a REVIEW_COMPLETED
receipt on a later produces[] write; nothing refused the write itself, so
a conductor could still void its own receipt and loop.

- freshReviewReceipts + producesArtifactFile/Unit + KNOWN_CODEKB_STAGES
  extracted from aidlc-state.ts into aidlc-lib.ts: ONE receipt scan shared
  by the completion precondition and the new hook, so the freeze window
  and the refusal window cannot diverge. verifyReviewerPrecondition now
  consumes the shared scan (behavior unchanged; t115/t205 green).
- core/hooks/aidlc-review-freeze.ts (15th hook): refuses Write/Edit/
  MultiEdit/NotebookEdit to a reviewer-bearing, not-yet-completed stage's
  declared produces[] artifact while a fresh READY receipt covers it
  (per-unit scoped on for_each stages). NOT-READY never freezes;
  GATE_REJECTED/jump/restart lift the freeze via the shared floor. Bash
  deliberately not inspected - symmetric with the audit-logger's
  Write/Edit-only invalidation feed. Fail-open everywhere; off-switch
  AIDLC_DISABLE_REVIEW_FREEZE_HOOK=1; emits REVIEW_FREEZE_BLOCKED (75th
  audit event).
- Registered: Claude settings.json (shared PreToolUse group), Codex
  emit.ts + adapter (apply_patch fan-out incl Delete/Move), Kiro CLI
  conductor fs_write, opencode plugin write/edit/apply_patch. Kiro IDE
  stays prose-governed (no stable PreToolUse tool inputs) - absence
  pinned.
- t264: pure decision table + real-ledger lifecycle (allow -> READY block
  -> reject release -> re-block; audit row; fail-open; off-switch;
  completed-stage exclusion) + per-harness registration pins.
  Mutation-checked: freeze arm removed from dist -> 3 cases red.
- Count pins bumped deliberately: t01 (71 paths, 15 hooks), t02, t28/t81/
  t111/t239 (75 events), t132 (fifteen), t148 (kiro fs_write), t150
  (codex trust pre_tool_use:3), t219 (17 project-dir refs); docs swept
  (14->15 hooks, 74->75 events, five flow-altering / four PreToolUse).
- Live verified: t-tui-t73 PASS with the hook active - one reviewer pass,
  zero freeze blocks on the healthy path (stamp 2026-08-02T04-29-57Z).
  Integration tier: t47 fixed (comment bracket trap), t193 +
  t-journey-workspace re-ran green alone (live-agent variance, reds also
  seen on base).
… budget in t73

Two findings from the full sliced pre-merge gate:

- t139's clean-approve control run showed a conductor rendering "Request
  Changes (apply the reviewer's fix)" as the FIRST gate option after a
  READY-with-suggestion review - steering recommended-option drivers (and
  habituated humans) into rejecting a stage the reviewer passed. §12a now
  states that riding suggestions never reorder the gate: the §1 approval
  question keeps its standard option order (Approve first). Pinned in t263.
- t73's 420s per-gate budget could not fit the protocol's own two-pass
  cycle (reviewer_max_iterations: 2): a live run recorded iteration 2's
  terminal receipt 5.8s before the cap fired, with a single pass alone
  consuming 350-400s. Raised to 900s; the overall timeout stays the hard
  ceiling. Live verified green: two review passes (NOT-READY -> fix ->
  READY), terminal receipt, gate answered, zero freeze blocks.
@apackeer apackeer changed the title fix: stop the reviewer-receipt invalidation loop - terminal ordering + deterministic review-freeze hook (2.5.35) fix: stop reviewer-receipt invalidation loop with deterministic review freeze (2.5.39) Aug 5, 2026
@apackeer
apackeer force-pushed the fix/issue-698-reviewer-receipt-loop branch from 3e0fd41 to 4764fc2 Compare August 5, 2026 18:54
@apackeer
apackeer merged commit ebfa3ce into v2 Aug 5, 2026
5 checks passed
@apackeer
apackeer deleted the fix/issue-698-reviewer-receipt-loop branch August 5, 2026 19:08
apackeer added a commit that referenced this pull request Aug 6, 2026
Union-merges with the two reviewer-adjacent PRs that merged mid-gate:
- #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH
  the terminal-receipt discipline and the advisory/adversarial class
  branch; the freeze prose applies to any receipt no further pass follows.
- #703 (summary confirmation): schema/graph/lib field rosters carry
  review_class AND summary_confirmation.
Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count
pins re-swept; coverage registry regenerated; export golden regenerated.
apackeer added a commit that referenced this pull request Aug 6, 2026
Union-merges with the two reviewer-adjacent PRs that merged mid-gate:
- #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH
  the terminal-receipt discipline and the advisory/adversarial class
  branch; the freeze prose applies to any receipt no further pass follows.
- #703 (summary confirmation): schema/graph/lib field rosters carry
  review_class AND summary_confirmation.
Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count
pins re-swept; coverage registry regenerated; export golden regenerated.
apackeer added a commit that referenced this pull request Aug 6, 2026
Union-merges with the two reviewer-adjacent PRs that merged mid-gate:
- #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH
  the terminal-receipt discipline and the advisory/adversarial class
  branch; the freeze prose applies to any receipt no further pass follows.
- #703 (summary confirmation): schema/graph/lib field rosters carry
  review_class AND summary_confirmation.
Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count
pins re-swept; coverage registry regenerated; export golden regenerated.
apackeer added a commit that referenced this pull request Aug 7, 2026
Union-merges with the two reviewer-adjacent PRs that merged mid-gate:
- #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH
  the terminal-receipt discipline and the advisory/adversarial class
  branch; the freeze prose applies to any receipt no further pass follows.
- #703 (summary confirmation): schema/graph/lib field rosters carry
  review_class AND summary_confirmation.
Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count
pins re-swept; coverage registry regenerated; export golden regenerated.
apackeer added a commit that referenced this pull request Aug 7, 2026
… dial (2.5.54) (#718)

* feat: reviewer class - adversarial | advisory | none, the review-cost dial (2.5.40)

Stage reviews now run under a review class resolved by the engine from
three inputs (low-wins): the stage's declared class, the active scope's
review_cap, and a per-run --review override. The 7 human-gated
ideation/inception prose stages default to a single advisory pass whose
findings are quoted verbatim at the approval gate; the 5 Construction
design/build stages keep the full adversarial refute-fix-re-review loop.
Live A/B runs against the real harness measured the adversarial loop at
12+ minutes of review choreography per inception stage (2.4x the
no-reviewer stage baseline) with the two-pass cycle occurring even on
clean artifacts; advisory single-pass surfaced the same finding classes
at roughly a quarter of the wall-clock.

- review_class stage frontmatter (schema-validated, requires reviewer;
  compile defaults adversarial) rides through stage-graph.json onto the
  run-stage directive; a none resolution omits the reviewer block.
- Scope review_cap (bugfix/poc/workshop ship advisory) + per-run
  /aidlc --review via config-change; Review Override state field,
  REVIEW_CLASS_CHANGED audit event (75-event taxonomy), config get/list.
- Engine-enforced iteration ceiling: aidlc-log review refuses a
  REVIEW_REQUESTED beyond the effective budget (advisory 1, adversarial
  reviewer_max_iterations, none 0); refusal text teaches the terminal
  path; REVIEW_COMPLETED receipts never refused; fail-open on resolution
  errors; per-unit (--unit) requests use the declared class (swarm
  exemption - inside a Bolt the reviewer is the only pre-merge check).
- stage-protocol 12a + all 5 harness SKILL.md reviewer steps carry the
  class branch; reviewer personas gain the Advisory Dispatch stance.
- balanced tier (the reviewer tier) pins medium effort on Claude, Codex,
  and opencode: live A/B showed medium reviews at ~half the xhigh
  wall-clock with no finding-quality loss.
- t265 (schema/graph/resolution/prose pins) + t266 (iteration ceiling
  via the real CLI); count pins bumped (t28/t81/t216/t220/t231/t239,
  coverage registry); docs swept (guide 06/10/12, reference 04/12/15,
  event-count mentions, onboarding fills).

* fix: emit review_class in emitStageFrontmatter FIELD_ORDER + regen designer-export golden

t65's parse->emit->parse round-trip dropped review_class (the emitter's
FIELD_ORDER did not know the new key), and t66's designer-export golden
predates the field. Verified the golden diff is exactly the 12
review_class rows (7 advisory + 5 adversarial), nothing else.

* fix: post-rebase reconciliation onto v2 @ ebfa3ce (702/703 landed)

Union-merges with the two reviewer-adjacent PRs that merged mid-gate:
- #702 (review freeze + terminal receipts): §12a step 3 now carries BOTH
  the terminal-receipt discipline and the advisory/adversarial class
  branch; the freeze prose applies to any receipt no further pass follows.
- #703 (summary confirmation): schema/graph/lib field rosters carry
  review_class AND summary_confirmation.
Event taxonomy re-based 76 -> 77 (REVIEW_CLASS_CHANGED); README/docs/count
pins re-swept; coverage registry regenerated; export golden regenerated.

* fix: close reviewer-class enforcement gaps

* test: pair reviewer receipt fixtures

* fix: close reviewer-class recovery gaps

* fix: address reviewer-class approval follow-ups
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants