Skip to content

fix: deliver stage steering deterministically - bounded rule chunks, dispatch-boundary enforcement, absorbed reviewer checklists (2.5.33) - #658

Merged
apackeer merged 11 commits into
v2from
fix/issue-495-rules-delivery
Aug 1, 2026
Merged

fix: deliver stage steering deterministically - bounded rule chunks, dispatch-boundary enforcement, absorbed reviewer checklists (2.5.33)#658
apackeer merged 11 commits into
v2from
fix/issue-495-rules-delivery

Conversation

@apackeer

@apackeer apackeer commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Fixes #495.

Problem

The engine resolved each stage's rules into rules_in_context but shipped only the file paths in the run-stage directive; whether the content ever reached the model depended on the conductor choosing to read each path. The issue documents live runs where 0/4 memory rules were read across three consecutive ideation stages - the directive "delivered" steering that silently never arrived, and the artifacts contradicted rules that should have shaped them. Two sibling surfaces had the same prose-only delivery: the reviewer agents' reviewing.md checklists, and the rules relayed into dispatched-subagent briefs.

Fix - deterministic delivery on every hop

Engine to conductor. next now emits the active stage's substantive rules as bounded load-steering directives (ordered {path, text} content) before run-stage. Opaque continue tokens bind stage, workflow state, bundle hash, and next part; a fresh next restarts at part 1. Every directive stays below the common 28 KiB output floor - oversized rules split at Markdown headings, then UTF-8 boundaries, with no size-based path fallback. A missing, unreadable, or invalid-UTF-8 required rule stops before stage work with repair guidance (fail closed). Rules re-read every stage, so a §13 learning admitted mid-workflow reaches the very next stage. Comment-only template placeholders (team.md/project.md as shipped) are excluded until they carry an affirmed practice.

Conductor to dispatched worker. A new PreToolUse hook (aidlc-dispatch-rules.ts, backed by the shared resolver aidlc-steering.ts so the two hops cannot drift) appends the exact hash-marked rule bundle to every AI-DLC subagent brief that lacks it. Claude Code, Codex, and opencode rewrite the tool input (updatedInput); idempotent when the bundle is already present. Stage resolution precedence: explicit stage-file path in the brief, then the state file's Current Stage, then a unique slug mention.

Kiro CLI degradation (deliberate). Kiro cannot rewrite tool input, and a block-with-retry contract deadlocked in live testing (the conductor cannot reproduce a multi-KB bundle byte-exactly; every retry re-blocked). Kiro is also the one harness where the invariant already holds without the brief: every delegated agent preloads the full active memory tree natively via its resources glob. So on Kiro an incomplete brief proceeds with an advisory warning; an unloadable required rule (real missing steering, no preload fallback) still blocks with repair guidance. Kiro IDE keeps native memory-resource preload. This mirrors the existing harness-capability pattern (reviewer dispatch record, reviewer-scope enforcement).

Reviewer checklists. knowledge/<agent>/reviewing.md is absorbed into the two reviewer agents' generated bodies at build time on every harness (the reviewer set is derived from stage frontmatter, not hardcoded). The redundant per-agent knowledge globs and self-.md resources entries were removed from the Kiro reviewer JSONs - the checklist now arrives exactly once.

Deliberate non-goal

Persona and supplemental knowledge files remain path-loaded (with actionable context_warnings for missing/unreadable optional files) pending a retrieval system - wholesale content injection of ~130KB knowledge trees is the wrong shape. This is the issue's "Additional Context" sibling surface, scoped out intentionally; follow-up issue to come.

Verification

  • Full tiers green on the branch: smoke+unit 178/178, integration green (t72 latency flake green solo; t145 codex-emitter case red only on a box with stray /tmp package roots - environmental).
  • 9-slice live pre-merge gate run across Claude SDK/TUI, Kiro ACP/TUI, Codex exec, opencode; every red either fixed in-branch, flake-confirmed green solo, or reproduced on clean v2 tip (pre-existing: e2e t113, t-tui-kiro-bugfix, t-tui-t51/t73, t-acp-kiro-journey; these fail without this PR too). Kiro IDE slice skips clean on Linux (needs macOS + Kiro.app) - unverified there, low risk (no dispatch hook registers on the IDE).
  • Live-verified behavior: the load-steering bundle delivered and applied on Claude SDK runs; the dispatch hook's exact-bundle rewrite observed on Codex; the earlier Kiro dispatch deadlock eliminated after the advisory change (zero blocked dispatches).
  • New/updated tests: t248 (steering transport: substantive filter, chunking, budget, continue-token invalidation, dispatch rewrites per harness, stage-resolution precedence), t147 (Kiro adapter advisory contract), t238 (mob briefs carry rule content), t118/t53 (load-steering-aware test contracts), plus t89 fixtures + t66 golden realigned with the claim-sources sensor (pre-existing tip reds folded in per review).

Notes for reviewers

  • BASH_MAX_OUTPUT_LENGTH=150000 + AIDLC_DIRECTIVE_MAX_BYTES=120000 ship in the Claude settings so the directive line survives the tool-output cap; the engine default stays 28 KiB for the other harnesses.
  • An adversarial two-reviewer pass ran mid-development; its P1 (a deliver-once/budget composition that silently downgraded overflowed files to path-reads) was reproduced, fixed, and superseded by the chunked design.
  • Version 2.5.33 + CHANGELOG + README badge bump per policy (re-slotted from 2.5.12 at the merge-prep rebase onto v2 @ 57ae3d0; 2.5.32 was taken by fix: warn when a plugin ships an undiscoverable sensor manifest (2.5.32) #672).

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found six behavioral regressions that should be addressed before merge.

  1. [P1] Stop recovery skips steering content

core/hooks/aidlc-stop.ts:750-815 discards rules_content and forwards only the continuation token. The conductor is told to apply content "already returned," but that content was visible only to the hook subprocess. I reproduced this with a real active workflow: the block reason contained the token but none of org.md.

  1. [P1] Kiro IDE lacks the claimed preload fallback

Kiro IDE delegates use Markdown frontmatter, while memory resources exist only in agent-v1 JSON documented as CLI-only (harness/kiro-ide/manifest.ts:104-107). The IDE also has no dispatch-rule hook registration. Consequently, its workers receive neither deterministic brief augmentation nor native memory preload.

  1. [P1] Large dispatch bundles produce truncated JSON

core/hooks/aidlc-dispatch-rules.ts:92-130,267-279 recombines all chunks into one unbounded hook response. With a 1.19 MB rule file, the hook exited successfully after emitting only 786,432 bytes of invalid JSON. This defeats the bounded engine transport precisely for rules large enough to require chunking.

  1. [P1] Blockquoted policies are silently dropped

core/tools/aidlc-steering.ts:23-33 classifies every > line as non-substantive, although core/knowledge/aidlc-shared/rules-reading.md:24-33 defines only blank and HTML-comment bodies as empty. A team.md containing > NEVER deploy without approval. was absent from both rules_content and rules_in_context in my reproduction.

  1. [P2] Continuation tokens can be altered to skip chunks

Tokens are unsigned base64url JSON (core/tools/aidlc-orchestrate.ts:1668-1707). Changing only i to parts passes the bundle, state, and route checks and immediately emits run-stage. I reproduced this with four parts by changing the first token index to 4.

  1. [P2] Readable knowledge paths can overflow the global directive cap

Only warnings are bounded. inline_context_paths remains unlimited (core/tools/aidlc-orchestrate.ts:1399-1441), while emit() rejects every directive above 28 KiB. Adding 260 valid knowledge files caused the final continuation to exit 1 with refusing to emit a directive larger than 28672 bytes.

Verification passed: 141 focused tests, package drift check, typecheck, lint, and git diff --check. The new tests do not cover these failure paths.

@apackeer

Copy link
Copy Markdown
Contributor Author

Addressed all six review findings in 5069d96:

  • Stop-hook recovery now carries the exact rules_content.
  • Kiro IDE delegates receive active memory through always-included steering.
  • Oversized dispatch responses fail before emitting partial JSON.
  • Blockquoted policies are treated as substantive.
  • Steering continuation tokens reject altered chunk indexes.
  • inline_context_paths is bounded with an omission warning.

Also added regressions for these paths and corrected the related Kiro/plugin documentation and projections. Verification passed: generated-tree parity, typecheck, lint, git diff --check, and 238 focused tests across the affected files.

@leandrodamascena, please re-review when you have a chance.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found five remaining issues that should be addressed before merge.

  1. [P1] Continuation tokens can still be forged to skip chunks

The HMAC key is derived only from the public project path. I recomputed the MAC after changing the first token's index and skipped directly from part 1 of 4 to run-stage.

core/tools/aidlc-orchestrate.ts:1693-1700,1831-1846

  1. [P1] An invalid explicit stage path bypasses rule injection

promptStage() returns null immediately when a path-shaped stage reference contains an unknown slug, instead of falling back to the workflow's Current Stage. I reproduced a dispatch completing with no rule augmentation by including not-a-real-stage.md in the brief.

core/hooks/aidlc-dispatch-rules.ts:80-85

  1. [P1] Plugin agents are excluded from deterministic delivery

isAidlcAgent() requires the aidlc- prefix, excluding valid plugin agents such as test-pro-metrics-agent. Their dispatches therefore receive no active-stage rule bundle.

core/hooks/aidlc-dispatch-rules.ts:47-53

  1. [P2] Markerless or adversarially framed content is accepted as an exact bundle

hasExactBundle() only checks whether each rule text appears somewhere in the prompt. It does not validate the generated digest or framing markers. I reproduced a prompt describing the exact rules as obsolete being accepted without appending the authoritative bundle.

core/hooks/aidlc-dispatch-rules.ts:97-115,126-130

  1. [P2] Kiro blocks valid oversized rules despite its advisory fallback

The Kiro adapter treats every dispatch-hook exit 2 as an unloadable required rule. The hook also returns exit 2 when a valid bundle exceeds its output limit. I reproduced a 600 KB valid rule being blocked rather than proceeding through Kiro's native resource preload.

harness/kiro/hooks/aidlc-kiro-adapter.ts:583-601
core/hooks/aidlc-dispatch-rules.ts:279-287

The other previous findings appear resolved. Verification passed: package parity, typecheck, lint, smoke, all 167 unit files, and 225 focused tests. These adversarial paths are not covered by the current tests.

@apackeer

Copy link
Copy Markdown
Contributor Author

Addressed all five remaining review findings in 80cd9387:

  • Continuation HMACs now use a random 32-byte machine-local key (0600 on POSIX), including an ignored pre-intent fallback; the public project path can no longer re-sign altered chunk indexes.
  • Unknown explicit stage paths now fall back to the workflow's Current Stage.
  • Deterministic dispatch delivery recognizes installed plugin agents through the agent roster.
  • Idempotency now requires the complete digest-framed generated bundle; markerless or adversarially framed rule text cannot suppress injection.
  • Kiro distinguishes valid oversized bundles from unloadable rules: oversized bundles use native active-memory preload, while missing/unreadable rules still block.

Added regressions for each path and updated the reference docs/changelog. Verification passed: package parity, TypeScript typecheck, Biome lint, git diff --check, and the full smoke + unit profile (179 files, 4,445 assertions). Focused steering tests passed 32/32. The integration profile passed aside from the documented live-LLM t72-stage-reverse-engineering timeout; the unrelated t92 PATH issue passed all 47 tests when rerun with the repository toolchain on PATH.

apackeer added 11 commits August 1, 2026 03:31
…tent in the directive, reviewer knowledge absorbed at build time (2.5.12)

The engine shipped rule/persona/knowledge PATHS in the run-stage directive
with prose telling the conductor to read them; live runs showed the reads
skipped non-deterministically (0/4 memory rules read across three
consecutive ideation stages on Kiro IDE). Give steering the same
deterministic anchor the conductor persona has had since SPIKE 6: content
in the directive, not a path the conductor may skip.

- rules_content / rules_content_omitted: each substantive active-space
  rule file's text, re-read every stage (mid-workflow learnings reach the
  next stage); comment-only template placeholders dropped per the
  documented emptiness rule; active-space re-rooting of the
  compile-frozen default-space paths.
- inline_context_content / inline_context_omitted: persona + knowledge
  roster content, delivered once per agent per workflow (derived from
  state checkboxes like the D-E persona signal).
- Total emitted-line byte budget (AIDLC_DIRECTIVE_MAX_BYTES, default
  28KB) with an exact-budget backstop; overflow lands in the *_omitted
  lists - visible, never silent. Shipped Claude settings raise
  BASH_MAX_OUTPUT_LENGTH to 150000 and set the budget to 120000.
- Reviewer knowledge (knowledge/<agent>/reviewing.md) absorbed into the
  reviewer agent bodies at build time on every harness (the reviewer set
  is derived from stage frontmatter); redundant Kiro resources globs
  removed.
- Protocol/skill/docs updated: apply delivered content, read by path only
  what the directive names as omitted; dispatched briefs paste
  rules_content verbatim.
- t248 unit coverage: substantive-filter, populated placeholder, missing
  file, budget partition, tight-budget overflow, deliver-once.

Fixes #495
The mob support-brief assertion pinned the OLD contract (exact rule PATHS
in every brief). Under the steering-delivery change the conductor pastes
or faithfully summarizes the directive's rules_content into briefs - live
runs show both forms - so the pin now requires, per substantive seeded
rule file, a distinctive content token that survives paste AND summary
("first-class" for the org testing posture, "Given/When/Then" for the
inception guardrails) OR the exact path (the rules_content_omitted
fallback). Comment-only team.md/project.md placeholders are no longer
asserted. A brief that drops the rules entirely - the defect class the
change fixes - still fails.

Verified live: t238 solo PASS, stamp tests/logs/2026-07-25T01-38-44Z.
…ew's silent-downgrade gap

Adversarial review (2 reviewers) findings applied:

P1 - deliver-once suppression composed with budget overflow into a
permanent silent steering gap: a roster file that overflowed to
inline_context_omitted on its delivery stage was marked delivered and
then filtered from BOTH lists on every later stage, reverting it to the
discretionary path-read this change eliminates (reproduced at the 28KB
default: the architect persona, omitted on intent-capture, vanished from
feasibility's directive entirely). deliveredInlineContext is now a
content-PRIORITY signal, not a visibility gate: delivered agents' files
skip content but always appear in inline_context_omitted, so the roster
partitions into content + omitted with nothing silently absent, and the
conductor reads omitted entries not already in context. Also covers the
review's P2 (recomposed rosters crediting past stages): an overcount now
costs one visible re-read, never missing steering.

P2 (contract reviewer) - three shipped surfaces still instructed
paths-only briefs, contradicting the new paste-rules_content rule
(conductor.md - baked into every workflow's first directive - the
user-stories round-1 step, 15-stage-definition.md, and the topology
paragraph in all five SKILL.md files). All rewritten to 'artifacts by
path, rules as pasted rules_content'; zero 'paths-only' left outside
dist.

P3s - backstop comment now states the shed-content-only limit honestly;
rulesContentEntries reads a marker-less (plugin-shaped) rule path from
the project dir instead of building a garbage path; the directive CLI
self-check validates a POPULATED rules_content/inline_context_content
example, not just field absence.

t248 grows the partition + budget-starved stage-2 visibility cases
(the old deliver-once test pinned the buggy suppression; corrected).
SKILL.md/stage-protocol/docs prose updated to the partition contract.
…th the claim-sources sensor

Pre-existing reds on the v2 tip since the Intent Capture grounding merge
(2.5.11) added the claim-sources sensor import to intent-capture:

- t89: the five fixture sensor dirs that compile against the REAL stage
  tree lacked an aidlc-claim-sources.md manifest, so compileStageGraph
  threw 'imports unknown sensor id' on every case (13 assertions). Add
  the fixture manifest to those dirs; intent-capture count assertions go
  2 -> 3 and the loadSensors key list gains claim-sources.
- t66: regenerate tests/fixtures/designer-export/export.json from the
  live surface (intent-capture's claim-sources sensor row + reviewer
  fields, approval-handoff's stakeholder-map consume).

Verified: t89 + t66 both green, stamp tests/logs/2026-07-25T03-51-05Z
(111/111 assertions). Test-only - no version bump per changelog policy.
Close the conductor-to-worker hop deterministically: the engine-to-conductor
hop delivers rules as bounded load-steering parts, and this makes the brief
the conductor writes for a dispatched aidlc-*-agent carry the exact same
bundle, no longer trusting the conductor to paste it.

- core/tools/aidlc-steering.ts: shared resolver (substantive-rule filter,
  active-space re-rooting, fail-closed UTF-8 read) extracted from the engine
  so the dispatch hop and the engine hop cannot drift.
- core/hooks/aidlc-dispatch-rules.ts: PreToolUse hook that resolves the
  active stage from the brief (stage-file path, unique slug mention, or the
  state file's Current Stage), and appends the hash-marked rule bundle when
  the brief lacks the exact content. Exit 2 with repair guidance when a
  required rule cannot be loaded. aidlc-composer-agent exempt.
- Harness wiring: Claude settings PreToolUse Task|Agent matcher; Codex
  adapter dispatch-rules route (spawn_agent, updatedInput contract);
  opencode plugin rewrites task args in tool.execute.before; Kiro CLI
  cannot rewrite tool input, so its adapter validates and BLOCKS an
  incomplete subagent brief with verbatim-paste retry guidance; Kiro IDE
  keeps native memory-resource preload.
- sdk-drive captures the post-rewrite executed prompt from Agent/Task tool
  results (allowed tools can bypass canUseTool), so live assertions see
  what the subagent actually received.
- t238 seeds live knowledge evidence and asserts the exact-bundle contract;
  t248 grows dispatch-rewrite cases (Claude prompt rewrite, Codex items
  append, reviewer-checklist embedding per harness); hook-count and
  packaging pins updated for the new hook on every harness.

Tiers on this tree: smoke+unit 178/178 (stamp 2026-07-25T11-35-56Z),
typecheck + package --check + lint green.
… t53

Two test-contract stragglers from the load-steering migration (the unit
twins were migrated in the redesign; these twins were not):

- integration/t118: the WALK cases used createTestProject (deliberately
  empty memory) and a raw next spawn. Since the migration, next on a
  run-stage route fails closed when a required rule file is missing, and
  the routing directive arrives after the load-steering parts. Seed the
  shipped method tree (seedAidlcMemory) and route the six WALK next calls
  through runOrchestrateNext, matching the unit twin.
- e2e/t53: stopAfterToolResult matched '"stage":"requirements-analysis"',
  which the load-steering part-1 result now also carries, stopping the
  drive one beat before run-stage. Pin the adjacent
  '"kind":"run-stage","stage":...' form instead. (Swept the other
  stop conditions: the jump/init/space tokens match strings load-steering
  never emits.)

Verified: t118 15/15 (stamp 2026-07-25T12-33-57Z), t53 1/1 (stamp
2026-07-25T12-35-02Z). Test-only.
…etry deadlocked live

The Kiro CLI dispatch-rules route blocked an incomplete subagent brief
with verbatim-paste retry guidance. The live ACP gate showed the contract
never converges: Kiro cannot rewrite tool input, the conductor cannot
reproduce a multi-KB bundle byte-exactly, and every retry re-blocked
(2-3 blocks per run, zero successful dispatches - both reviewer and
journey files deadlocked mid-stage).

Kiro is the one harness where the rules invariant already holds without
the brief: every delegated agent's config preloads the full active memory
tree via its resources glob before the brief is read. So the validator
becomes an observer there - an incomplete brief proceeds with an advisory
stderr warning; an unloadable REQUIRED rule file (real missing steering,
no preload to fall back on) still blocks with the core hook's repair
guidance. The strict updatedInput rewrite path on Claude, Codex, and
opencode (live-verified green) is unchanged.

Same harness-capability degradation pattern as the reviewer dispatch
record and reviewer-scope enforcement (enforcement-capable harnesses
enforce; Kiro documents). t147 5b pins the new contract; CHANGELOG and
hooks reference updated. Verified: t147 17/17 (stamp
2026-07-25T19-43-17Z), typecheck + package --check green.
…erate recovered live fumbles

Three follow-ups from the gate review, applied in-branch:

- aidlc-dispatch-rules promptStage precedence: stage-file path, then the
  state file's Current Stage, then unique-slug-mention LAST (previously a
  single incidental mention of another stage's slug in brief prose
  outranked the live workflow's active stage and bound the wrong bundle).
  New t248 case pins the scenario (feasibility active, user-stories
  mentioned in passing -> ideation rules delivered).
- The four Kiro reviewer agent JSONs no longer list their own .md in
  resources - the prompt field already loads it, and the absorbed
  reviewing checklist made the second copy ~4KB of duplicate context per
  reviewer dispatch. Reviewers only; the same pattern on the other twelve
  agents is untouched.
- t-acp-kiro-reviewer's toolCallIssues pin rejects only SYSTEMATIC
  issues (hook blocks, repeated failures of one call): a transient
  argument fumble the live agent recovers from - observed on clean v2
  with the stage still completing - is live noise, not the dispatch
  contract under test. Verified live green (stamp 2026-07-25T20-55-45Z).

t248 24/24, t147 17/17, typecheck + package --check + lint green.
@apackeer
apackeer force-pushed the fix/issue-495-rules-delivery branch from 80cd938 to 17a4a82 Compare August 1, 2026 07:30
@apackeer apackeer changed the title fix: deliver stage steering deterministically - bounded rule chunks, dispatch-boundary enforcement, absorbed reviewer checklists (2.5.12) fix: deliver stage steering deterministically - bounded rule chunks, dispatch-boundary enforcement, absorbed reviewer checklists (2.5.33) Aug 1, 2026
@apackeer
apackeer merged commit 9c9201b into v2 Aug 1, 2026
5 checks passed
@apackeer
apackeer deleted the fix/issue-495-rules-delivery branch August 1, 2026 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants