fix: deliver stage steering deterministically - bounded rule chunks, dispatch-boundary enforcement, absorbed reviewer checklists (2.5.33) - #658
Conversation
leandrodamascena
left a comment
There was a problem hiding this comment.
I found six behavioral regressions that should be addressed before merge.
- [P1] Stop recovery skips steering content
core/hooks/aidlc-stop.ts:750-815 discards rules_content and forwards only the continuation token. The conductor is told to apply content "already returned," but that content was visible only to the hook subprocess. I reproduced this with a real active workflow: the block reason contained the token but none of org.md.
- [P1] Kiro IDE lacks the claimed preload fallback
Kiro IDE delegates use Markdown frontmatter, while memory resources exist only in agent-v1 JSON documented as CLI-only (harness/kiro-ide/manifest.ts:104-107). The IDE also has no dispatch-rule hook registration. Consequently, its workers receive neither deterministic brief augmentation nor native memory preload.
- [P1] Large dispatch bundles produce truncated JSON
core/hooks/aidlc-dispatch-rules.ts:92-130,267-279 recombines all chunks into one unbounded hook response. With a 1.19 MB rule file, the hook exited successfully after emitting only 786,432 bytes of invalid JSON. This defeats the bounded engine transport precisely for rules large enough to require chunking.
- [P1] Blockquoted policies are silently dropped
core/tools/aidlc-steering.ts:23-33 classifies every > line as non-substantive, although core/knowledge/aidlc-shared/rules-reading.md:24-33 defines only blank and HTML-comment bodies as empty. A team.md containing > NEVER deploy without approval. was absent from both rules_content and rules_in_context in my reproduction.
- [P2] Continuation tokens can be altered to skip chunks
Tokens are unsigned base64url JSON (core/tools/aidlc-orchestrate.ts:1668-1707). Changing only i to parts passes the bundle, state, and route checks and immediately emits run-stage. I reproduced this with four parts by changing the first token index to 4.
- [P2] Readable knowledge paths can overflow the global directive cap
Only warnings are bounded. inline_context_paths remains unlimited (core/tools/aidlc-orchestrate.ts:1399-1441), while emit() rejects every directive above 28 KiB. Adding 260 valid knowledge files caused the final continuation to exit 1 with refusing to emit a directive larger than 28672 bytes.
Verification passed: 141 focused tests, package drift check, typecheck, lint, and git diff --check. The new tests do not cover these failure paths.
|
Addressed all six review findings in
Also added regressions for these paths and corrected the related Kiro/plugin documentation and projections. Verification passed: generated-tree parity, typecheck, lint, @leandrodamascena, please re-review when you have a chance. |
5069d96 to
a29795c
Compare
leandrodamascena
left a comment
There was a problem hiding this comment.
I found five remaining issues that should be addressed before merge.
- [P1] Continuation tokens can still be forged to skip chunks
The HMAC key is derived only from the public project path. I recomputed the MAC after changing the first token's index and skipped directly from part 1 of 4 to run-stage.
core/tools/aidlc-orchestrate.ts:1693-1700,1831-1846
- [P1] An invalid explicit stage path bypasses rule injection
promptStage() returns null immediately when a path-shaped stage reference contains an unknown slug, instead of falling back to the workflow's Current Stage. I reproduced a dispatch completing with no rule augmentation by including not-a-real-stage.md in the brief.
core/hooks/aidlc-dispatch-rules.ts:80-85
- [P1] Plugin agents are excluded from deterministic delivery
isAidlcAgent() requires the aidlc- prefix, excluding valid plugin agents such as test-pro-metrics-agent. Their dispatches therefore receive no active-stage rule bundle.
core/hooks/aidlc-dispatch-rules.ts:47-53
- [P2] Markerless or adversarially framed content is accepted as an exact bundle
hasExactBundle() only checks whether each rule text appears somewhere in the prompt. It does not validate the generated digest or framing markers. I reproduced a prompt describing the exact rules as obsolete being accepted without appending the authoritative bundle.
core/hooks/aidlc-dispatch-rules.ts:97-115,126-130
- [P2] Kiro blocks valid oversized rules despite its advisory fallback
The Kiro adapter treats every dispatch-hook exit 2 as an unloadable required rule. The hook also returns exit 2 when a valid bundle exceeds its output limit. I reproduced a 600 KB valid rule being blocked rather than proceeding through Kiro's native resource preload.
harness/kiro/hooks/aidlc-kiro-adapter.ts:583-601
core/hooks/aidlc-dispatch-rules.ts:279-287
The other previous findings appear resolved. Verification passed: package parity, typecheck, lint, smoke, all 167 unit files, and 225 focused tests. These adversarial paths are not covered by the current tests.
|
Addressed all five remaining review findings in
Added regressions for each path and updated the reference docs/changelog. Verification passed: package parity, TypeScript typecheck, Biome lint, |
…tent in the directive, reviewer knowledge absorbed at build time (2.5.12) The engine shipped rule/persona/knowledge PATHS in the run-stage directive with prose telling the conductor to read them; live runs showed the reads skipped non-deterministically (0/4 memory rules read across three consecutive ideation stages on Kiro IDE). Give steering the same deterministic anchor the conductor persona has had since SPIKE 6: content in the directive, not a path the conductor may skip. - rules_content / rules_content_omitted: each substantive active-space rule file's text, re-read every stage (mid-workflow learnings reach the next stage); comment-only template placeholders dropped per the documented emptiness rule; active-space re-rooting of the compile-frozen default-space paths. - inline_context_content / inline_context_omitted: persona + knowledge roster content, delivered once per agent per workflow (derived from state checkboxes like the D-E persona signal). - Total emitted-line byte budget (AIDLC_DIRECTIVE_MAX_BYTES, default 28KB) with an exact-budget backstop; overflow lands in the *_omitted lists - visible, never silent. Shipped Claude settings raise BASH_MAX_OUTPUT_LENGTH to 150000 and set the budget to 120000. - Reviewer knowledge (knowledge/<agent>/reviewing.md) absorbed into the reviewer agent bodies at build time on every harness (the reviewer set is derived from stage frontmatter); redundant Kiro resources globs removed. - Protocol/skill/docs updated: apply delivered content, read by path only what the directive names as omitted; dispatched briefs paste rules_content verbatim. - t248 unit coverage: substantive-filter, populated placeholder, missing file, budget partition, tight-budget overflow, deliver-once. Fixes #495
The mob support-brief assertion pinned the OLD contract (exact rule PATHS
in every brief). Under the steering-delivery change the conductor pastes
or faithfully summarizes the directive's rules_content into briefs - live
runs show both forms - so the pin now requires, per substantive seeded
rule file, a distinctive content token that survives paste AND summary
("first-class" for the org testing posture, "Given/When/Then" for the
inception guardrails) OR the exact path (the rules_content_omitted
fallback). Comment-only team.md/project.md placeholders are no longer
asserted. A brief that drops the rules entirely - the defect class the
change fixes - still fails.
Verified live: t238 solo PASS, stamp tests/logs/2026-07-25T01-38-44Z.
…ew's silent-downgrade gap Adversarial review (2 reviewers) findings applied: P1 - deliver-once suppression composed with budget overflow into a permanent silent steering gap: a roster file that overflowed to inline_context_omitted on its delivery stage was marked delivered and then filtered from BOTH lists on every later stage, reverting it to the discretionary path-read this change eliminates (reproduced at the 28KB default: the architect persona, omitted on intent-capture, vanished from feasibility's directive entirely). deliveredInlineContext is now a content-PRIORITY signal, not a visibility gate: delivered agents' files skip content but always appear in inline_context_omitted, so the roster partitions into content + omitted with nothing silently absent, and the conductor reads omitted entries not already in context. Also covers the review's P2 (recomposed rosters crediting past stages): an overcount now costs one visible re-read, never missing steering. P2 (contract reviewer) - three shipped surfaces still instructed paths-only briefs, contradicting the new paste-rules_content rule (conductor.md - baked into every workflow's first directive - the user-stories round-1 step, 15-stage-definition.md, and the topology paragraph in all five SKILL.md files). All rewritten to 'artifacts by path, rules as pasted rules_content'; zero 'paths-only' left outside dist. P3s - backstop comment now states the shed-content-only limit honestly; rulesContentEntries reads a marker-less (plugin-shaped) rule path from the project dir instead of building a garbage path; the directive CLI self-check validates a POPULATED rules_content/inline_context_content example, not just field absence. t248 grows the partition + budget-starved stage-2 visibility cases (the old deliver-once test pinned the buggy suppression; corrected). SKILL.md/stage-protocol/docs prose updated to the partition contract.
…th the claim-sources sensor Pre-existing reds on the v2 tip since the Intent Capture grounding merge (2.5.11) added the claim-sources sensor import to intent-capture: - t89: the five fixture sensor dirs that compile against the REAL stage tree lacked an aidlc-claim-sources.md manifest, so compileStageGraph threw 'imports unknown sensor id' on every case (13 assertions). Add the fixture manifest to those dirs; intent-capture count assertions go 2 -> 3 and the loadSensors key list gains claim-sources. - t66: regenerate tests/fixtures/designer-export/export.json from the live surface (intent-capture's claim-sources sensor row + reviewer fields, approval-handoff's stakeholder-map consume). Verified: t89 + t66 both green, stamp tests/logs/2026-07-25T03-51-05Z (111/111 assertions). Test-only - no version bump per changelog policy.
Close the conductor-to-worker hop deterministically: the engine-to-conductor hop delivers rules as bounded load-steering parts, and this makes the brief the conductor writes for a dispatched aidlc-*-agent carry the exact same bundle, no longer trusting the conductor to paste it. - core/tools/aidlc-steering.ts: shared resolver (substantive-rule filter, active-space re-rooting, fail-closed UTF-8 read) extracted from the engine so the dispatch hop and the engine hop cannot drift. - core/hooks/aidlc-dispatch-rules.ts: PreToolUse hook that resolves the active stage from the brief (stage-file path, unique slug mention, or the state file's Current Stage), and appends the hash-marked rule bundle when the brief lacks the exact content. Exit 2 with repair guidance when a required rule cannot be loaded. aidlc-composer-agent exempt. - Harness wiring: Claude settings PreToolUse Task|Agent matcher; Codex adapter dispatch-rules route (spawn_agent, updatedInput contract); opencode plugin rewrites task args in tool.execute.before; Kiro CLI cannot rewrite tool input, so its adapter validates and BLOCKS an incomplete subagent brief with verbatim-paste retry guidance; Kiro IDE keeps native memory-resource preload. - sdk-drive captures the post-rewrite executed prompt from Agent/Task tool results (allowed tools can bypass canUseTool), so live assertions see what the subagent actually received. - t238 seeds live knowledge evidence and asserts the exact-bundle contract; t248 grows dispatch-rewrite cases (Claude prompt rewrite, Codex items append, reviewer-checklist embedding per harness); hook-count and packaging pins updated for the new hook on every harness. Tiers on this tree: smoke+unit 178/178 (stamp 2026-07-25T11-35-56Z), typecheck + package --check + lint green.
… t53 Two test-contract stragglers from the load-steering migration (the unit twins were migrated in the redesign; these twins were not): - integration/t118: the WALK cases used createTestProject (deliberately empty memory) and a raw next spawn. Since the migration, next on a run-stage route fails closed when a required rule file is missing, and the routing directive arrives after the load-steering parts. Seed the shipped method tree (seedAidlcMemory) and route the six WALK next calls through runOrchestrateNext, matching the unit twin. - e2e/t53: stopAfterToolResult matched '"stage":"requirements-analysis"', which the load-steering part-1 result now also carries, stopping the drive one beat before run-stage. Pin the adjacent '"kind":"run-stage","stage":...' form instead. (Swept the other stop conditions: the jump/init/space tokens match strings load-steering never emits.) Verified: t118 15/15 (stamp 2026-07-25T12-33-57Z), t53 1/1 (stamp 2026-07-25T12-35-02Z). Test-only.
…etry deadlocked live The Kiro CLI dispatch-rules route blocked an incomplete subagent brief with verbatim-paste retry guidance. The live ACP gate showed the contract never converges: Kiro cannot rewrite tool input, the conductor cannot reproduce a multi-KB bundle byte-exactly, and every retry re-blocked (2-3 blocks per run, zero successful dispatches - both reviewer and journey files deadlocked mid-stage). Kiro is the one harness where the rules invariant already holds without the brief: every delegated agent's config preloads the full active memory tree via its resources glob before the brief is read. So the validator becomes an observer there - an incomplete brief proceeds with an advisory stderr warning; an unloadable REQUIRED rule file (real missing steering, no preload to fall back on) still blocks with the core hook's repair guidance. The strict updatedInput rewrite path on Claude, Codex, and opencode (live-verified green) is unchanged. Same harness-capability degradation pattern as the reviewer dispatch record and reviewer-scope enforcement (enforcement-capable harnesses enforce; Kiro documents). t147 5b pins the new contract; CHANGELOG and hooks reference updated. Verified: t147 17/17 (stamp 2026-07-25T19-43-17Z), typecheck + package --check green.
…erate recovered live fumbles Three follow-ups from the gate review, applied in-branch: - aidlc-dispatch-rules promptStage precedence: stage-file path, then the state file's Current Stage, then unique-slug-mention LAST (previously a single incidental mention of another stage's slug in brief prose outranked the live workflow's active stage and bound the wrong bundle). New t248 case pins the scenario (feasibility active, user-stories mentioned in passing -> ideation rules delivered). - The four Kiro reviewer agent JSONs no longer list their own .md in resources - the prompt field already loads it, and the absorbed reviewing checklist made the second copy ~4KB of duplicate context per reviewer dispatch. Reviewers only; the same pattern on the other twelve agents is untouched. - t-acp-kiro-reviewer's toolCallIssues pin rejects only SYSTEMATIC issues (hook blocks, repeated failures of one call): a transient argument fumble the live agent recovers from - observed on clean v2 with the stage still completing - is live noise, not the dispatch contract under test. Verified live green (stamp 2026-07-25T20-55-45Z). t248 24/24, t147 17/17, typecheck + package --check + lint green.
80cd938 to
17a4a82
Compare
Fixes #495.
Problem
The engine resolved each stage's rules into
rules_in_contextbut shipped only the file paths in the run-stage directive; whether the content ever reached the model depended on the conductor choosing to read each path. The issue documents live runs where 0/4 memory rules were read across three consecutive ideation stages - the directive "delivered" steering that silently never arrived, and the artifacts contradicted rules that should have shaped them. Two sibling surfaces had the same prose-only delivery: the reviewer agents'reviewing.mdchecklists, and the rules relayed into dispatched-subagent briefs.Fix - deterministic delivery on every hop
Engine to conductor.
nextnow emits the active stage's substantive rules as boundedload-steeringdirectives (ordered{path, text}content) beforerun-stage. Opaquecontinuetokens bind stage, workflow state, bundle hash, and next part; a freshnextrestarts at part 1. Every directive stays below the common 28 KiB output floor - oversized rules split at Markdown headings, then UTF-8 boundaries, with no size-based path fallback. A missing, unreadable, or invalid-UTF-8 required rule stops before stage work with repair guidance (fail closed). Rules re-read every stage, so a §13 learning admitted mid-workflow reaches the very next stage. Comment-only template placeholders (team.md/project.mdas shipped) are excluded until they carry an affirmed practice.Conductor to dispatched worker. A new PreToolUse hook (
aidlc-dispatch-rules.ts, backed by the shared resolveraidlc-steering.tsso the two hops cannot drift) appends the exact hash-marked rule bundle to every AI-DLC subagent brief that lacks it. Claude Code, Codex, and opencode rewrite the tool input (updatedInput); idempotent when the bundle is already present. Stage resolution precedence: explicit stage-file path in the brief, then the state file's Current Stage, then a unique slug mention.Kiro CLI degradation (deliberate). Kiro cannot rewrite tool input, and a block-with-retry contract deadlocked in live testing (the conductor cannot reproduce a multi-KB bundle byte-exactly; every retry re-blocked). Kiro is also the one harness where the invariant already holds without the brief: every delegated agent preloads the full active memory tree natively via its
resourcesglob. So on Kiro an incomplete brief proceeds with an advisory warning; an unloadable required rule (real missing steering, no preload fallback) still blocks with repair guidance. Kiro IDE keeps native memory-resource preload. This mirrors the existing harness-capability pattern (reviewer dispatch record, reviewer-scope enforcement).Reviewer checklists.
knowledge/<agent>/reviewing.mdis absorbed into the two reviewer agents' generated bodies at build time on every harness (the reviewer set is derived from stage frontmatter, not hardcoded). The redundant per-agent knowledge globs and self-.mdresourcesentries were removed from the Kiro reviewer JSONs - the checklist now arrives exactly once.Deliberate non-goal
Persona and supplemental knowledge files remain path-loaded (with actionable
context_warningsfor missing/unreadable optional files) pending a retrieval system - wholesale content injection of ~130KB knowledge trees is the wrong shape. This is the issue's "Additional Context" sibling surface, scoped out intentionally; follow-up issue to come.Verification
/tmppackage roots - environmental).t113,t-tui-kiro-bugfix,t-tui-t51/t73,t-acp-kiro-journey; these fail without this PR too). Kiro IDE slice skips clean on Linux (needs macOS + Kiro.app) - unverified there, low risk (no dispatch hook registers on the IDE).claim-sourcessensor (pre-existing tip reds folded in per review).Notes for reviewers
BASH_MAX_OUTPUT_LENGTH=150000+AIDLC_DIRECTIVE_MAX_BYTES=120000ship in the Claude settings so the directive line survives the tool-output cap; the engine default stays 28 KiB for the other harnesses.