diff --git a/docs/api-reference.md b/docs/api-reference.md index 9235b8de..709dfd7d 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -209,9 +209,11 @@ An activity references techniques via a flat `techniques[]` array, and a workflo Full delivery on every call is the default: in disposable-worker topologies each `get_activity` lands in a fresh agent context, and the repeated bundle is load-bearing. A session in which a **single agent context** executes the whole walk can opt into reference delivery instead — content the session+agent has already received is replaced by a short marker rather than repeated. - **Opt-in:** `start_session { context_mode: "persistent" }` (also accepted by `dispatch_child` for the child session), or per call via `get_activity { bundle: "reference" }`. The per-call opt-in exists only on `get_activity`; `get_technique` delta mode follows the session's `context_mode`. -- **Ledger:** the server records a per-agent hash of every `get_activity` bundle technique, rules block, `activity_rules` block, and full `get_technique` payload it delivers (in `session.json#deliveredContent`, keyed by `agentId`). Recording happens in every mode, so a per-call `bundle: "reference"` can refer back to content delivered under the default full mode. `get_workflow`'s orchestrator bundle is outside the ledger and always delivered in full. -- **`get_activity` reference mode:** the response carries `bundle_mode: reference` plus a `bundle_note`; each bundle technique whose composed content is byte-identical to a prior delivery collapses to `{ delivery: "unchanged", content_hash }`, as do the `rules` and `activity_rules` blocks. Techniques new to the activity (or whose content changed) arrive in full. The activity body itself is always delivered. +- **Ledger:** the server records a per-agent hash of every `get_activity` bundle technique, rules block, `activity_rules` block, full `get_technique` payload, and `get_workflow` orchestrator ops bundle it delivers (in `session.json#deliveredContent`, keyed by `agentId`). Recording happens in every mode, so a per-call `bundle: "reference"` can refer back to content delivered under the default full mode. Content keys are namespaced by delivery channel (`bundle:*`, `technique:*`, `activity_rules:*`, `workflow_bundle:*`) so a marker only ever references content delivered in that same channel. +- **`get_workflow` ops-bundle slimming:** under `context_mode: "persistent"`, the orchestrator ops bundle (above the `---` separator) is content-keyed under `workflow_bundle:`; on a resume where this agent already holds it, the whole bundle collapses to a single `{ delivery: "unchanged", content_hash }` marker while the post-separator workflow summary stays full. Fresh/default sessions always receive the ops bundle in full. +- **`get_activity` reference mode:** the response carries `bundle_mode: reference` plus a `bundle_note`; each bundle technique whose composed content is byte-identical to a prior delivery collapses to `{ delivery: "unchanged", content_hash }`, as do the `rules` and `activity_rules` blocks. Techniques new to the activity (or whose content changed) arrive in full, and within a full eagerly-inlined `step_techniques` entry the shared contract/rules blocks may themselves be markers (see Block-level delivery below). The activity body itself is always delivered. - **`get_technique` delta mode:** active when the session's `context_mode` is `"persistent"`; a byte-identical refetch returns `delivery: unchanged` + `content_hash` instead of the composed technique. Step-bound provenance annotations (`source:`/`destination:`) are part of the composed content; they are static per corpus and step, so a same-step refetch stays byte-identical and collapses, while fetching the same op from a *different* step (different binding context) re-delivers in full rather than answering with a stale reference. +- **Block-level delivery:** finer-grained than the whole-technique collapse above. A not-yet-seen technique's shared blocks — the contract-inherited `inherited_inputs` / `inherited_outputs` and the merged `rules` — are content-keyed individually (`technique::`). When such a block was already delivered by a sibling technique that shares the workflow contract, it collapses to a `{ delivery: "unchanged", content_hash }` marker at its position in the payload while the technique-specific core stays full. This applies on both the `get_technique` full-delivery path and each eagerly-inlined `get_activity` `step_techniques` entry. Content-keying keeps it stale-free: a block annotated with binding-seam provenance hashes differently and correctly delivers in full. `get_technique { full: true }` / `get_activity { bundle: "full" }` re-deliver every block. - **Full-content escapes:** `get_activity { bundle: "full" }` and `get_technique { full: true }` force full delivery — use them when the calling context no longer holds the earlier payload (e.g. it was summarized away). - **Agent scoping:** the ledger is keyed by the session's recorded `agentId`; resuming a session under a different `agent_id` starts that agent from an empty ledger, so its first deliveries are full even in reference mode. @@ -221,7 +223,7 @@ Eager step-technique bundling is **automatic and corpus-wide**: `get_activity` i - **Budget:** the eager-delivery budget is a **cumulative per-activity character budget** = `context_tokens × headroomFraction × charsPerToken`, where `headroomFraction` (default 0.80) and `charsPerToken` (default 4) are server config (env-overridable: `BUNDLE_HEADROOM_FRACTION`, `BUNDLE_CHARS_PER_TOKEN`). The budget accounts for **technique body sizes only** — a technique's `resources[]` references count, but not the resolved content of those resources (resources are never inlined; see below). Ungated technique steps are inlined in document order until adding the next would overflow the budget; the remainder stay lazy. - **What is bundled:** each ungated technique-kind step, in document order, until the cumulative budget is exhausted. A `when`/`condition` on the step itself or on an enclosing loop keeps it lazy, so bundling never delivers content for a step that may not execute. A per-activity `bundleTechniques: { maxChars: }` is an explicit **per-technique size cap** layered on the budget (any single technique larger than `maxChars` stays lazy); `maxChars: 0` opts the activity out of eager bundling entirely. Everything not inlined remains a `get_technique { step_id }` fetch. -- **Resources stay lazy:** bundling inlines step **techniques only**, never their referenced resources. An inlined entry carries the technique's `resources[]` references exactly as a `get_technique` fetch does, but the worker still calls `get_resource { resource_id }` on demand for each one. Resources are frequently shared across techniques, so inlining them per-technique would duplicate content; dedup-aware shared-resource delivery is deferred to the block-level delivery-ledger work (C2). Leaving `get_resource` lazy also preserves it as a purposeful call. +- **Resources stay lazy:** bundling inlines step **techniques only**, never their referenced resources. An inlined entry carries the technique's `resources[]` references exactly as a `get_technique` fetch does, but the worker still calls `get_resource { resource_id }` on demand for each one. Resources are frequently shared across techniques, so inlining them per-technique would duplicate content. Leaving `get_resource` lazy also preserves it as a purposeful call. - **Response shape:** bundled entries arrive in the ops section under `step_techniques`, keyed by step id. Each entry leads with a discrete `▼ STEP · technique ` arrival marker, then the step's full composition — activity-group resolution, ancestor contract, inherited-input blocks, and binding-seam provenance annotations — identical to what the step-bound `get_technique` fetch returns. A `step_techniques_note` prescribes the deliberate in-order engagement (below) and states that steps absent from the map still require a fetch, and `_meta.bundled_steps` lists the bundled step ids. - **Intentional stepping:** the *act* of calling `get_technique` is a deliberate "I now turn to this step" beat. Inlining removes that call, so the worker substitutes for it: process inlined `step_techniques` strictly in step order and, on reaching each step, EMIT a one-line `▶ step ` begin-beat before executing it. That emitted beat carries the intentional act and **is** the stepwise observability trace for bundled steps — the worker does NOT ping the server per bundled step; the delivery-time `technique_bundled` events already record coverage. - **Ledger interplay:** bundled entries share the `technique:` ledger key with `get_technique`, so in a persistent-context session a bundled delivery collapses a later step-bound refetch to an unchanged-reference — and reference-mode re-delivery of the activity collapses already-delivered bundled entries to `{ delivery: "unchanged", content_hash }` markers (the `▼ STEP` marker rides along). `bundle: "full"` re-delivers everything. diff --git a/site/api/tools.html b/site/api/tools.html index d6956b4a..22406e7a 100644 --- a/site/api/tools.html +++ b/site/api/tools.html @@ -364,7 +364,7 @@

Techniques and resources

Fetch technique definitions and lazy-loaded reference material.

get_technique

-

Load a single composed technique within the current workflow or activity.

+

Load one fully composed technique.

Full description

Fetches one technique for the current workflow or activity.

diff --git a/src/tools/resource-tools.ts b/src/tools/resource-tools.ts index 0956e184..ea028818 100644 --- a/src/tools/resource-tools.ts +++ b/src/tools/resource-tools.ts @@ -5,7 +5,7 @@ import { withAuditLog } from '../logging.js'; import { loadWorkflow, loadWorkflowWithDiagnostics, getActivity } from '../loaders/workflow-loader.js'; import { readResourceStructured } from '../loaders/resource-loader.js'; -import { composeActivityTechnique, projectTechniqueToYaml } from '../loaders/technique-loader.js'; +import { composeActivityTechnique, projectTechnique } from '../loaders/technique-loader.js'; import { sessionIndexParam, assertNoActiveCheckpoint, @@ -44,7 +44,7 @@ import { buildProvenanceContext, decorateTechniqueProvenance } from '../utils/bi import { seedDefaults } from '../utils/variable-seed.js'; import { buildValidation, validateWorkflowVersion } from '../utils/validation.js'; import { stringifyForResponse } from '../utils/serialization.js'; -import { contentHash, deliveredHash, recordDeliveries, unchangedMarker } from '../utils/delivery.js'; +import { contentHash, deliveredHash, dedupTechniqueBlocks, recordDeliveries, unchangedMarker } from '../utils/delivery.js'; import { createTraceEvent } from '../trace.js'; import { randomUUID } from 'node:crypto'; import { basename, isAbsolute, resolve } from 'node:path'; @@ -506,7 +506,7 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): server.tool( 'get_technique', - 'Load a single composed technique within the current workflow or activity. If called before next_activity (no current activity), it loads the workflow\'s first declared technique. During an activity, it resolves the technique reference from the activity definition; with step_id, it loads the technique assigned to that step; without step_id, the activity\'s first declared technique. The returned technique is fully COMPOSED: it inherits its workflow-root `techniques/TECHNIQUE.md` base contract recursively — never the meta root for a non-meta workflow. Contract-inherited inputs/outputs are delivered under marked `inherited_inputs`/`inherited_outputs` blocks (each with a scope note) distinct from the technique\'s own `inputs`/`outputs`; rules are merged; ancestor Initial/Final protocol blocks wrap the descendant protocol and the server renumbers. A step-bound fetch also annotates the binding seam: each of the technique\'s own inputs carries a `source:` (step-binding value, workflow variable, prior step output, declared default, or UNRESOLVED — the latter also a warn-only validation warning), inherited entries carry one only where it adds to their scope note (step-binding override or a producer positioned later in the workflow), each remapped output carries a `destination:`, and a `provenance_note` states the output delivery mechanics. Annotations are static — resolved from declarations and document order — so payloads are deterministic per corpus and step. Techniques are loaded one at a time. In a session with context_mode "persistent", a refetch whose composed content is byte-identical to what this session+agent already received returns a short unchanged-reference ({ delivery: unchanged, content_hash }) instead of the full payload; pass full: true to force full content. Every fetch (either delivery path) is recorded in the session history as a technique_fetched event (resolved technique id, bound step_id when supplied, agent); next_activity\'s manifest validation reads these events and warns — advisory only — when a manifested technique step had no fetch during the activity.', + 'Load one fully composed technique. With step_id, loads the technique bound to that step; without it, the activity\'s (or, before next_activity, the workflow\'s) first technique. The result inherits its workflow-root `techniques/TECHNIQUE.md` contract recursively (never the meta root for a non-meta workflow): contract inputs/outputs arrive as marked `inherited_inputs`/`inherited_outputs` blocks distinct from the technique\'s own, rules are merged, and ancestor Initial/Final protocol blocks wrap and renumber the protocol. A step-bound fetch annotates the binding seam — own inputs carry a `source:`, remapped outputs a `destination:`, plus a `provenance_note`; annotations are static per corpus and step. Under context_mode "persistent", a byte-identical refetch returns a short unchanged-reference ({ delivery: unchanged, content_hash }) instead of the payload, and on a new technique any shared inherited_inputs/inherited_outputs/rules block already delivered by a sibling is replaced with that marker while the core stays full; full: true forces full content (every block). Every fetch is recorded as a technique_fetched event that next_activity\'s manifest validation reads (advisory).', { ...sessionIndexParam, step_id: z.string().optional().describe('Optional. Step ID within the current activity (e.g., "define-problem"). If omitted, returns the activity\'s first declared technique, or the workflow\'s first declared technique if no activity is active.'), @@ -602,7 +602,10 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): provenanceWarnings.push(...decorated.warnings); } } - const text = projectTechniqueToYaml(technique); + // Hash the whole technique over the pre-marker projected text so the whole-marker + // branch below still collapses an identical refetch; block dedup runs later. + const ordered = projectTechnique(technique); + const text = stringifyForResponse(ordered); const view = sessionView(state); const validation = buildValidation( @@ -658,15 +661,24 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): }; } + // Full-delivery branch. Under reference delivery, collapse any shared contract/rules + // block already delivered by a sibling technique to a marker while the core stays full; + // block hashes are recorded alongside the whole-technique key. + let body = text; + const blockDeliveries: Record = {}; + if (state.contextMode === 'persistent' && full !== true) { + const deduped = dedupTechniqueBlocks(ordered, state, blockDeliveries); + body = stringifyForResponse(deduped); + } const next = advanceSession(state, (draft) => { draft.currentTechnique = techniqueId as string; - recordDeliveries(draft, state.agentId, { [ledgerKey]: hash }); + recordDeliveries(draft, state.agentId, { [ledgerKey]: hash, ...blockDeliveries }); recordFetch(draft); }); await saveSessionForTool(loaded, next); return { - content: [{ type: 'text' as const, text: `session_index: ${session_index}\n\n${text}` }], + content: [{ type: 'text' as const, text: `session_index: ${session_index}\n\n${body}` }], _meta: { session_index, validation }, }; }), traceOpts) diff --git a/src/tools/workflow-tools.ts b/src/tools/workflow-tools.ts index 3298c8ef..c1e235f7 100644 --- a/src/tools/workflow-tools.ts +++ b/src/tools/workflow-tools.ts @@ -12,7 +12,7 @@ import { buildProvenanceContext, decorateTechniqueProvenance } from '../utils/bi import { withAuditLog, logWarn } from '../logging.js'; import { jsonTypeOf, isTemplateReference } from '../utils/variable-seed.js'; import { stringifyForResponse } from '../utils/serialization.js'; -import { contentHash, deliveredHash, recordDeliveries, unchangedMarker } from '../utils/delivery.js'; +import { contentHash, deliveredHash, dedupTechniqueBlocks, recordDeliveries, unchangedMarker } from '../utils/delivery.js'; import { sessionIndexParam, contextTokensParam, @@ -295,17 +295,38 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): validateWorkflowVersion(view, wf), ); - // Advance state (bump seq+ts) and persist before returning. - const next = advanceSession(state); - await saveSessionForTool(loaded, next); - // Bundle the workflow's orchestrator-level technique refs (`techniques.workflow`) and the core // orchestrator techniques. Deduplicate by ref so a workflow that explicitly lists a core // technique resolves it once. const wfTechRefs = (wf as { techniques?: { workflow?: string[] } }).techniques?.workflow ?? []; const orchestratorTechniques = Array.from(new Set([...wfTechRefs, ...CORE_ORCHESTRATOR_TECHNIQUES])); const resolvedOrchestrator = await resolveTechniques(orchestratorTechniques, config.workflowDir, workflow_id); - const opsBlock = stringifyForResponse(formatTechniqueBundle(resolvedOrchestrator)); + const opsText = stringifyForResponse(formatTechniqueBundle(resolvedOrchestrator)); + + // Reference-not-repeat for the orchestrator ops bundle: under persistent mode, once this + // agent has received it in full, collapse the rebuilt bundle to a content-keyed + // `workflow_bundle:` marker. Decided before advanceSession so the new ledger key + // commits in the same mutator; fresh/default sessions pay nothing. + let opsBlock = opsText; + const workflowBundleDeliveries: Record = {}; + if (state.contextMode === 'persistent') { + const opsHash = contentHash(opsText); + const opsKey = `workflow_bundle:${opsHash}`; + if (deliveredHash(state, opsKey) === opsHash) { + opsBlock = stringifyForResponse({ + ...unchangedMarker(opsHash), + note: 'Orchestrator ops bundle unchanged from an earlier delivery this session — reuse it from your context.', + }); + } else { + workflowBundleDeliveries[opsKey] = opsHash; + } + } + + // Advance state and commit any new workflow-bundle ledger entry in one mutator. + const next = advanceSession(state, (draft) => { + recordDeliveries(draft, state.agentId, workflowBundleDeliveries); + }); + await saveSessionForTool(loaded, next); // Pre-separator preamble holds the resolved-operations bundle. Tests and clients split on // the first '\n\n---\n\n' to recover the workflow section, so we keep that single separator. @@ -495,7 +516,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): }; }), traceOpts)); - server.tool('get_activity', 'Load the complete activity definition for the current activity in the session. This is the worker\'s tool for retrieving the full activity details after the orchestrator has called next_activity to transition. Returns the complete activity definition including all steps, checkpoints, transitions to subsequent activities, rules, and technique references — everything needed to execute the activity. The activity is determined from the session state on disk, so no activity_id parameter is needed. context_tokens is REQUIRED — the worker declares its own context window; the server derives an eager step-technique bundling budget from it (availability headroom × a token→char factor, both server config) and inlines the activity\'s ungated step-bound techniques that fit, in document order, under that budget. When reference delivery is active (session context_mode "persistent" or bundle: "reference"), inherited techniques/rules content already delivered to this session+agent arrives as short { delivery: "unchanged", content_hash } markers instead of being repeated; bundle: "full" forces full delivery. Eligible ungated step-bound techniques are inlined corpus-wide under a step_techniques map (keyed by step id, each entry a discrete ▼ STEP block identical to a get_technique { step_id } fetch — engage those steps in order without refetching); gated steps and any technique that would overflow the derived budget (or a per-activity bundleTechniques.maxChars size cap; maxChars 0 opts the activity out) stay lazy and still require get_technique. Bundled deliveries are recorded as technique_bundled history events and count as fetch coverage for next_activity\'s manifest fidelity check.', + server.tool('get_activity', 'Load the current activity\'s full definition (steps, checkpoints, transitions, rules, technique references) — everything needed to execute it. The activity is read from session state, so no activity_id is needed. context_tokens is REQUIRED: the worker declares its context window and the server derives an eager bundling budget (headroom × token→char factor, server config), inlining the activity\'s ungated step-bound techniques that fit, in document order. Eligible techniques arrive corpus-wide under a step_techniques map (keyed by step id, each a discrete ▼ STEP block identical to a get_technique { step_id } fetch — engage them in order without refetching); gated steps and any technique overflowing the budget or a per-activity bundleTechniques.maxChars cap (maxChars 0 opts out) stay lazy. Under reference delivery (context_mode "persistent" or bundle: "reference"), already-delivered inherited techniques/rules collapse to { delivery: "unchanged", content_hash } markers, including block-by-block within an otherwise-full step technique (its shared inherited_inputs/inherited_outputs/rules); bundle: "full" forces full delivery. Bundled deliveries are recorded as technique_bundled events counting as fetch coverage for next_activity\'s manifest check.', { ...sessionIndexParam, ...contextTokensParam, @@ -656,9 +677,8 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): // Budget accounting measures the TECHNIQUE BODY only (including its resources[] refs, // but NOT the resolved content of those resources). Inlining bundles techniques, never // their referenced resources: the worker still calls get_resource on demand for each - // ref, exactly as for a lazy get_technique fetch. Shared-resource inlining (dedup-aware) - // is deferred to the C2 block-level-delivery-ledger cluster; inlining resources here - // without that ledger would duplicate content shared across techniques. + // ref, exactly as for a lazy get_technique fetch. Inlining resources here would duplicate + // content shared across techniques, so resources stay lazy. const text = projectTechniqueToYaml(technique); // Per-technique size cap: an oversized single technique is skipped outright. if (text.length > perTechniqueCap) continue; @@ -682,7 +702,12 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): newDeliveries[ledgerKey] = hash; // The arrival marker leads the block; the composed technique fields follow at the same // level, so a bundled entry reads exactly like a get_technique fetch with a step header. - bundledStepTechniques[step.id!] = { marker: stepMarker, ...projectTechnique(technique) }; + // Under reference mode, collapse any shared contract/rules block already delivered (by a + // sibling bundled step or an earlier fetch) to a marker while the core stays full. + const projected = referenceMode + ? dedupTechniqueBlocks(projectTechnique(technique), state, newDeliveries) + : projectTechnique(technique); + bundledStepTechniques[step.id!] = { marker: stepMarker, ...projected }; } bundledSteps.push({ stepId: step.id!, techniqueId }); bundlingWarnings.push(...provenanceWarnings); @@ -703,7 +728,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): const opsData = referenceMode ? { bundle_mode: 'reference', - bundle_note: 'Entries marked { delivery: "unchanged", content_hash } are byte-identical to content already delivered in this session for this agent — reuse them from your context. Re-fetch a step-bound technique with get_technique { step_id, full: true }; for inherited/core bundle entries and rules, call get_activity with bundle: "full" to re-deliver the whole bundle.', + bundle_note: 'Entries marked { delivery: "unchanged", content_hash } are already in your context — reuse them. A marker may also replace a single inherited_inputs/inherited_outputs/rules block inside an otherwise-full step_techniques entry (that shared block came from a sibling technique). Re-fetch a technique with get_technique { step_id, full: true }, or get_activity { bundle: "full" } to re-deliver the whole bundle.', ...bundleData, } : bundleData; diff --git a/src/utils/delivery.ts b/src/utils/delivery.ts index de846597..00a2ffc6 100644 --- a/src/utils/delivery.ts +++ b/src/utils/delivery.ts @@ -1,5 +1,6 @@ import { createHash } from 'node:crypto'; import type { SessionFile } from '../schema/session.schema.js'; +import { stringifyForResponse } from './serialization.js'; /** * Reference-not-repeat delivery. @@ -14,13 +15,18 @@ import type { SessionFile } from '../schema/session.schema.js'; * bundle path and by `get_technique`. Full content is always recoverable: * `get_activity { bundle: 'full' }`, `get_technique { full: true }`. * - * Content keys are namespaced by delivery channel so the two composition - * paths never cross-reference each other's payloads: - * - `bundle:` — one composed technique inside the - * `get_activity` techniques bundle - * - `bundle:rules` — the `get_activity` rules bundle - * - `activity_rules` — the inherited worker rules block - * - `technique:` — a full `get_technique` composed payload + * Content keys are namespaced by delivery channel so the composition paths + * never cross-reference each other's payloads: + * - `bundle:` — one composed technique in the `get_activity` bundle + * - `bundle:rules:` — the `get_activity` rules bundle + * - `activity_rules:` — the inherited worker rules block + * - `technique:` — a full `get_technique` composed payload + * - `technique::` — one shared block (`inherited_inputs` / + * `inherited_outputs` / `rules`) of a composed technique + * - `workflow_bundle:` — the `get_workflow` orchestrator ops bundle + * + * ``-suffixed keys are content-keyed — the key IS the content hash, so a + * changed payload gets a different key and delivers in full; no invalidation logic. */ /** Hash used for delivery-ledger comparison: sha256, truncated for payload brevity. */ @@ -52,3 +58,40 @@ export function recordDeliveries(draft: SessionFile, agentId: string, entries: R export function unchangedMarker(hash: string): { delivery: 'unchanged'; content_hash: string } { return { delivery: 'unchanged', content_hash: hash }; } + +/** + * Projected-technique keys eligible for block-level dedup — the contract-inherited + * blocks shared across a workflow's techniques. These mirror `projectTechnique`'s + * key strings, so renaming those keys must update this list. + */ +export const DEDUP_BLOCKS = ['inherited_inputs', 'inherited_outputs', 'rules'] as const; + +/** + * Replace already-delivered shared blocks of a projected technique record with + * unchanged-markers, hashing each block over the single-key projection the reader + * hashes so hashes match across techniques sharing a contract. Returns a shallow + * copy (input not mutated); newly-delivered hashes are staged into `newDeliveries` + * for the caller to commit, and a block already staged there collapses too. + * + * @param projected `projectTechnique` output. + * @param state session, for the delivery-ledger lookup. + * @param newDeliveries accumulator of block-hashes to record. + */ +export function dedupTechniqueBlocks( + projected: Record, + state: SessionFile, + newDeliveries: Record, +): Record { + const out = { ...projected }; + for (const block of DEDUP_BLOCKS) { + if (out[block] === undefined) continue; + const hash = contentHash(stringifyForResponse({ [block]: out[block] })); + const key = `technique:${block}:${hash}`; + if (deliveredHash(state, key) === hash || newDeliveries[key] === hash) { + out[block] = unchangedMarker(hash); + } else { + newDeliveries[key] = hash; + } + } + return out; +} diff --git a/tests/reference-delivery.test.ts b/tests/reference-delivery.test.ts index be9899a6..cd6621e2 100644 --- a/tests/reference-delivery.test.ts +++ b/tests/reference-delivery.test.ts @@ -713,4 +713,260 @@ describe('reference-not-repeat delivery (B1)', () => { } }); }); + + // Block-level delivery ledger. Finer-grained than the whole-technique dedup above: + // a NOT-yet-seen technique whose shared contract/rules blocks were already delivered + // (by a sibling technique or an earlier fetch) returns those blocks as markers while + // its technique-specific core stays full — the case whole-payload hashing cannot catch, + // since the core always changes the whole hash. + describe('block-level delivery ledger', () => { + // Parse a get_technique response body into its technique record (drops the + // `session_index:` header line before the first blank line). + function parseTechniqueBody(result: { content: Array<{ text: string }> }): Record { + const text = responseText(result as never); + return parse(text.substring(text.indexOf('\n\n') + 2)) as Record; + } + + // Two distinct technique-bound steps within an activity, discovered on a THROWAWAY + // fresh session so the probe's get_activity does not pollute the test session's + // ledger (eager bundling records whole-technique keys in every mode). These tests + // then fetch these steps directly via get_technique, never calling get_activity, so a + // block-marker (not a whole-technique marker) is exercised on the second fetch. + async function findTwoTechniqueStepIds(activityId: string): Promise<[string, string]> { + const probe = await startSession({ workflow_id: 'work-package', agent_id: 'probe' }); + const probeIdx = probe['session_index'] as string; + await enterActivity(probeIdx, activityId); + const parsed = splitActivityResponse(await getActivity(probeIdx, { bundle: 'full' })); + const body = parse(parsed.bodyText) as { steps?: Array<{ id?: string; technique?: unknown }> }; + const flat: Array<{ id?: string; technique?: unknown }> = []; + const walk = (steps?: Array<{ id?: string; technique?: unknown; steps?: unknown }>): void => { + for (const s of steps ?? []) { + if (Array.isArray((s as { steps?: unknown }).steps)) walk((s as { steps?: Array<{ id?: string; technique?: unknown }> }).steps); + else if (typeof s.technique === 'string' && s.id) flat.push(s); + } + }; + walk(body.steps as never); + const ids = flat.map(s => s.id!).filter((v, i, a) => a.indexOf(v) === i); + expect(ids.length, 'expected at least two technique-bound steps').toBeGreaterThanOrEqual(2); + return [ids[0], ids[1]]; + } + + it('collapses already-delivered contract/rules blocks to markers while the core stays full', async () => { + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder('2026-07-12-block-dedup-cross-technique'), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + const [stepA, stepB] = await findTwoTechniqueStepIds('implement'); + await enterActivity(idx, 'implement'); + + // Technique A (persistent, no prior get_activity) delivers in full and establishes + // the shared contract blocks in the ledger. + const first = await client.callTool({ + name: 'get_technique', + arguments: { session_index: idx, step_id: stepA }, + }); + expect(first.isError).toBeFalsy(); + const bodyA = parseTechniqueBody(first as never); + expect(bodyA['capability']).toBeDefined(); + // At least one shared block is present and delivered full (an object, not a marker). + const sharedBlocks = ['inherited_inputs', 'inherited_outputs', 'rules'] as const; + const presentInA = sharedBlocks.filter(b => bodyA[b] !== undefined); + expect(presentInA.length, 'technique A should carry at least one shared block').toBeGreaterThan(0); + for (const b of presentInA) expect(isUnchangedMarker(bodyA[b])).toBe(false); + + // Technique B (not yet seen): its OWN core is delivered full, but any shared block + // whose content matches one already delivered by A collapses to a marker. + const second = await client.callTool({ + name: 'get_technique', + arguments: { session_index: idx, step_id: stepB }, + }); + expect(second.isError).toBeFalsy(); + const bodyB = parseTechniqueBody(second as never); + // B is not the same technique as A — its core (capability) is delivered full. + expect(bodyB['capability']).toBeDefined(); + // The inherited contract is shared across a workflow's techniques, so at least one + // block collapses to a marker whose hash matches A's block projection. + const { stringify } = await import('yaml'); + const collapsed = sharedBlocks.filter(b => isUnchangedMarker(bodyB[b])); + expect(collapsed.length, 'expected at least one shared block to collapse for technique B').toBeGreaterThan(0); + for (const b of collapsed) { + const marker = bodyB[b] as UnchangedMarker; + expect(marker.content_hash).toBe(contentHash(stringify({ [b]: bodyA[b] }, { lineWidth: 0 }))); + } + }); + + it('full: true re-delivers every block full even when block-delivered', async () => { + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder('2026-07-12-block-dedup-full-escape'), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + const [stepA, stepB] = await findTwoTechniqueStepIds('implement'); + await enterActivity(idx, 'implement'); + + await client.callTool({ name: 'get_technique', arguments: { session_index: idx, step_id: stepA } }); + // B under reference delivery would collapse shared blocks; full: true forces full. + const forced = await client.callTool({ + name: 'get_technique', + arguments: { session_index: idx, step_id: stepB, full: true }, + }); + expect(forced.isError).toBeFalsy(); + const body = parseTechniqueBody(forced as never); + for (const b of ['inherited_inputs', 'inherited_outputs', 'rules'] as const) { + if (body[b] !== undefined) expect(isUnchangedMarker(body[b]), `expected full ${b} under full:true`).toBe(false); + } + }); + + it('fresh mode never markers blocks', async () => { + const session = await startSession({ workflow_id: 'work-package', agent_id: 'w1' }); + const idx = session['session_index'] as string; + const [stepA, stepB] = await findTwoTechniqueStepIds('implement'); + await enterActivity(idx, 'implement'); + + await client.callTool({ name: 'get_technique', arguments: { session_index: idx, step_id: stepA } }); + const second = await client.callTool({ name: 'get_technique', arguments: { session_index: idx, step_id: stepB } }); + const body = parseTechniqueBody(second as never); + for (const b of ['inherited_inputs', 'inherited_outputs', 'rules'] as const) { + if (body[b] !== undefined) expect(isUnchangedMarker(body[b]), `fresh mode must not marker ${b}`).toBe(false); + } + }); + + it('records block hashes under the technique:: channel', async () => { + const slug = '2026-07-12-block-dedup-ledger-keys'; + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder(slug), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + const [stepA] = await findTwoTechniqueStepIds('implement'); + await enterActivity(idx, 'implement'); + await client.callTool({ name: 'get_technique', arguments: { session_index: idx, step_id: stepA } }); + + const onDisk = JSON.parse(readFileSync(join(planningFolder(slug), 'session.json'), 'utf8')); + const keys = Object.keys(onDisk.deliveredContent.solo as Record); + expect(keys.some(k => /^technique:(inherited_inputs|inherited_outputs|rules):[0-9a-f]{16}$/.test(k))).toBe(true); + }); + + // The eager-bundle path is the second call site of dedupTechniqueBlocks (the cases above + // exercise get_technique). Within a single persistent get_activity, the bundled step + // techniques are projected in document order, so a later step whose shared contract/rules + // block matches one an earlier bundled step already delivered collapses to a marker while + // its own core stays full — all in one response. + it('collapses shared blocks inside get_activity eager step_techniques entries', async () => { + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder('2026-07-12-block-dedup-eager-bundle'), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + // Entry activity: no transition prerequisites, and it eager-bundles several + // technique steps that share the work-package contract. + await enterActivity(idx, 'start-work-package'); + + const { bundle } = splitActivityResponse(await getActivity(idx)); + expect(bundle['bundle_mode']).toBe('reference'); + const stq = bundle['step_techniques'] as Record>; + expect(stq, 'expected an eager step_techniques bundle').toBeDefined(); + expect(Object.keys(stq).length, 'need >=2 bundled steps to dedup across siblings').toBeGreaterThanOrEqual(2); + + const sharedBlocks = ['inherited_inputs', 'inherited_outputs', 'rules'] as const; + let fullBlockSeen = false; + let markerBlockSeen = false; + for (const entry of Object.values(stq)) { + // A whole-technique unchanged marker carries no block fields; skip it — we assert on + // otherwise-full entries whose individual blocks may be markered. + if (isUnchangedMarker(entry)) continue; + expect(entry['capability'], 'a full entry keeps its technique-specific core').toBeDefined(); + for (const b of sharedBlocks) { + if (entry[b] === undefined) continue; + if (isUnchangedMarker(entry[b])) markerBlockSeen = true; + else fullBlockSeen = true; + } + } + // The first bundled occurrence of each shared block is delivered full; a later sibling + // sharing that block collapses it to a marker. + expect(fullBlockSeen, 'at least one shared block delivered full').toBe(true); + expect(markerBlockSeen, 'at least one sibling shared block collapsed to a marker').toBe(true); + }); + }); + + // get_workflow orchestrator ops-bundle slimming. Under persistent mode the ops bundle + // (above the `---` separator) collapses to a single content-keyed workflow_bundle: + // marker on the second (resume) call; fresh mode always sends it full. + describe('get_workflow ops-bundle slimming', () => { + function splitWorkflowResponse(result: { content: Array<{ text: string }> }): { opsBlock: string; summary: Record } { + const text = responseText(result as never); + const sepIdx = text.indexOf('\n\n---\n\n'); + expect(sepIdx).toBeGreaterThan(0); + return { + opsBlock: text.substring(0, sepIdx), + summary: parse(text.substring(sepIdx + 7)) as Record, + }; + } + + it('collapses the ops bundle on a second persistent-mode call; summary stays full', async () => { + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder('2026-07-12-ops-bundle-slimming-persistent'), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + + const first = await client.callTool({ name: 'get_workflow', arguments: { session_index: idx } }); + expect(first.isError).toBeFalsy(); + const firstSplit = splitWorkflowResponse(first as never); + // First call: ops bundle delivered full (carries technique bodies, not a marker). + expect(firstSplit.opsBlock).toContain('capability:'); + expect(isUnchangedMarker(parse(firstSplit.opsBlock))).toBe(false); + + const second = await client.callTool({ name: 'get_workflow', arguments: { session_index: idx } }); + expect(second.isError).toBeFalsy(); + const secondSplit = splitWorkflowResponse(second as never); + // Second (resume) call: ops bundle collapses to the canonical marker; summary stays full. + const marker = parse(secondSplit.opsBlock) as Record; + expect(marker['delivery']).toBe('unchanged'); + expect(marker['content_hash']).toBe(contentHash(firstSplit.opsBlock)); + expect(marker['note']).toBeDefined(); + expect(secondSplit.summary['initialActivity']).toBeDefined(); + expect(secondSplit.summary['activities']).toBeDefined(); + }); + + it('never markers the ops bundle in fresh mode', async () => { + const session = await startSession({ workflow_id: 'work-package', agent_id: 'w1' }); + const idx = session['session_index'] as string; + + const first = await client.callTool({ name: 'get_workflow', arguments: { session_index: idx } }); + const second = await client.callTool({ name: 'get_workflow', arguments: { session_index: idx } }); + expect(first.isError).toBeFalsy(); + expect(second.isError).toBeFalsy(); + // Ops bundle repeats in full on every call — byte-identical, never a marker. + expect(splitWorkflowResponse(first as never).opsBlock).toContain('capability:'); + expect(splitWorkflowResponse(second as never).opsBlock).toBe(splitWorkflowResponse(first as never).opsBlock); + }); + + it('records the workflow_bundle: channel key on first persistent delivery', async () => { + const slug = '2026-07-12-ops-bundle-slimming-ledger-key'; + const session = await startSession({ + workflow_id: 'work-package', + agent_id: 'solo', + planning_folder: planningFolder(slug), + context_mode: 'persistent', + }); + const idx = session['session_index'] as string; + await client.callTool({ name: 'get_workflow', arguments: { session_index: idx } }); + + const onDisk = JSON.parse(readFileSync(join(planningFolder(slug), 'session.json'), 'utf8')); + const keys = Object.keys(onDisk.deliveredContent.solo as Record); + expect(keys.some(k => /^workflow_bundle:[0-9a-f]{16}$/.test(k))).toBe(true); + }); + }); });