feat: cluster 3 delivery ledger (block-level dedup + get_workflow slimming)

docs/api-reference.md

@@ -209,9 +209,11 @@ An activity references techniques via a flat `techniques[]` array, and a workflo
 Full delivery on every call is the default: in disposable-worker topologies each `get_activity` lands in a fresh agent context, and the repeated bundle is load-bearing. A session in which a **single agent context** executes the whole walk can opt into reference delivery instead — content the session+agent has already received is replaced by a short marker rather than repeated.
 
 - **Opt-in:** `start_session { context_mode: "persistent" }` (also accepted by `dispatch_child` for the child session), or per call via `get_activity { bundle: "reference" }`. The per-call opt-in exists only on `get_activity`; `get_technique` delta mode follows the session's `context_mode`.
-- **Ledger:** the server records a per-agent hash of every `get_activity` bundle technique, rules block, `activity_rules` block, and full `get_technique` payload it delivers (in `session.json#deliveredContent`, keyed by `agentId`). Recording happens in every mode, so a per-call `bundle: "reference"` can refer back to content delivered under the default full mode. `get_workflow`'s orchestrator bundle is outside the ledger and always delivered in full.
-- **`get_activity` reference mode:** the response carries `bundle_mode: reference` plus a `bundle_note`; each bundle technique whose composed content is byte-identical to a prior delivery collapses to `{ delivery: "unchanged", content_hash }`, as do the `rules` and `activity_rules` blocks. Techniques new to the activity (or whose content changed) arrive in full. The activity body itself is always delivered.
+- **Ledger:** the server records a per-agent hash of every `get_activity` bundle technique, rules block, `activity_rules` block, full `get_technique` payload, and `get_workflow` orchestrator ops bundle it delivers (in `session.json#deliveredContent`, keyed by `agentId`). Recording happens in every mode, so a per-call `bundle: "reference"` can refer back to content delivered under the default full mode. Content keys are namespaced by delivery channel (`bundle:*`, `technique:*`, `activity_rules:*`, `workflow_bundle:*`) so a marker only ever references content delivered in that same channel.
+- **`get_workflow` ops-bundle slimming:** under `context_mode: "persistent"`, the orchestrator ops bundle (above the `---` separator) is content-keyed under `workflow_bundle:<hash>`; on a resume where this agent already holds it, the whole bundle collapses to a single `{ delivery: "unchanged", content_hash }` marker while the post-separator workflow summary stays full. Fresh/default sessions always receive the ops bundle in full.
+- **`get_activity` reference mode:** the response carries `bundle_mode: reference` plus a `bundle_note`; each bundle technique whose composed content is byte-identical to a prior delivery collapses to `{ delivery: "unchanged", content_hash }`, as do the `rules` and `activity_rules` blocks. Techniques new to the activity (or whose content changed) arrive in full, and within a full eagerly-inlined `step_techniques` entry the shared contract/rules blocks may themselves be markers (see Block-level delivery below). The activity body itself is always delivered.
 - **`get_technique` delta mode:** active when the session's `context_mode` is `"persistent"`; a byte-identical refetch returns `delivery: unchanged` + `content_hash` instead of the composed technique. Step-bound provenance annotations (`source:`/`destination:`) are part of the composed content; they are static per corpus and step, so a same-step refetch stays byte-identical and collapses, while fetching the same op from a *different* step (different binding context) re-delivers in full rather than answering with a stale reference.
+- **Block-level delivery:** finer-grained than the whole-technique collapse above. A not-yet-seen technique's shared blocks — the contract-inherited `inherited_inputs` / `inherited_outputs` and the merged `rules` — are content-keyed individually (`technique:<block>:<hash>`). When such a block was already delivered by a sibling technique that shares the workflow contract, it collapses to a `{ delivery: "unchanged", content_hash }` marker at its position in the payload while the technique-specific core stays full. This applies on both the `get_technique` full-delivery path and each eagerly-inlined `get_activity` `step_techniques` entry. Content-keying keeps it stale-free: a block annotated with binding-seam provenance hashes differently and correctly delivers in full. `get_technique { full: true }` / `get_activity { bundle: "full" }` re-deliver every block.
 - **Full-content escapes:** `get_activity { bundle: "full" }` and `get_technique { full: true }` force full delivery — use them when the calling context no longer holds the earlier payload (e.g. it was summarized away).
 - **Agent scoping:** the ledger is keyed by the session's recorded `agentId`; resuming a session under a different `agent_id` starts that agent from an empty ledger, so its first deliveries are full even in reference mode.
 
@@ -221,7 +223,7 @@ Eager step-technique bundling is **automatic and corpus-wide**: `get_activity` i
 
 - **Budget:** the eager-delivery budget is a **cumulative per-activity character budget** = `context_tokens × headroomFraction × charsPerToken`, where `headroomFraction` (default 0.80) and `charsPerToken` (default 4) are server config (env-overridable: `BUNDLE_HEADROOM_FRACTION`, `BUNDLE_CHARS_PER_TOKEN`). The budget accounts for **technique body sizes only** — a technique's `resources[]` references count, but not the resolved content of those resources (resources are never inlined; see below). Ungated technique steps are inlined in document order until adding the next would overflow the budget; the remainder stay lazy.
 - **What is bundled:** each ungated technique-kind step, in document order, until the cumulative budget is exhausted. A `when`/`condition` on the step itself or on an enclosing loop keeps it lazy, so bundling never delivers content for a step that may not execute. A per-activity `bundleTechniques: { maxChars: <n> }` is an explicit **per-technique size cap** layered on the budget (any single technique larger than `maxChars` stays lazy); `maxChars: 0` opts the activity out of eager bundling entirely. Everything not inlined remains a `get_technique { step_id }` fetch.
-- **Resources stay lazy:** bundling inlines step **techniques only**, never their referenced resources. An inlined entry carries the technique's `resources[]` references exactly as a `get_technique` fetch does, but the worker still calls `get_resource { resource_id }` on demand for each one. Resources are frequently shared across techniques, so inlining them per-technique would duplicate content; dedup-aware shared-resource delivery is deferred to the block-level delivery-ledger work (C2). Leaving `get_resource` lazy also preserves it as a purposeful call.
+- **Resources stay lazy:** bundling inlines step **techniques only**, never their referenced resources. An inlined entry carries the technique's `resources[]` references exactly as a `get_technique` fetch does, but the worker still calls `get_resource { resource_id }` on demand for each one. Resources are frequently shared across techniques, so inlining them per-technique would duplicate content. Leaving `get_resource` lazy also preserves it as a purposeful call.
 - **Response shape:** bundled entries arrive in the ops section under `step_techniques`, keyed by step id. Each entry leads with a discrete `▼ STEP <step_id> · technique <name>` arrival marker, then the step's full composition — activity-group resolution, ancestor contract, inherited-input blocks, and binding-seam provenance annotations — identical to what the step-bound `get_technique` fetch returns. A `step_techniques_note` prescribes the deliberate in-order engagement (below) and states that steps absent from the map still require a fetch, and `_meta.bundled_steps` lists the bundled step ids.
 - **Intentional stepping:** the *act* of calling `get_technique` is a deliberate "I now turn to this step" beat. Inlining removes that call, so the worker substitutes for it: process inlined `step_techniques` strictly in step order and, on reaching each step, EMIT a one-line `▶ step <step_id>` begin-beat before executing it. That emitted beat carries the intentional act and **is** the stepwise observability trace for bundled steps — the worker does NOT ping the server per bundled step; the delivery-time `technique_bundled` events already record coverage.
 - **Ledger interplay:** bundled entries share the `technique:<resolvedId>` ledger key with `get_technique`, so in a persistent-context session a bundled delivery collapses a later step-bound refetch to an unchanged-reference — and reference-mode re-delivery of the activity collapses already-delivered bundled entries to `{ delivery: "unchanged", content_hash }` markers (the `▼ STEP` marker rides along). `bundle: "full"` re-delivers everything.

site/api/tools.html

@@ -364,7 +364,7 @@ <h2>Techniques and resources</h2>
       <p>Fetch technique definitions and lazy-loaded reference material.</p>
       <section class="tool" id="get_technique">
         <h3><code>get_technique</code></h3>
-        <p class="tool-summary">Load a single composed technique within the current workflow or activity.</p>
+        <p class="tool-summary">Load one fully composed technique.</p>
         <details class="tool-details">
           <summary>Full description</summary>
           <p>Fetches one technique for the current workflow or activity.</p>

src/tools/resource-tools.ts

@@ -5,7 +5,7 @@ import { withAuditLog } from '../logging.js';
 
 import { loadWorkflow, loadWorkflowWithDiagnostics, getActivity } from '../loaders/workflow-loader.js';
 import { readResourceStructured } from '../loaders/resource-loader.js';
-import { composeActivityTechnique, projectTechniqueToYaml } from '../loaders/technique-loader.js';
+import { composeActivityTechnique, projectTechnique } from '../loaders/technique-loader.js';
 import {
   sessionIndexParam,
   assertNoActiveCheckpoint,
@@ -44,7 +44,7 @@ import { buildProvenanceContext, decorateTechniqueProvenance } from '../utils/bi
 import { seedDefaults } from '../utils/variable-seed.js';
 import { buildValidation, validateWorkflowVersion } from '../utils/validation.js';
 import { stringifyForResponse } from '../utils/serialization.js';
-import { contentHash, deliveredHash, recordDeliveries, unchangedMarker } from '../utils/delivery.js';
+import { contentHash, deliveredHash, dedupTechniqueBlocks, recordDeliveries, unchangedMarker } from '../utils/delivery.js';
 import { createTraceEvent } from '../trace.js';
 import { randomUUID } from 'node:crypto';
 import { basename, isAbsolute, resolve } from 'node:path';
@@ -506,7 +506,7 @@ export function registerResourceTools(server: McpServer, config: ServerConfig):
 
   server.tool(
     'get_technique',
-    'Load a single composed technique within the current workflow or activity. If called before next_activity (no current activity), it loads the workflow\'s first declared technique. During an activity, it resolves the technique reference from the activity definition; with step_id, it loads the technique assigned to that step; without step_id, the activity\'s first declared technique. The returned technique is fully COMPOSED: it inherits its workflow-root `techniques/TECHNIQUE.md` base contract recursively — never the meta root for a non-meta workflow. Contract-inherited inputs/outputs are delivered under marked `inherited_inputs`/`inherited_outputs` blocks (each with a scope note) distinct from the technique\'s own `inputs`/`outputs`; rules are merged; ancestor Initial/Final protocol blocks wrap the descendant protocol and the server renumbers. A step-bound fetch also annotates the binding seam: each of the technique\'s own inputs carries a `source:` (step-binding value, workflow variable, prior step output, declared default, or UNRESOLVED — the latter also a warn-only validation warning), inherited entries carry one only where it adds to their scope note (step-binding override or a producer positioned later in the workflow), each remapped output carries a `destination:`, and a `provenance_note` states the output delivery mechanics. Annotations are static — resolved from declarations and document order — so payloads are deterministic per corpus and step. Techniques are loaded one at a time. In a session with context_mode "persistent", a refetch whose composed content is byte-identical to what this session+agent already received returns a short unchanged-reference ({ delivery: unchanged, content_hash }) instead of the full payload; pass full: true to force full content. Every fetch (either delivery path) is recorded in the session history as a technique_fetched event (resolved technique id, bound step_id when supplied, agent); next_activity\'s manifest validation reads these events and warns — advisory only — when a manifested technique step had no fetch during the activity.',
+    'Load one fully composed technique. With step_id, loads the technique bound to that step; without it, the activity\'s (or, before next_activity, the workflow\'s) first technique. The result inherits its workflow-root `techniques/TECHNIQUE.md` contract recursively (never the meta root for a non-meta workflow): contract inputs/outputs arrive as marked `inherited_inputs`/`inherited_outputs` blocks distinct from the technique\'s own, rules are merged, and ancestor Initial/Final protocol blocks wrap and renumber the protocol. A step-bound fetch annotates the binding seam — own inputs carry a `source:`, remapped outputs a `destination:`, plus a `provenance_note`; annotations are static per corpus and step. Under context_mode "persistent", a byte-identical refetch returns a short unchanged-reference ({ delivery: unchanged, content_hash }) instead of the payload, and on a new technique any shared inherited_inputs/inherited_outputs/rules block already delivered by a sibling is replaced with that marker while the core stays full; full: true forces full content (every block). Every fetch is recorded as a technique_fetched event that next_activity\'s manifest validation reads (advisory).',
     {
       ...sessionIndexParam,
       step_id: z.string().optional().describe('Optional. Step ID within the current activity (e.g., "define-problem"). If omitted, returns the activity\'s first declared technique, or the workflow\'s first declared technique if no activity is active.'),
@@ -602,7 +602,10 @@ export function registerResourceTools(server: McpServer, config: ServerConfig):
           provenanceWarnings.push(...decorated.warnings);
         }
       }
-      const text = projectTechniqueToYaml(technique);
+      // Hash the whole technique over the pre-marker projected text so the whole-marker
+      // branch below still collapses an identical refetch; block dedup runs later.
+      const ordered = projectTechnique(technique);
+      const text = stringifyForResponse(ordered);
 
       const view = sessionView(state);
       const validation = buildValidation(
@@ -658,15 +661,24 @@ export function registerResourceTools(server: McpServer, config: ServerConfig):
         };
       }
 
+      // Full-delivery branch. Under reference delivery, collapse any shared contract/rules
+      // block already delivered by a sibling technique to a marker while the core stays full;
+      // block hashes are recorded alongside the whole-technique key.
+      let body = text;
+      const blockDeliveries: Record<string, string> = {};
+      if (state.contextMode === 'persistent' && full !== true) {
+        const deduped = dedupTechniqueBlocks(ordered, state, blockDeliveries);
+        body = stringifyForResponse(deduped);
+      }
       const next = advanceSession(state, (draft) => {
         draft.currentTechnique = techniqueId as string;
-        recordDeliveries(draft, state.agentId, { [ledgerKey]: hash });
+        recordDeliveries(draft, state.agentId, { [ledgerKey]: hash, ...blockDeliveries });
         recordFetch(draft);
       });
       await saveSessionForTool(loaded, next);
 
       return {
-        content: [{ type: 'text' as const, text: `session_index: ${session_index}\n\n${text}` }],
+        content: [{ type: 'text' as const, text: `session_index: ${session_index}\n\n${body}` }],
         _meta: { session_index, validation },
       };
     }), traceOpts)