feat: clean-restart attempt isolation — CCRM retry-contamination fix (#972)

.github/workflows/test.yml

@@ -320,3 +320,30 @@ jobs:
 
       - name: Context Codec benchmark (offline gate)
         run: python benchmark/context-codec/run.py
+
+  retry-isolation-bench:
+    # Offline, seeded gate: clean-restart attempt isolation (#972) must
+    # show the IID overestimate of pass@K >= 8pp at the ~7.1x cascade
+    # ratio, clean restart must recover it, the closed-form allocation
+    # must match exactly, and the fence demo must quarantine cleanly.
+    # No network, no API key — imports the built artifact.
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09  # v5 (2026-07-16)
+
+      - name: Set up Python
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5 (2025-04-24)
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install pyyaml
+
+      - name: Verify build artifact is in sync
+        run: python scripts/build.py --check
+
+      - name: Retry-isolation benchmark (offline gate)
+        run: python benchmark/retry-isolation/run.py

CHANGELOG.md

@@ -13,6 +13,7 @@ else (installer, docs) is generated by `scripts/release.sh`.
 ## [Unreleased]
 
 ### Added
+- **Clean-restart attempt isolation — CCRM retry-contamination fix (#972).** `retry_isolation`: digest-sealed transactional context snapshots at attempt boundaries; on failure, restore the pre-attempt snapshot and inject a bounded, structured failure summary — the failed attempt's turns are quarantined and the builder fails closed if any leak back into the restored context. Contamination events are digest-sealed for observability. Attempt-budget allocation uses the paper's closed form T* = sqrt(B·log(1/(1-eps1))/log(1/(1-eps0))). Seeded CCRM simulation reproduces the paper's quantitative shape (IID overestimate of pass@3 ≥ 8pp at the ~7.1x cascade ratio; clean restart recovers it). Offline benchmark `benchmark/retry-isolation/` with a CI leg. Composes with #968 (fenced summaries feed TRACE as explicit dissatisfaction signals).
 - **Commitment-preserving verifiable compression — Context Codec borrow (#971).** `commitment_codec`: typed semantic atoms (goals, constraints, decisions, preferences, tool results, evidence, safety boundaries) are extracted into a registry with canonical identity, equivalence, and conflict relations before compaction; after compression their preservation is VERIFIED (Critical Atom Recall, Weighted Atom Recall, Commitment Density, round-trip recoverability; error taxonomy dropped/altered/conflated/safety_boundary_loss). Fail-closed: any uncertified commitment — or a crashing advisory lossy compressor — returns the original text. Safety boundaries are never compressed lossily. Digest-sealed replay-first reports. Offline benchmark `benchmark/context-codec/` gates CAR ≥ 0.99 and zero safety losses with the fail-closed path exercised on every session, with a CI leg.
 - **Pluggable submodular context-selection engine over the pooled context (#970).** `pooled_selection`: session turns, memory entries, and tool outputs pool into one candidate set at prompt-assembly time, selected by a monotone submodular objective (relevance + relevance-weighted coverage under a hard token budget, with diminishing returns) via deterministic lazy-greedy with stable tie-breaking. Pluggable policy registry (`submodular_greedy`, `relevance_greedy`, `recent_first` baseline, `register_policy` for future policies); digest-sealed replay-first selection traces with kept/dropped reasons feed the #962 DAG. Offline benchmark `benchmark/pooled-selection/` gates 100% kept-set recall at ≤ 50% budget on 12 multi-turn scenarios (recency baseline measures ~58% on the same corpus), with a CI leg.
 - **Trajectory-mined context-source failure attribution (#968).** `trace_attribution`: a deterministic, stdlib-only diagnosis layer that mines agent trajectories for implicit dissatisfaction signals (corrections, rephrasing, abandonment), attributes each failure to the defective context source with cited evidence steps and source spans, and classifies remediation as CREATE vs UPDATE before any patch is proposed (six-category fault taxonomy adapted to Perseus source types; TRACE arXiv:2608.09153 borrow). Fail-closed: inconclusive attributions produce no proposal; advisory reading agents confirm but never flip decisive verdicts. Digest-sealed, replay-first reports via `run_trace_analysis`/`verify_trace_report`. Offline benchmark `benchmark/trace/` gates attribution top-1 ≥ 70% and CREATE/UPDATE ≥ 90% on a 36-episode planted-fault corpus, with a CI leg mirroring the selection-eval gate.

README.md

@@ -274,7 +274,7 @@ Published as [`io.github.Perseus-Computing-LLC/perseus`](https://registry.modelc
 
 ### MCP Tools
 
-<!-- test-count: 2201 — recount with: grep -rE "^\s*def test_" tests/ | wc -l -->
+<!-- test-count: 2317 — recount with: grep -rE "^\s*def test_" tests/ | wc -l -->
 <!-- The table below documents the current default output of _get_all_mcp_tools({}). Recount before editing. -->
 MCP tools resolve live state at invocation time, including the canonical Perseus Vault tool. Two additional sensitive tools — `perseus_query` (run a shell command) and `perseus_agent` (execute a local agent subprocess) — are **not** part of this default set: they require explicit `mcp.tool_allowlist` opt-in because they execute commands in the user's local shell (**not sandboxed, full user permissions apply**).
 

benchmark/retry-isolation/README.md

@@ -0,0 +1,33 @@
+# Perseus CCRM retry-isolation benchmark (#972)
+
+A **reproducible, fully offline, seeded** evaluation of clean-restart
+attempt isolation: the IID retry model vs the contaminated cascade vs
+fenced clean restart at the paper's ~7.1x cascade ratio, plus the closed
+form attempt-budget allocation and a concrete fence demonstration.
+
+## Run it
+
+```bash
+python scripts/build.py                    # ensure perseus.py is in sync with src/
+python benchmark/retry-isolation/run.py    # score, write results/report, gate
+```
+
+Exit code is **non-zero** when a gate fails, so CI can block a regression:
+
+| gate | requirement |
+|---|---|
+| IID overestimate | IID pass@3 exceeds the contaminated cascade by ≥ 8 points (paper: 17.4pp on SWE-bench Verified) |
+| clean-restart dominance | fenced retries recover the gap (≥ 8 points over the contaminated cascade) |
+| closed-form allocation | T* matches `sqrt(B · log(1/(1−ε1)) / log(1/(1−ε0)))` exactly |
+| fence demonstration | a failed attempt's turns appear nowhere in the restored portion of the retry context; the contamination flag is set; the event re-verifies |
+
+## What this measures
+
+The CCRM model's quantitative claim, reproduced in seeded simulation:
+retry context is ~7.1x more error-prone per step, so replaying the
+contaminated trace costs double-digit pass@3 points versus the IID
+assumption — and clearing context before retry recovers it. This is the
+simulation analog of the paper's SWE-bench Verified experiment; the live
+SWE-bench-style workload stays out of this offline gate (the issue's
+success criterion calls for closing the gap toward the paper's prediction,
+which this harness pins structurally).

benchmark/retry-isolation/report.md

@@ -0,0 +1,16 @@
+# CCRM retry-isolation benchmark — results (#972)
+
+- simulation: 4000 seeded trials, cascade ratio 7.1x (paper: ~7.1x)
+- gate: **PASS**
+
+| policy | pass@3 |
+|---|---|
+| IID (overestimate) | 99.5% |
+| contaminated cascade | 88.8% |
+| clean restart (fenced) | 99.5% |
+
+- IID overestimate: **10.63pp** (paper: 17.4pp on SWE-bench Verified; gate ≥ 8pp)
+- clean-restart recovery: **10.63pp** (gate ≥ 8pp)
+- allocation: T* = 87.853 → 16 attempts from budget 1000.0
+- fence demo: 3 turns quarantined, summary 58 tokens, event digest `15e281f26f598d21…`
+

benchmark/retry-isolation/results.json

@@ -0,0 +1,37 @@
+{
+  "schema_version": "perseus-retry-isolation-benchmark-results/v1",
+  "pass": true,
+  "errors": [],
+  "simulation": {
+    "trials": 4000,
+    "eps0": 0.025,
+    "eps1": 0.1775,
+    "cascade_ratio": 7.1,
+    "iid_pass_at_3": 0.9948,
+    "contaminated_pass_at_3": 0.8885,
+    "clean_restart_pass_at_3": 0.9948,
+    "iid_overestimate_pp": 10.63,
+    "clean_restart_recovery_pp": 10.63
+  },
+  "allocation": {
+    "schema_version": "perseus-attempt-allocation/v1",
+    "total_budget": 1000.0,
+    "eps0": 0.025,
+    "eps1": 0.1775,
+    "cascade_ratio": 7.1,
+    "t_star_continuous": 87.853,
+    "optimal_attempts": 16,
+    "per_attempt_budget": 62.5,
+    "derivation": {
+      "log0": 0.025318,
+      "log1": 0.195407,
+      "formula": "T* = sqrt(B * log(1/(1-eps1)) / log(1/(1-eps0)))"
+    }
+  },
+  "fence": {
+    "quarantined_turns": 3,
+    "summary_tokens": 58,
+    "retry_tokens": 82,
+    "event_digest": "15e281f26f598d21732dd9bc87c41c4fea4098ca3d50f6559f0697245c54d52d"
+  }
+}

benchmark/retry-isolation/run.py

@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+"""CCRM retry-isolation benchmark (#972).
+
+Offline, seeded evaluation of clean-restart attempt isolation:
+
+1. **CCRM simulation** — the IID model vs the contaminated cascade vs
+   clean-restart fencing, at the paper's ~7.1x cascade ratio. Gates: the
+   IID overestimate of pass@K is >= 8 points (paper: 17.4 on SWE-bench
+   Verified) and clean restart recovers it (clean-restart dominance).
+2. **Closed-form allocation** — the attempt-budget helper must match
+   ``T* = sqrt(B * log(1/(1-eps1)) / log(1/(1-eps0)))`` exactly.
+3. **Fence demonstration** — a concrete failed attempt with contaminated
+   turns: the retry context restores the pre-attempt snapshot, carries the
+   bounded summary, and contains NONE of the failed attempt's turns in its
+   restored portion (verifiable in the context trace).
+
+Exit code is non-zero when any gate fails, so CI can block a regression.
+No network, no API key, no LLM.
+
+Usage:
+    python benchmark/retry-isolation/run.py            # score, write, gate
+    python benchmark/retry-isolation/run.py --trials 8000
+"""
+import argparse
+import hashlib
+import importlib.util
+import json
+import math
+import sys
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+REPO = HERE.parent.parent
+
+
+def load_perseus():
+    artifact = REPO / "perseus.py"
+    if not artifact.is_file():
+        sys.exit("error: perseus.py not found. Build it (`python scripts/build.py`).")
+    spec = importlib.util.spec_from_file_location("perseus", artifact)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description="CCRM retry-isolation benchmark (#972)")
+    ap.add_argument("--trials", type=int, default=4000)
+    ap.add_argument("--out-results", default=str(HERE / "results.json"))
+    ap.add_argument("--out-report", default=str(HERE / "report.md"))
+    args = ap.parse_args()
+
+    perseus = load_perseus()
+    errors: list[str] = []
+
+    # ── 1. CCRM simulation at the paper's ~7.1x cascade ratio ──
+    total_budget, eps0, eps1 = 1000.0, 0.025, 0.1775
+    report = perseus.run_ccrm_analysis(
+        total_budget=total_budget, eps0=eps0, eps1=eps1,
+        trials=args.trials, seed=42, created_by="benchmark")
+    over_pp = report["iid_overestimate_pp"]
+    recovery_pp = report["clean_restart_recovery_pp"]
+    if over_pp < 8.0:
+        errors.append(f"IID overestimate only {over_pp}pp (gate >= 8; "
+                      f"paper: 17.4pp)")
+    if recovery_pp < 8.0:
+        errors.append(f"clean-restart recovery only {recovery_pp}pp "
+                      f"(gate >= 8)")
+
+    # ── 2. Closed-form allocation exactness ──
+    alloc = perseus.attempt_budget_allocation(total_budget, eps0, eps1)
+    log0 = math.log(1 / (1 - eps0))
+    log1 = math.log(1 / (1 - eps1))
+    expected_t = math.sqrt(total_budget * log1 / log0)
+    if alloc["t_star_continuous"] != round(expected_t, 4):
+        errors.append("allocation deviates from the closed form")
+
+    # ── 3. Fence demonstration ──
+    base = ("You are a deployment assistant.\n\n"
+            "Goal: ship the release.\n\n"
+            "The deploy tool takes --env staging.")
+    snap = perseus.snapshot_context(base, attempt_id="bench-attempt-0")
+    turns = [
+        "deploy --env prod",
+        "error: permission denied for environment prod",
+        "trying prod credentials instead",
+    ]
+    event = perseus.build_retry_context(
+        snap,
+        {"attempt_id": "bench-attempt-1", "failed_step": "deploy --env prod",
+         "observed_error": "error: permission denied for environment prod",
+         "failure_kind": "tool_error"},
+        attempt_turns=turns, created_by="benchmark")
+    restored = event["retry_context"].replace(event["failure_summary"], "")
+    leaked = [t for t in turns if t in restored]
+    if leaked:
+        errors.append(f"fence leak: {leaked}")
+    if not event["contamination_fenced"]:
+        errors.append("contamination flag not set")
+    check = perseus.verify_isolation_event(event, snapshot=snap)
+    if not check["valid"]:
+        errors.append("isolation event failed verification: "
+                      + "; ".join(check["errors"]))
+
+    results = {
+        "schema_version": "perseus-retry-isolation-benchmark-results/v1",
+        "pass": not errors,
+        "errors": errors,
+        "simulation": {
+            "trials": args.trials,
+            "eps0": eps0,
+            "eps1": eps1,
+            "cascade_ratio": round(eps1 / eps0, 3),
+            "iid_pass_at_3": report["pass_at_k"]["iid"],
+            "contaminated_pass_at_3": report["pass_at_k"]["contaminated"],
+            "clean_restart_pass_at_3": report["pass_at_k"]["clean_restart"],
+            "iid_overestimate_pp": over_pp,
+            "clean_restart_recovery_pp": recovery_pp,
+        },
+        "allocation": alloc,
+        "fence": {
+            "quarantined_turns": event["quarantined_turn_count"],
+            "summary_tokens": event["summary_tokens"],
+            "retry_tokens": event["retry_tokens"],
+            "event_digest": event["event_digest"],
+        },
+    }
+    Path(args.out_results).write_text(
+        json.dumps(results, indent=2) + "\n", encoding="utf-8")
+
+    lines = [
+        "# CCRM retry-isolation benchmark — results (#972)",
+        "",
+        f"- simulation: {args.trials} seeded trials, cascade ratio "
+        f"{round(eps1/eps0, 2)}x (paper: ~7.1x)",
+        f"- gate: **{'PASS' if not errors else 'FAIL'}**",
+        "",
+        "| policy | pass@3 |",
+        "|---|---|",
+        f"| IID (overestimate) | {report['pass_at_k']['iid']:.1%} |",
+        f"| contaminated cascade | {report['pass_at_k']['contaminated']:.1%} |",
+        f"| clean restart (fenced) | {report['pass_at_k']['clean_restart']:.1%} |",
+        "",
+        f"- IID overestimate: **{over_pp}pp** (paper: 17.4pp on SWE-bench "
+        f"Verified; gate ≥ 8pp)",
+        f"- clean-restart recovery: **{recovery_pp}pp** (gate ≥ 8pp)",
+        f"- allocation: T* = {alloc['t_star_continuous']} → "
+        f"{alloc['optimal_attempts']} attempts from budget "
+        f"{alloc['total_budget']}",
+        f"- fence demo: {event['quarantined_turn_count']} turns quarantined, "
+        f"summary {event['summary_tokens']} tokens, event digest "
+        f"`{event['event_digest'][:16]}…`",
+    ]
+    for err in errors:
+        lines.append(f"- ❌ {err}")
+    lines.append("")
+    Path(args.out_report).write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+    print(f"retry-isolation gate: {'PASS' if not errors else 'FAIL'}")
+    print(f"  IID overestimate {over_pp}pp | clean-restart recovery "
+          f"{recovery_pp}pp | fence {'clean' if not leaked else 'LEAK'}")
+    for err in errors:
+        print(" -", err)
+    if errors:
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

docs/retry-isolation.md

@@ -0,0 +1,48 @@
+# Clean-restart attempt isolation (#972)
+
+Attempt-scoped context isolation for multi-attempt agent workloads. When an
+attempt fails, its turns stay in the context window and contaminate the
+retry. CCRM ([arXiv:2605.08563](https://arxiv.org/abs/2605.08563))
+formalizes this: an IID model overestimates pass@3 by 17.4 points on
+SWE-bench Verified (98.6% vs 81.2%), the contaminated-cascade model fits
+with error < 0.001, and the cascade ratio ε1/ε0 ≈ 7.1 — retry context is
+~7x more error-prone per step. The paper's clean-restart dominance theorem
+quantifies what context-clearing buys. Implementation:
+`src/perseus/retry_isolation.py`. Evaluation: `benchmark/retry-isolation/`.
+
+## The primitives
+
+- **Transactional checkpoints** — `snapshot_context` takes a digest-sealed
+  snapshot of the context at each attempt boundary (Perseus owns assembly,
+  so this is cheap); `verify_snapshot` replays the digest.
+- **Fencing on failure** — `build_retry_context` restores the pre-attempt
+  snapshot and injects a bounded, structured failure summary (attempt,
+  failed step, failure kind, observed error, quarantine count). The failed
+  attempt's turns are **quarantined** — the function fails closed if any of
+  them resurface in the restored portion of the retry context. The summary
+  is truncated to a hard token cap, never unbounded, and never embeds the
+  raw trace.
+- **Contamination events** — every fence emits a digest-sealed event
+  (`contamination_fenced`, quarantine size, summary, token accounting) for
+  observability; `verify_isolation_event` replays it.
+- **Attempt-budget allocation** — `attempt_budget_allocation` applies the
+  paper's closed form
+  `T* = sqrt(B · log(1/(1−ε1)) / log(1/(1−ε0)))`
+  to a fixed total budget, returning the optimal attempt count, per-attempt
+  budget, and the derivation inputs for audit.
+
+## Relationship to #934
+
+#934 (redaction retry fail-closed) is a skill-mining safety fix; this is
+the context-engine primitive for all multi-attempt workloads. They compose:
+a fenced retry here can feed its structured failure summary to the #968
+TRACE attribution layer as an explicit dissatisfaction signal.
+
+## Evaluation
+
+`benchmark/retry-isolation/run.py` — seeded Monte Carlo at the ~7.1x
+cascade ratio: IID overestimate of pass@3 ≥ 8pp (paper: 17.4pp),
+clean-restart recovery ≥ 8pp, closed-form allocation exactness, and a
+fence demonstration verifying quarantine + event replay. Deterministic
+(seed-pinned); the live SWE-bench-style workload remains an opt-in,
+paid evaluation outside this offline gate.