Skip to content

feat(context): add evidence-aware execution and runtime contracts #983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tcconnally merged 1 commit into from
Aug 17, 2026
Merged

feat(context): add evidence-aware execution and runtime contracts #983

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
218 changes: 216 additions & 2 deletions claims.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"_meta": {
"description": "Canonical claim registry for Perseus Computing public surfaces. Every public figure derives from here. When a benchmark re-runs, update THIS file only; scripts/render_claims.py regenerates manifest strings and tests/test_claims_sync.py fails CI if any surface drifts.",
"updated": "2026-08-15",
"updated": "2026-08-17",
"public_headline": "longmemeval_cot",
"public_headline_policy": "The validated official-CoT result is the current public QA headline. The plain-prompt result remains separately labeled methodology context.",
"public_site_positioning": "Perseus is the system around the model: Perseus resolves live context, Vault persists memory, and Ledger records evidence."
Expand Down Expand Up @@ -272,5 +272,219 @@
"label": "measured-baseline",
"publishable": true
}
}
},
"capabilities": [
{
"id": "context-rendering",
"capability": "Deterministic live context rendering",
"owner": "Perseus renderer",
"lifecycle": "operational",
"evidence_class": "code_and_tests",
"evidence_refs": [
"src/perseus/renderer.py",
"tests/test_renderer.py"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when renderer, resolver, or generated artifact changes.",
"claim_ceiling": "The shipped local renderer resolves supported directives deterministically for the tested contract.",
"non_claims": [
"This does not certify arbitrary third-party directives or downstream model behavior."
],
"dependencies": [
"Python standard library",
"PyYAML"
],
"activation": "perseus render <source>",
"proof_surface": "docs/PRODUCT_CONTRACT.md"
},
{
"id": "portable-context-artifacts",
"capability": "Hash-bound portable context artifacts",
"owner": "Context artifact compiler",
"lifecycle": "implemented",
"evidence_class": "code_and_tests",
"evidence_refs": [
"src/perseus/context_artifacts.py",
"tests/test_context_artifacts.py"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when artifact schemas or budget accounting changes.",
"claim_ceiling": "Artifacts carry deterministic schema, source commitments, and bounded sections.",
"non_claims": [
"A commitment proves integrity of supplied inputs, not their truth."
],
"dependencies": [
"Python standard library"
],
"activation": "build_agent_context_artifact(...) or build_memento_artifact(...)",
"proof_surface": "docs/context-artifact-contract.md"
},
{
"id": "versioned-context-contracts",
"capability": "Versioned evidence-aware context contracts",
"owner": "Context contract boundary",
"lifecycle": "tested",
"evidence_class": "contract_tests",
"evidence_refs": [
"src/perseus/context_contract.py",
"tests/test_context_contracts_916_917.py"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when context-contract schema or projection fields change.",
"claim_ceiling": "The bounded rank, ask, and projection paths expose sanitized evidence metadata and explicit non-success states.",
"non_claims": [
"Ranking is not a truth oracle and does not replace Vault or Ledger authority."
],
"dependencies": [
"Caller-supplied records",
"Context contract schema"
],
"activation": "context_rank(...) or context_ask(...)",
"proof_surface": "docs/context-contracts.md"
},
{
"id": "context-dag-compilation",
"capability": "Auditable budgeted context-DAG compilation",
"owner": "Context DAG compiler",
"lifecycle": "tested",
"evidence_class": "offline_tests",
"evidence_refs": [
"src/perseus/context_dag.py",
"tests/test_context_dag.py"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when graph, expansion, verdict, or budget logic changes.",
"claim_ceiling": "The compiler emits digest-sealed graph and packet decisions under explicit hard limits.",
"non_claims": [
"Rendered accounting is not provider billing or a hardware benchmark."
],
"dependencies": [
"Caller fetch function",
"Explicit compilation policy"
],
"activation": "compile_context_dag(...)",
"proof_surface": "docs/context-dag.md"
},
{
"id": "vault-context-fallback",
"capability": "Vault-backed context with explicit degraded fallback",
"owner": "Vault connector",
"lifecycle": "degraded",
"evidence_class": "integration_contract",
"evidence_refs": [
"src/perseus/vault_connector.py",
"tests/test_vault_served_contract.py"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when connector transport, fallback, or serving policy changes.",
"claim_ceiling": "Unavailable Vault integration is surfaced as a bounded degraded path rather than an equivalent complete result.",
"non_claims": [
"The fallback is not a replacement for Vault durability, provenance, or retrieval authority."
],
"dependencies": [
"Optional Perseus Vault service",
"Workspace configuration"
],
"activation": "perseus_vault configuration with context rendering",
"proof_surface": "docs/DEPLOYMENT.md"
},
{
"id": "optional-context-extensions",
"capability": "Optional extension and plugin adapters",
"owner": "Registry and adapter boundary",
"lifecycle": "omitted",
"evidence_class": "boundary_documentation",
"evidence_refs": [
"src/perseus/registry.py",
"docs/context-cartridges.md"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-verify when extension loading or adapter registration changes.",
"claim_ceiling": "Optional extensions are isolated from the dependency-free core and require their declared boundary.",
"non_claims": [
"No unreviewed extension is part of the default runtime guarantee."
],
"dependencies": [
"Explicit extension configuration"
],
"activation": "Opt-in registry or adapter configuration",
"proof_surface": "docs/CONTEXT-ADAPTER-SDK.md"
},
{
"id": "retired-synthetic-headline",
"capability": "Retired asymmetric benchmark headline",
"owner": "Claims registry",
"lifecycle": "historical",
"evidence_class": "historical_artifact",
"evidence_refs": [
"benchmark/ultimate_suite_results.json",
"docs/claims-pipeline.md"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Retain only while the historical audit record is useful; never promote to current state.",
"claim_ceiling": "This row records why the former headline was retired and is not a current capability claim.",
"non_claims": [
"The historical figure is not a current performance result."
],
"dependencies": [
"Claims audit history"
],
"activation": "Historical documentation only",
"proof_surface": "docs/claims-pipeline.md"
},
{
"id": "external-model-inference",
"capability": "External model inference owned by Perseus",
"owner": "Runtime integration boundary",
"lifecycle": "not_demonstrated",
"evidence_class": "non_claim",
"evidence_refs": [
"docs/RESOLVER_VS_GENERATOR.md",
"docs/context-contracts.md"
],
"last_verified": {
"commit": "a33591e",
"build": "perseus v1.0.26",
"date": "2026-08-17"
},
"freshness_rule": "Re-evaluate only when a separately verified runtime adapter is published.",
"claim_ceiling": "Perseus supplies bounded context and evidence contracts; no owned inference capability is asserted.",
"non_claims": [
"No vendor, accelerator, model quality, or embedded execution result is demonstrated here."
],
"dependencies": [
"A separately qualified runtime adapter"
],
"activation": "Not available in the current core contract",
"proof_surface": "docs/RESOLVER_VS_GENERATOR.md"
}
]
}
20 changes: 20 additions & 0 deletions docs/CAPABILITY-EVIDENCE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Capability evidence matrix

Generated from [`claims.json`](../claims.json) by `scripts/render_claims.py`; edit the registry, not this file.

Lifecycle states are intentionally distinct: `implemented`, `tested`, `operational`, `degraded`, `omitted`, `historical`, and `not_demonstrated`.

| Capability | Owner | Lifecycle | Evidence | Verification | Claim ceiling | Non-claims | Dependencies / activation | Proof surface |
|---|---|---|---|---|---|---|---|---|
| Auditable budgeted context-DAG compilation | Context DAG compiler | `tested` | [src/perseus/context_dag.py](../src/perseus/context_dag.py), [tests/test_context_dag.py](../tests/test_context_dag.py) | 2026-08-17 / `a33591e` / perseus v1.0.26 | The compiler emits digest-sealed graph and packet decisions under explicit hard limits. | Rendered accounting is not provider billing or a hardware benchmark. | Caller fetch function, Explicit compilation policy; compile_context_dag(...) | [docs/context-dag.md](../docs/context-dag.md) |
| Deterministic live context rendering | Perseus renderer | `operational` | [src/perseus/renderer.py](../src/perseus/renderer.py), [tests/test_renderer.py](../tests/test_renderer.py) | 2026-08-17 / `a33591e` / perseus v1.0.26 | The shipped local renderer resolves supported directives deterministically for the tested contract. | This does not certify arbitrary third-party directives or downstream model behavior. | Python standard library, PyYAML; perseus render <source> | [docs/PRODUCT_CONTRACT.md](../docs/PRODUCT_CONTRACT.md) |
| External model inference owned by Perseus | Runtime integration boundary | `not_demonstrated` | [docs/RESOLVER_VS_GENERATOR.md](../docs/RESOLVER_VS_GENERATOR.md), [docs/context-contracts.md](../docs/context-contracts.md) | 2026-08-17 / `a33591e` / perseus v1.0.26 | Perseus supplies bounded context and evidence contracts; no owned inference capability is asserted. | No vendor, accelerator, model quality, or embedded execution result is demonstrated here. | A separately qualified runtime adapter; Not available in the current core contract | [docs/RESOLVER_VS_GENERATOR.md](../docs/RESOLVER_VS_GENERATOR.md) |
| Optional extension and plugin adapters | Registry and adapter boundary | `omitted` | [src/perseus/registry.py](../src/perseus/registry.py), [docs/context-cartridges.md](../docs/context-cartridges.md) | 2026-08-17 / `a33591e` / perseus v1.0.26 | Optional extensions are isolated from the dependency-free core and require their declared boundary. | No unreviewed extension is part of the default runtime guarantee. | Explicit extension configuration; Opt-in registry or adapter configuration | [docs/CONTEXT-ADAPTER-SDK.md](../docs/CONTEXT-ADAPTER-SDK.md) |
| Hash-bound portable context artifacts | Context artifact compiler | `implemented` | [src/perseus/context_artifacts.py](../src/perseus/context_artifacts.py), [tests/test_context_artifacts.py](../tests/test_context_artifacts.py) | 2026-08-17 / `a33591e` / perseus v1.0.26 | Artifacts carry deterministic schema, source commitments, and bounded sections. | A commitment proves integrity of supplied inputs, not their truth. | Python standard library; build_agent_context_artifact(...) or build_memento_artifact(...) | [docs/context-artifact-contract.md](../docs/context-artifact-contract.md) |
| Retired asymmetric benchmark headline | Claims registry | `historical` | [benchmark/ultimate_suite_results.json](../benchmark/ultimate_suite_results.json), [docs/claims-pipeline.md](../docs/claims-pipeline.md) | 2026-08-17 / `a33591e` / perseus v1.0.26 | This row records why the former headline was retired and is not a current capability claim. | The historical figure is not a current performance result. | Claims audit history; Historical documentation only | [docs/claims-pipeline.md](../docs/claims-pipeline.md) |
| Vault-backed context with explicit degraded fallback | Vault connector | `degraded` | [src/perseus/vault_connector.py](../src/perseus/vault_connector.py), [tests/test_vault_served_contract.py](../tests/test_vault_served_contract.py) | 2026-08-17 / `a33591e` / perseus v1.0.26 | Unavailable Vault integration is surfaced as a bounded degraded path rather than an equivalent complete result. | The fallback is not a replacement for Vault durability, provenance, or retrieval authority. | Optional Perseus Vault service, Workspace configuration; perseus_vault configuration with context rendering | [docs/DEPLOYMENT.md](../docs/DEPLOYMENT.md) |
| Versioned evidence-aware context contracts | Context contract boundary | `tested` | [src/perseus/context_contract.py](../src/perseus/context_contract.py), [tests/test_context_contracts_916_917.py](../tests/test_context_contracts_916_917.py) | 2026-08-17 / `a33591e` / perseus v1.0.26 | The bounded rank, ask, and projection paths expose sanitized evidence metadata and explicit non-success states. | Ranking is not a truth oracle and does not replace Vault or Ledger authority. | Caller-supplied records, Context contract schema; context_rank(...) or context_ask(...) | [docs/context-contracts.md](../docs/context-contracts.md) |

## Freshness and scope

Each row carries its own freshness rule and claim ceiling. A lifecycle value is not a certification: it describes the evidence state of this repository at the recorded verification point.
Loading
Loading