diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3a7f142..a50d701 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -39,7 +39,13 @@ jobs: node-version: ${{ matrix.node }} cache: npm - run: npm ci - - run: npm run check + - name: Run quality and publishable-history audit + run: npm run check + env: + # pull_request checks run on a synthetic GitHub merge commit. Audit the + # real publishable head history while all build/test steps still run + # against the synthetic merge result. + AGENTIC_STRATA_AUDIT_HISTORY_REF: ${{ github.event.pull_request.head.sha || '' }} - run: npm run demo - run: node dist/cli.js validate examples/application.manifest.yaml - run: node dist/cli.js conformance examples/generic-change/run.bundle.json --profile enterprise diff --git a/README.md b/README.md index fb43095..99dfa38 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,8 @@ Logical strata may share one process. The rule is a dependency and responsibilit - A composite cache fingerprint that requires both semantic and operational equivalence: policy, capability set, authoritative context, tenant, privacy, constraints, and outcome are all bound. - An RFC 8785 JSON Canonicalization Scheme implementation with cross-language golden vectors, strict duplicate-member rejection at the JSON boundary, and a hash-linked receipt chain with tamper and missing-evidence detection. - Runtime conformance checks for rooted authority, causal evidence, actual budget consumption, exact-action idempotency, approval, read-back or compensation, predeclared criterion evidence bindings, layer direction, and model/authority separation. -- Mapping-only adapter documents for AG-UI, A2A, MCP, OpenTelemetry GenAI, CloudEvents, and an external policy engine. +- An [Execution Passport](docs/spec/attestations/execution-passport/v1/README.md) that binds one run bundle, its complete conformance report, and one externally validated OASF record in a strict in-toto Statement without embedding those documents. +- Mapping-only adapter documents for AG-UI, A2A, MCP, OpenTelemetry GenAI, CloudEvents, an external policy engine, OASF, and content-free execution-placement evidence. - A neutral prepare/commit/read-back demo with exact-action approval and duplicate suppression. ## Quick start @@ -60,6 +61,19 @@ node dist/cli.js replay examples/generic-change/run.bundle.json node dist/cli.js explain examples/generic-change/run.bundle.json --profile enterprise ``` +After producing a report, bind it to a real OASF record that has been validated by its owning ecosystem: + +```bash +node dist/cli.js passport .tmp/demo/run.bundle.json \ + --report .tmp/demo/conformance-report.json \ + --oasf-record /path/to/validated-oasf-record.json \ + --oasf-media-type "$OASF_MEDIA_TYPE" \ + --output .tmp/demo/execution-passport.json +node dist/cli.js validate .tmp/demo/execution-passport.json +``` + +The output file contains exact RFC 8785 Statement bytes suitable for an external DSSE/Sigstore adapter. Repeat `--execution-evidence` only with strict `ExecutionEvidenceBinding` documents containing descriptors and digests, never provider output. + All commands accept JSON; `validate` and `lint` also accept YAML. Build once with `npm run build` before invoking the repository-local CLI. Input documents are bounded to 16 MiB, JSON member names must be unique, and YAML alias expansion is limited. ## Profiles @@ -85,6 +99,7 @@ This makes it complementary to: - [MCP](https://modelcontextprotocol.io/specification/2025-11-25) for tools and context; - [OpenTelemetry GenAI](https://github.com/open-telemetry/semantic-conventions-genai) for operational telemetry; - [CloudEvents](https://github.com/cloudevents/spec) for event interchange; +- [in-toto Statement v1](https://github.com/in-toto/attestation/tree/main/spec/v1) for the Execution Passport envelope; - [Open Agent Spec](https://github.com/oracle/agent-spec) and [OSSA](https://openstandardagents.org/specification/) for portable pre-runtime definitions. Pattern and source-layout scanners can discover whether a project appears to use architectural conventions. AgenticStrata instead evaluates a declared manifest together with runtime receipts and artifacts. These are different, compatible jobs. @@ -98,6 +113,9 @@ Read [Architecture](docs/architecture.md), [Contracts and conformance](docs/cont - Prepared actions, observed results, and compensation results use distinct attested evidence roles. Each evidence-required acceptance criterion predeclares the exact capability, action digest, and role it accepts; reusing pre-effect or unrelated evidence fails conformance. - Run status and conformance status are separate: a failed run can still have an intact, conformant evidence record, but it is never explained as a completed outcome. - Reports bind the evaluator name, package version, semantic revision, selected rule set, and effective profile configuration. The evaluator digest identifies the declared implementation revision; it is not a code signature. +- An unsigned Execution Passport proves deterministic integrity and cross-artifact binding only. Authenticity requires an externally verified DSSE envelope or equivalent trusted channel, and consumers must make that requirement explicit so removing a signature cannot silently downgrade policy. +- The OASF subject digest proves which opaque record was bound; it does not prove that the record is semantically valid OASF. Validate it before passport creation with the specification owner’s tooling. +- Execution-placement evidence enters the predicate only as strict descriptors with no payload or `content` field. Descriptor metadata must come from a trusted producer and be redacted or pseudonymized when sensitive; its optional stable URI may be omitted. Full provider result objects are outside the Passport contract. - A safe fingerprint binds the declared canonical intent. It does not prove that an upstream intent normalizer chose the correct meaning. - Adapter documents are integration contracts, not bundled protocol SDKs or policy engines. - The current release is an alpha contract surface. Version consumers explicitly before production adoption. diff --git a/adapters/oasf.mapping.yaml b/adapters/oasf.mapping.yaml new file mode 100644 index 0000000..d5f3332 --- /dev/null +++ b/adapters/oasf.mapping.yaml @@ -0,0 +1,12 @@ +contractType: AdapterMapping +apiVersion: agenticstrata.dev/v1 +mappingId: adapter/oasf +protocol: oasf +protocolVersion: externally-validated +direction: inbound +contractBindings: + - strataContract: ExecutionPassport + protocolObject: OASF agent record ResourceDescriptor + notes: The external record remains opaque; only its canonical digest, media type, and optional bounded stable HTTPS or URN reference enter the ordered subject tuple. +lossPolicy: explicit-loss +extensionPoint: true diff --git a/adapters/stagefabric.mapping.yaml b/adapters/stagefabric.mapping.yaml new file mode 100644 index 0000000..3d6f2a3 --- /dev/null +++ b/adapters/stagefabric.mapping.yaml @@ -0,0 +1,15 @@ +contractType: AdapterMapping +apiVersion: agenticstrata.dev/v1 +mappingId: adapter/stagefabric +protocol: stagefabric +protocolVersion: configurable +direction: inbound +contractBindings: + - strataContract: ExecutionEvidenceBinding + protocolObject: ExecutionPlacementEvidence ResourceDescriptor + notes: Only role, producer, disclosure mode, resource name, canonical digest, media type, and an optional bounded stable HTTPS or URN reference are admitted; metadata must be trusted and redacted or pseudonymized when sensitive. + - strataContract: ExecutionPassport + protocolObject: predicate executionEvidence entry + notes: No payload or content field is admitted; the full provider result remains outside the mapping. +lossPolicy: explicit-loss +extensionPoint: true diff --git a/docs/adr/0004-execution-passport.md b/docs/adr/0004-execution-passport.md new file mode 100644 index 0000000..a5b550a --- /dev/null +++ b/docs/adr/0004-execution-passport.md @@ -0,0 +1,28 @@ +# ADR 0004: Execution Passport as a strict external attestation statement + +Status: accepted + +## Context + +A portable run lineage needs to bind the runtime bundle, the assessment actually performed over it, and the external agent record without copying those documents into another proprietary envelope. It must also preserve failed and incomplete state honestly, avoid carrying provider payloads, and allow deployments to add authenticity without coupling the contract core to one signing service. + +## Decision + +AgenticStrata defines Execution Passport v1 as a strict in-toto Statement. Its predicate URI is the project-controlled, versioned documentation path: + +`https://github.com/aantenore/AgenticStrata/tree/main/docs/spec/attestations/execution-passport/v1` + +The Statement has exactly three ordered subjects: `RunBundle`, complete `ConformanceReport`, and one opaque OASF record. Each uses a strict `ResourceDescriptor` with no payload or `content` field and at most one bounded stable HTTPS or URN reference. The predicate records run and conformance status separately and accepts optional execution-placement evidence only as descriptor bindings whose metadata remains subject to producer trust and redaction policy. + +The core verifies schema, seals, ordered unique check-to-rule binding, derived conformance status, artifact bindings, receipt integrity, run identity, terminal status, report-after-evidence chronology, and issuance ordering before emission. OASF semantic validation remains external. Passport creation follows conformance evaluation and is not itself a conformance prerequisite. + +The core emits RFC 8785 canonical Statement bytes but performs no signing. DSSE/Sigstore or another signature mechanism is an external adapter and consumer-policy concern. + +## Consequences + +- Consumers can reproduce every subject digest without importing a provider SDK. +- A failed or incomplete run can carry honest lineage without being described as successful. +- The execution-evidence contract has no payload or `content` field and does not accept a full provider result; descriptor metadata still requires trust, redaction, or pseudonymization. +- The OASF digest proves which bytes were bound, not that they satisfy OASF semantics. +- Unsigned Statements provide integrity and binding only. Authenticity, signer policy, transparency, revocation, and trusted time require external verification. +- Consumers that require authentication must reject bare Statements rather than silently accepting them after signature removal. diff --git a/docs/architecture.md b/docs/architecture.md index 90970a1..6ffd372 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -33,6 +33,7 @@ The `lint` command validates the complete set, canonical ordinal, unique declara - Runtime payloads are externalized as artifacts and bound by attestations. - Decisions expose concise summaries, selected and rejected options, and evidence references; private model reasoning is not a contract. - Distributed mode requires node attribution on every receipt. +- An Execution Passport projects completed, failed, or incomplete run state into a strict in-toto Statement while keeping the run bundle, report, OASF record, and provider evidence out of the Statement body. ### Lifecycle & Conformance @@ -69,6 +70,18 @@ sequenceDiagram Control-->>User: outcome + explanation + conformance ``` +## Execution Passport boundary + +Passport creation is a deterministic packaging step after conformance evaluation, not another conformance rule. The builder verifies the bundle and report schemas, report and evaluator seals, the ordered unique check-to-rule binding and derived status, exact bundle and manifest bindings, receipt-chain integrity, run identity, terminal state, report-after-evidence chronology, and issuance ordering. It then emits exactly three ordered subjects: + +1. the canonical `RunBundle` digest; +2. the canonical digest of the complete `ConformanceReport`, including its own `digest` field; +3. the canonical digest of one opaque, externally validated OASF record. + +Optional execution-placement evidence is already reduced by its producer to a `ResourceDescriptor` with no payload or `content` field. AgenticStrata admits only the descriptor, producer identity, fixed role, and `content-free` disclosure marker; descriptor metadata must be trusted and redacted or pseudonymized when sensitive, and its URI may be omitted. It never consumes a provider’s full execution result through this boundary. + +The core emits an unsigned Statement. A replaceable external adapter may wrap its exact RFC 8785 bytes in DSSE and use Sigstore or another trust system. Signature verification, signer identity, transparency-log policy, and trusted time remain consumer-owned controls. + ## Safe semantic caching Two utterances may normalize to the same canonical intent, but a cached answer is admitted only when operational context is also equivalent. The cache key binds: diff --git a/docs/contracts-and-conformance.md b/docs/contracts-and-conformance.md index 59a6eb4..28f99b5 100644 --- a/docs/contracts-and-conformance.md +++ b/docs/contracts-and-conformance.md @@ -11,6 +11,7 @@ | Capability | `CapabilityContract` | Defines risk, effect, schemas, scopes, and side-effect lifecycle. | | Evidence | `DecisionEvidence`, `RuntimeArtifact`, `ArtifactAttestation`, `TraceEvent` | Reconstructs what was selected, applied, observed, and attested. | | Assessment | `ConformanceReport` | Records profile checks and evidence-backed pass/fail status. | +| Attestation exchange | `ExecutionPassport`, `ExecutionEvidenceBinding`, `ResourceDescriptor` | Binds a run, assessment, external agent record, and content-free placement evidence without embedding payloads. | | Integration | `AdapterMapping` | Describes protocol projection and explicit information loss. | All integrity-bearing records use deterministic SHA-256 digests over [RFC 8785 JSON Canonicalization Scheme](https://www.rfc-editor.org/rfc/rfc8785.html) output. AgenticStrata uses the maintained `canonicalize` implementation referenced by the RFC and tests standard key-ordering and number vectors. Inputs must be I-JSON values; the file boundary rejects duplicate JSON member names before the parsed value can enter the contract runtime. Digest verification proves that a record matches its bytes and references; issuer authenticity requires an external signature or trusted append-only store. @@ -41,6 +42,18 @@ The core profile checks: Every report binds the run bundle, selected rule list, effective merged profile configuration, and evaluator identity by digest. Evaluator identity contains the package version plus an explicitly versioned conformance revision. This makes the assessment reproducible, prevents a custom profile from silently downgrading a built-in name, and exposes which evaluator semantics produced the result. The evaluator digest binds declared release metadata; authenticity and executable-byte attestation remain deployment concerns. +## Execution Passport invariants + +The [v1 specification](spec/attestations/execution-passport/v1/README.md) is an in-toto Statement with one project-owned predicate URI. Its subject tuple has fixed order and length: run bundle, complete conformance report, external OASF record. Each subject is a strict `ResourceDescriptor` containing only a name, one SHA-256 digest, a media type, and an optional bounded stable HTTPS or URN reference. The report subject uses `digestValue(report)`, not `report.digest`: the former binds the complete serialized report, while the latter seals the report content before its own digest field is added. + +The predicate records `runStatus` and `conformanceStatus` separately. Creation does not require either status to be successful, so an incomplete or failed run can be bound honestly. It does require unambiguous run identity, a valid receipt chain, a sealed report and evaluator identity, ordered unique check identifiers bound by `rulesDigest`, status derived from those checks, matching bundle and manifest digests, report/receipt status agreement, report generation no earlier than any included observed evidence, and issuance no earlier than report generation. + +OASF input is accepted only as an opaque I-JSON object and canonicalized for its subject digest. AgenticStrata deliberately does not carry an OASF schema and therefore does not claim semantic validity. A caller must validate the record externally and provide the owner-declared media type. + +`executionEvidence` accepts only strict bindings with no payload or `content` field. The producer computes and publishes the evidence artifact digest; AgenticStrata carries its descriptor but never receives the full provider result through the CLI. Descriptor metadata must be trusted and redacted or pseudonymized when sensitive, and the URI may be omitted. + +The Passport is created after assessment and is not required by a conformance rule. Making it a prerequisite of the report it contains would create a circular dependency. + ## Evidence, not file detection A conformance run consumes a `RunBundle`. It does not infer architecture from directory names, imports, or keywords. This avoids declaring success simply because a project contains a policy file or telemetry package. Conversely, a source scanner remains useful before runtime evidence exists; its result should be treated as a discovery signal rather than runtime proof. diff --git a/docs/interoperability.md b/docs/interoperability.md index 68e2bc6..860fe45 100644 --- a/docs/interoperability.md +++ b/docs/interoperability.md @@ -10,9 +10,23 @@ AgenticStrata owns application-level runtime invariants. It deliberately leaves | OpenTelemetry GenAI | Spans, decision summaries, conformance status | Private reasoning and sensitive payloads stay out of telemetry. | | CloudEvents | Receipt and artifact references | Correlation, tenant, privacy, and digest bindings survive transport. | | Policy engine | Grant, approval, attenuation query | The external enforcement point remains authoritative. | +| OASF | Opaque agent-record subject descriptor | AgenticStrata binds canonical bytes but delegates semantic validation and media-type ownership. | +| in-toto / DSSE | Execution Passport Statement and optional external envelope | The core owns deterministic binding; signing, identity, transparency, and trusted time stay outside it. | +| Execution-placement provider | Content-free evidence descriptor | No payload or `content` field is admitted; descriptor metadata remains subject to producer trust, redaction, and pseudonymization policy. | Adapter YAML files are declarative mappings with an explicit `lossPolicy`. They contain no vendor SDK and can be replaced without changing the core contract registry. +## Attestation exchange + +The Execution Passport uses the project-controlled predicate namespace documented at [`docs/spec/attestations/execution-passport/v1`](spec/attestations/execution-passport/v1/README.md). The core writes exact RFC 8785 Statement bytes. A deployment may pass those bytes to an external DSSE/Sigstore adapter using payload type `application/vnd.in-toto+json`; no signing key, certificate flow, or Sigstore dependency enters AgenticStrata. + +Consumers choose one of two explicit policies: + +- **unsigned allowed**: verify schema, canonical digests, report binding, and local transport trust; this establishes integrity and association, not signer authenticity; +- **authenticated required**: reject a bare Statement and require successful external envelope, signer-identity, and trust-material verification. + +There is no implicit fallback from the second policy to the first. This prevents signature removal from becoming a silent trust downgrade. + ## Agent and flow specifications Portable agent specifications answer questions such as “what is this agent, which tools may it request, and how is a flow assembled?” AgenticStrata answers “what did this application run actually request, authorize, change, verify, and prove?” diff --git a/docs/spec/attestations/execution-passport/v1/README.md b/docs/spec/attestations/execution-passport/v1/README.md new file mode 100644 index 0000000..8860063 --- /dev/null +++ b/docs/spec/attestations/execution-passport/v1/README.md @@ -0,0 +1,120 @@ +# AgenticStrata Execution Passport v1 + +Status: alpha, versioned contract + +Predicate type: + +`https://github.com/aantenore/AgenticStrata/tree/main/docs/spec/attestations/execution-passport/v1` + +## Purpose + +Execution Passport v1 is an in-toto Statement that binds one runtime bundle, the complete conformance report produced for that bundle, and one external OASF agent record. It packages lineage; it is not a success certificate, an OASF validator, a digital signature, or a trusted timestamp. + +The normative machine-readable shape is `$defs.ExecutionPassport` in [`schemas/v1/agentic-strata.schema.json`](../../../../../schemas/v1/agentic-strata.schema.json). + +## Statement envelope + +The top-level object contains exactly four fields: + +| Field | Required value | +| --- | --- | +| `_type` | `https://in-toto.io/Statement/v1` | +| `subject` | The exact ordered three-element tuple below | +| `predicateType` | The predicate URI at the top of this document | +| `predicate` | The strict Execution Passport predicate | + +Additional top-level fields, including embedded signatures, are invalid. A signature belongs to an external envelope. + +## Ordered subjects + +Subject order and cardinality are normative: + +| Index | `name` | `mediaType` | SHA-256 input | +| --- | --- | --- | --- | +| 0 | `agentic-strata-run-bundle` | `application/vnd.aantenore.agenticstrata.run-bundle+json` | RFC 8785 canonical `RunBundle` | +| 1 | `agentic-strata-conformance-report` | `application/vnd.aantenore.agenticstrata.conformance-report+json` | RFC 8785 canonical complete `ConformanceReport` | +| 2 | `oasf-agent-record` | Declared by the external record owner | RFC 8785 canonical opaque OASF JSON record | + +The report subject digest is `digestValue(report)`. It is deliberately not `report.digest`, because `report.digest` seals the report before its own digest field is present while the subject binds the complete exchanged artifact. + +Each subject is a strict ResourceDescriptor with only: + +- `name`; +- `digest`, containing exactly one lowercase hexadecimal `sha256` value; +- `mediaType`; +- optional bounded stable `uri`: either HTTPS without user information, query, or fragment, or a URN without query or fragment. + +`content`, `downloadLocation`, annotations, inline URI schemes, local paths, extra digest algorithms, and any other extension are invalid in v1. URI values are limited to 2,048 characters. + +## Predicate + +The predicate contains exactly these fields: + +| Field | Meaning | +| --- | --- | +| `apiVersion` | AgenticStrata contract API version | +| `kind` | Fixed value `ExecutionPassport` | +| `runId` | One run identifier shared by the bound runtime records | +| `profile` | Profile evaluated by the bound report | +| `runStatus` | `completed`, `failed`, or `incomplete` | +| `conformanceStatus` | `pass` or `fail`, independent of run status | +| `evaluatorDigest` | Digest of the report evaluator identity | +| `terminalReceiptDigest` | Final terminal receipt digest, or `null` for an incomplete run | +| `issuedAt` | Self-asserted date-time no earlier than report generation | +| `executionEvidence` | Zero or more strict content-free evidence bindings | + +Passport presence never changes either status and never implies success. + +## Execution evidence + +An execution-evidence entry contains exactly: + +- `role: execution-placement`; +- a provider-neutral `producer` identifier; +- `disclosure: content-free`; +- one strict ResourceDescriptor for a separately stored evidence artifact. + +The producer computes the descriptor digest before Passport creation. The AgenticStrata CLI reads the binding document, not the provider’s full execution result, and the contract has no payload or `content` field. Descriptor metadata is not inherently non-sensitive: `producer`, resource name, media type, and an optional stable URI must come from a trusted producer and be redacted or pseudonymized when necessary. The URI may be omitted. Evidence descriptors remain predicate support and do not add a fourth Statement subject. + +## Creation checks + +A conforming builder fails closed unless: + +1. the bundle and report satisfy their JSON Schemas; +2. the report seal and nested evaluator seal match their content; +3. report check identifiers are unique, their ordered list matches `rulesDigest`, and report status is derived from those checks; +4. `report.runBundleDigest` equals the canonical bundle digest; +5. `report.manifestDigest` equals the canonical bound manifest digest; +6. the receipt chain is intact and every observed run identifier agrees; +7. report run status agrees with the final receipt shape; +8. `report.generatedAt` is no earlier than every included trace, usage, boundary, criterion, decision, attestation, or approval observation time; +9. `issuedAt` is a valid date-time no earlier than `report.generatedAt`; +10. every descriptor and execution-evidence binding is strict and duplicate evidence digests are absent. + +These checks establish internal consistency. Hashes alone do not authenticate a producer and cannot prevent wholesale replacement by an actor able to create a new self-consistent set. + +## OASF boundary + +AgenticStrata requires the OASF input to be an I-JSON object so it can compute a deterministic digest. It does not define or import an OASF schema. Callers must validate the record with tooling and version rules from its owning ecosystem before invoking Passport creation. A matching subject digest proves byte-equivalent canonical content, not semantic OASF validity. + +## Optional DSSE and Sigstore + +Signing is external to the core: + +1. obtain the exact RFC 8785 Statement bytes emitted by the CLI output file; +2. use DSSE payload type `application/vnd.in-toto+json`; +3. sign and verify with Sigstore or another configured trust system; +4. enforce signer identity, trust material, transparency, revocation, and freshness in consumer policy. + +The Statement does not contain keys, certificates, signatures, or Sigstore bundles. External verification must return the authenticated Statement bytes; the consumer then validates this schema and its subject bindings. + +Consumer policy has two explicit modes: + +- **unsigned allowed** establishes deterministic integrity and association only; +- **authenticated required** rejects a bare Statement and requires successful external envelope verification. + +There is no automatic downgrade between modes. + +## Versioning + +The predicate URI and this directory identify v1 semantics. Breaking field, subject, digest, or validation changes require a new versioned path and predicate URI. Additive package releases may keep v1 only when every existing v1 document retains the same meaning and validation result. diff --git a/docs/threat-model.md b/docs/threat-model.md index dc81dc7..204b5df 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -32,10 +32,18 @@ Model output, retrieved content, external messages, tool descriptions, and impor | Duplicate JSON keys or oversized/alias-heavy input confuse the CLI | Duplicate-member rejection, file-size limits, and YAML alias limits. | Deep but valid JSON may still require process-level resource limits for hostile multi-tenant use. | | A report is replayed under ambiguous evaluator semantics | Reports bind evaluator name, package version, semantic revision, rule IDs, and effective configuration by digest. | The declared evaluator digest is not a code signature; trusted distribution or executable attestation is required for stronger provenance. | | Imported agent definition becomes mutable | Reference plus digest; embedded redefinition is rejected. | Availability and signature verification belong to the artifact resolver. | +| A Passport mixes a run with another report or manifest | Creation recomputes canonical bundle and manifest digests, verifies the complete report and evaluator seals, and rejects mismatched run identity or terminal state. | Hashes do not establish who produced otherwise self-consistent artifacts. | +| A Passport is interpreted as proof of success | The predicate carries separate explicit run and conformance statuses and permits failed or incomplete states. | Consumers must still enforce the statuses appropriate to their decision. | +| Raw execution data leaks through placement evidence | The schema and CLI reject payload/content fields, inline or local URI schemes, credentialed HTTPS references, query strings, fragments, and oversized URIs. | Descriptor names, producer identifiers, media types, HTTPS paths, and URNs remain metadata channels; accept them only from trusted producers and redact, pseudonymize, or omit optional URIs. | +| A report claims success after checks are removed, reordered, duplicated, or changed | Creation requires unique ordered check identifiers, binds that order to `rulesDigest`, and derives report status from the checks. | Hash consistency does not authenticate the evaluator or its implementation. | +| A report or Passport appears to predate evidence it assesses | Creation requires report generation after every included observed evidence timestamp and issuance after report generation. | These are self-asserted consistency checks, not trusted timestamps; non-repudiation requires external infrastructure. | +| An OASF digest is mistaken for semantic validation | The record is treated as opaque I-JSON and documentation requires validation by the owning ecosystem before binding. | A caller can still bind an invalid record if it ignores that prerequisite. | +| A DSSE envelope is removed before consumption | Authenticated consumer policy rejects a bare Statement; unsigned acceptance is a separate explicit mode. | Trust still depends on external verifier configuration, signer identity constraints, and trust material. | +| Statement bytes and digest semantics diverge | Subject digests use RFC 8785 semantic JSON canonicalization, and the CLI emits those exact canonical Statement bytes for an external signer. | Other tooling must preserve the documented payload type and verify the exact DSSE payload bytes. | ## Non-goals of the alpha -- key management, digital signatures, or trusted timestamping; +- built-in key management, digital-signature services, or trusted timestamping; - a policy language or identity provider; - credential brokering; - sandboxing model-generated code; diff --git a/package-lock.json b/package-lock.json index 0f8934a..f267802 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "agentic-strata", - "version": "0.1.0-alpha.1", + "version": "0.2.0-alpha.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "agentic-strata", - "version": "0.1.0-alpha.1", + "version": "0.2.0-alpha.1", "license": "Apache-2.0", "dependencies": { "ajv": "^8.17.1", diff --git a/package.json b/package.json index 42aa2f4..f5ec14f 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "agentic-strata", - "version": "0.1.0-alpha.1", + "version": "0.2.0-alpha.1", "description": "Executable, contract-first reference architecture for enterprise agentic applications.", "type": "module", "license": "Apache-2.0", diff --git a/schemas/v1/agentic-strata.schema.json b/schemas/v1/agentic-strata.schema.json index 0e1ef20..b8b6aaa 100644 --- a/schemas/v1/agentic-strata.schema.json +++ b/schemas/v1/agentic-strata.schema.json @@ -20,6 +20,7 @@ { "$ref": "#/$defs/ArtifactAttestation" }, { "$ref": "#/$defs/TraceEvent" }, { "$ref": "#/$defs/ConformanceReport" }, + { "$ref": "#/$defs/ExecutionPassport" }, { "$ref": "#/$defs/AdapterMapping" }, { "$ref": "#/$defs/RunBundle" } ], @@ -32,6 +33,55 @@ "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "Sha256DigestSet": { + "type": "object", + "additionalProperties": false, + "properties": { + "sha256": { "$ref": "#/$defs/Digest" } + }, + "required": ["sha256"] + }, + "MediaType": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9!#$&^_.+~-]*/[a-z0-9][a-z0-9!#$&^_.+~-]*$" + }, + "ExternalUri": { + "type": "string", + "format": "uri", + "maxLength": 2048, + "oneOf": [ + { + "description": "Stable HTTPS reference without credentials, query, or fragment.", + "pattern": "^https://[^/?#@]+(?:/[^?#]*)?$" + }, + { + "description": "Bounded stable URN without query or fragment.", + "pattern": "^urn:[a-zA-Z0-9][a-zA-Z0-9-]{0,31}:[a-zA-Z0-9][a-zA-Z0-9._~:/%+-]*$" + } + ] + }, + "ResourceDescriptor": { + "type": "object", + "additionalProperties": false, + "properties": { + "name": { "$ref": "#/$defs/Identifier" }, + "digest": { "$ref": "#/$defs/Sha256DigestSet" }, + "mediaType": { "$ref": "#/$defs/MediaType" }, + "uri": { "$ref": "#/$defs/ExternalUri" } + }, + "required": ["name", "digest", "mediaType"] + }, + "ExecutionEvidenceBinding": { + "type": "object", + "additionalProperties": false, + "properties": { + "role": { "type": "string", "const": "execution-placement" }, + "producer": { "$ref": "#/$defs/Identifier" }, + "resource": { "$ref": "#/$defs/ResourceDescriptor" }, + "disclosure": { "type": "string", "const": "content-free" } + }, + "required": ["role", "producer", "resource", "disclosure"] + }, "Identifier": { "type": "string", "pattern": "^[a-zA-Z0-9][a-zA-Z0-9._:/-]{1,127}$" @@ -1005,7 +1055,7 @@ "additionalProperties": false, "properties": { "name": { "type": "string", "const": "agentic-strata" }, - "version": { "type": "string", "const": "0.1.0-alpha.1" }, + "version": { "type": "string", "const": "0.2.0-alpha.1" }, "revision": { "type": "string", "const": "conformance-2026-07-17.3" @@ -1043,6 +1093,112 @@ "digest" ] }, + "RunBundleResourceDescriptor": { + "allOf": [ + { "$ref": "#/$defs/ResourceDescriptor" }, + { + "type": "object", + "properties": { + "name": { "type": "string", "const": "agentic-strata-run-bundle" }, + "mediaType": { + "type": "string", + "const": "application/vnd.aantenore.agenticstrata.run-bundle+json" + } + } + } + ] + }, + "ConformanceReportResourceDescriptor": { + "allOf": [ + { "$ref": "#/$defs/ResourceDescriptor" }, + { + "type": "object", + "properties": { + "name": { "type": "string", "const": "agentic-strata-conformance-report" }, + "mediaType": { + "type": "string", + "const": "application/vnd.aantenore.agenticstrata.conformance-report+json" + } + } + } + ] + }, + "OasfRecordResourceDescriptor": { + "allOf": [ + { "$ref": "#/$defs/ResourceDescriptor" }, + { + "type": "object", + "properties": { + "name": { "type": "string", "const": "oasf-agent-record" } + } + } + ] + }, + "ExecutionPassportPredicate": { + "type": "object", + "additionalProperties": false, + "properties": { + "apiVersion": { "$ref": "#/$defs/ApiVersion" }, + "kind": { "type": "string", "const": "ExecutionPassport" }, + "runId": { "$ref": "#/$defs/Identifier" }, + "profile": { + "type": "string", + "enum": ["core", "local-private", "enterprise", "regulated", "distributed"] + }, + "runStatus": { + "type": "string", + "enum": ["completed", "failed", "incomplete"] + }, + "conformanceStatus": { "type": "string", "enum": ["pass", "fail"] }, + "evaluatorDigest": { "$ref": "#/$defs/Digest" }, + "terminalReceiptDigest": { + "oneOf": [{ "$ref": "#/$defs/Digest" }, { "type": "null" }] + }, + "issuedAt": { "type": "string", "format": "date-time" }, + "executionEvidence": { + "type": "array", + "maxItems": 64, + "uniqueItems": true, + "items": { "$ref": "#/$defs/ExecutionEvidenceBinding" } + } + }, + "required": [ + "apiVersion", + "kind", + "runId", + "profile", + "runStatus", + "conformanceStatus", + "evaluatorDigest", + "terminalReceiptDigest", + "issuedAt", + "executionEvidence" + ] + }, + "ExecutionPassport": { + "type": "object", + "additionalProperties": false, + "properties": { + "_type": { "type": "string", "const": "https://in-toto.io/Statement/v1" }, + "subject": { + "type": "array", + "minItems": 3, + "maxItems": 3, + "prefixItems": [ + { "$ref": "#/$defs/RunBundleResourceDescriptor" }, + { "$ref": "#/$defs/ConformanceReportResourceDescriptor" }, + { "$ref": "#/$defs/OasfRecordResourceDescriptor" } + ], + "items": false + }, + "predicateType": { + "type": "string", + "const": "https://github.com/aantenore/AgenticStrata/tree/main/docs/spec/attestations/execution-passport/v1" + }, + "predicate": { "$ref": "#/$defs/ExecutionPassportPredicate" } + }, + "required": ["_type", "subject", "predicateType", "predicate"] + }, "AdapterMapping": { "type": "object", "additionalProperties": false, @@ -1052,7 +1208,16 @@ "mappingId": { "$ref": "#/$defs/Identifier" }, "protocol": { "type": "string", - "enum": ["ag-ui", "a2a", "mcp", "opentelemetry-genai", "cloudevents", "policy-engine"] + "enum": [ + "ag-ui", + "a2a", + "mcp", + "opentelemetry-genai", + "cloudevents", + "policy-engine", + "oasf", + "stagefabric" + ] }, "protocolVersion": { "type": "string", "minLength": 1 }, "direction": { "type": "string", "enum": ["inbound", "outbound", "bidirectional"] }, diff --git a/scripts/audit-repository.mjs b/scripts/audit-repository.mjs index 22a169e..3a9d31a 100644 --- a/scripts/audit-repository.mjs +++ b/scripts/audit-repository.mjs @@ -9,6 +9,7 @@ const blockedFragments = [ ["aantenore", "re", "ply"].join(""), ["no", "re", "ply"].join("") ]; +const configuredHistoryRef = process.env.AGENTIC_STRATA_AUDIT_HISTORY_REF?.trim() ?? ""; function git(args, options = {}) { return execFileSync("git", args, { @@ -46,10 +47,21 @@ try { } if (hasHistory) { + const historyRevision = configuredHistoryRef === "" ? ["--all"] : [configuredHistoryRef]; + if (configuredHistoryRef !== "") { + if (!/^[a-f0-9]{40,64}$/i.test(configuredHistoryRef)) { + throw new Error("Configured audit history ref must be a full commit object id."); + } + try { + git(["cat-file", "-e", `${configuredHistoryRef}^{commit}`]); + } catch { + throw new Error("Configured audit history ref is not an available commit."); + } + } const metadata = git([ "log", "--format=%H%x00%an%x00%ae%x00%cn%x00%ce%x00%s", - "--all" + ...historyRevision ]); assertClean("reachable commit metadata", metadata); for (const row of metadata.split("\n").filter(Boolean)) { @@ -79,7 +91,7 @@ if (hasHistory) { } } - const reachableObjects = git(["rev-list", "--objects", "--all"]) + const reachableObjects = git(["rev-list", "--objects", ...historyRevision]) .split("\n") .filter(Boolean); const visited = new Set(); @@ -110,6 +122,7 @@ console.log( { filesScanned: files.length, historyScanned: hasHistory, + historyScope: configuredHistoryRef === "" ? "all refs" : configuredHistoryRef, historyBlobsScanned: hasHistory ? "all reachable text blobs up to 16 MiB" : "none", canonicalIdentity: `${canonicalName} <${canonicalEmail}>` }, diff --git a/scripts/package-smoke.mjs b/scripts/package-smoke.mjs index 5e4efc1..9f4f620 100644 --- a/scripts/package-smoke.mjs +++ b/scripts/package-smoke.mjs @@ -35,11 +35,12 @@ try { ); const probe = [ - 'import { runConformance, runDemo } from "agentic-strata";', + 'import { createExecutionPassport, runConformance, runDemo, validateDocument } from "agentic-strata";', 'const result = runDemo();', 'const report = runConformance(result.bundle, "enterprise");', - 'if (report.status !== "pass" || report.runStatus !== "completed" || result.readBack !== true) process.exit(2);', - 'console.log(JSON.stringify({ status: report.status, runStatus: report.runStatus, readBack: result.readBack }));' + 'const passport = createExecutionPassport({ bundle: result.bundle, report, oasfRecord: { opaqueExternalRecord: true }, oasfMediaType: "application/json" });', + 'if (report.status !== "pass" || report.runStatus !== "completed" || result.readBack !== true || !validateDocument(passport).valid || passport.subject.length !== 3) process.exit(2);', + 'console.log(JSON.stringify({ status: report.status, runStatus: report.runStatus, readBack: result.readBack, passport: "pass" }));' ].join("\n"); const output = run(process.execPath, ["--input-type=module", "--eval", probe], temporary); process.stdout.write(output); @@ -51,13 +52,37 @@ try { process.platform === "win32" ? "agentic-strata.cmd" : "agentic-strata" ); const version = run(executable, ["--version"], temporary).trim(); - if (version !== "0.1.0-alpha.1") { + if (version !== "0.2.0-alpha.1") { throw new Error(`Installed CLI reported unexpected version: ${version}`); } const cliOutput = join(temporary, "cli-demo"); run(executable, ["demo", "--output", cliOutput, "--profile", "enterprise"], temporary); run(executable, ["validate", join(cliOutput, "run.bundle.json")], temporary); - console.log(JSON.stringify({ cliVersion: version, cliDemo: "pass" })); + const oasfRecord = join(temporary, "external-oasf.json"); + const passportOutput = join(temporary, "execution-passport.json"); + writeFileSync(oasfRecord, '{"opaqueExternalRecord":true}\n', "utf8"); + run( + executable, + [ + "passport", + join(cliOutput, "run.bundle.json"), + "--report", + join(cliOutput, "conformance-report.json"), + "--oasf-record", + oasfRecord, + "--oasf-media-type", + "application/json", + "--issued-at", + "2026-07-17T12:00:01.000Z", + "--output", + passportOutput + ], + temporary + ); + run(executable, ["validate", passportOutput], temporary); + console.log( + JSON.stringify({ cliVersion: version, cliDemo: "pass", cliPassport: "pass" }) + ); } finally { rmSync(temporary, { recursive: true, force: true }); } diff --git a/src/adapters/documents.ts b/src/adapters/documents.ts index dbe360e..82167a5 100644 --- a/src/adapters/documents.ts +++ b/src/adapters/documents.ts @@ -44,6 +44,11 @@ export function writeJson(path: string, value: unknown): void { writeFileSync(path, `${JSON.stringify(value, null, 2)}\n`, "utf8"); } +export function writeCanonicalJson(path: string, value: unknown): void { + mkdirSync(dirname(path), { recursive: true }); + writeFileSync(path, canonicalize(value), "utf8"); +} + export function writeText(path: string, value: string): void { mkdirSync(dirname(path), { recursive: true }); writeFileSync(path, value.endsWith("\n") ? value : `${value}\n`, "utf8"); diff --git a/src/cli-program.ts b/src/cli-program.ts new file mode 100644 index 0000000..a3155cd --- /dev/null +++ b/src/cli-program.ts @@ -0,0 +1,241 @@ +import { resolve } from "node:path"; + +import { Command, InvalidArgumentError } from "commander"; + +import { + readDocument, + writeCanonicalJson, + writeJson, + writeText +} from "./adapters/documents.js"; +import { validateAs, validateDocument } from "./contracts/registry.js"; +import { CONFORMANCE_PROFILES, EVALUATOR_VERSION } from "./contracts/types.js"; +import type { + ApplicationManifest, + ConformanceProfile, + ConformanceReport, + ExecutionEvidenceBinding, + RunBundle, + ValidationResult +} from "./contracts/types.js"; +import { loadProfileConfiguration } from "./conformance/config.js"; +import { reportIssues, runConformance } from "./conformance/engine.js"; +import { lintManifest } from "./conformance/manifest.js"; +import { canonicalize } from "./core/canonical.js"; +import { explainRun } from "./core/explain.js"; +import { createExecutionPassport } from "./core/passport.js"; +import { evidenceIndex, verifyTraceChain } from "./core/receipts.js"; +import { runDemo } from "./demo/demo.js"; + +function printValidation(result: ValidationResult): void { + if (result.valid) { + console.log("valid"); + return; + } + for (const issue of result.issues) { + console.error(`${issue.code} ${issue.path}: ${issue.message}`); + } + process.exitCode = 2; +} + +function parseProfile(value: string): ConformanceProfile { + if (!(CONFORMANCE_PROFILES as readonly string[]).includes(value)) { + throw new InvalidArgumentError(`Profile must be one of: ${CONFORMANCE_PROFILES.join(", ")}.`); + } + return value as ConformanceProfile; +} + +function collectPath(value: string, previous: string[]): string[] { + return [...previous, value]; +} + +function requireBundle(path: string): RunBundle { + const document = readDocument(path); + const validation = validateAs("RunBundle", document); + if (!validation.valid) { + printValidation(validation); + throw new Error("The input is not a valid RunBundle."); + } + return document as RunBundle; +} + +function requireReport(path: string): ConformanceReport { + const document = readDocument(path); + const validation = validateAs("ConformanceReport", document); + if (!validation.valid) { + printValidation(validation); + throw new Error("The input is not a valid ConformanceReport."); + } + return document as ConformanceReport; +} + +function requireExecutionEvidence(path: string): ExecutionEvidenceBinding { + const document = readDocument(path); + const validation = validateAs("ExecutionEvidenceBinding", document); + if (!validation.valid) { + printValidation(validation); + throw new Error("The input is not a content-free ExecutionEvidenceBinding."); + } + return document as ExecutionEvidenceBinding; +} + +interface PassportCommandOptions { + report: string; + oasfRecord: string; + oasfMediaType: string; + bundleUri?: string; + reportUri?: string; + oasfUri?: string; + executionEvidence: string[]; + issuedAt?: string; + output?: string; +} + +export function createCliProgram(): Command { + const program = new Command() + .name("agentic-strata") + .description("Validate and explain executable AgenticStrata contracts.") + .version(EVALUATOR_VERSION); + + program + .command("validate") + .description("Validate one JSON or YAML contract against the language-neutral schema.") + .argument("") + .action((file: string) => { + printValidation(validateDocument(readDocument(resolve(file)))); + }); + + program + .command("lint") + .description("Check stratum completeness, ordinals, planes, and dependency direction.") + .argument("") + .action((file: string) => { + const document = readDocument(resolve(file)); + const schema = validateAs("ApplicationManifest", document); + if (!schema.valid) { + printValidation(schema); + return; + } + printValidation(lintManifest(document as ApplicationManifest)); + }); + + program + .command("conformance") + .description("Evaluate a manifest plus real runtime artifacts and receipt chain.") + .argument("") + .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") + .option("--profiles ", "extend the non-removable packaged profile baseline") + .option("-o, --output ", "write the report as JSON") + .action( + ( + file: string, + options: { profile: ConformanceProfile; profiles?: string; output?: string } + ) => { + const bundle = requireBundle(resolve(file)); + const configuration = loadProfileConfiguration( + options.profiles === undefined ? undefined : resolve(options.profiles) + ); + const report = runConformance(bundle, options.profile, { configuration }); + console.log(JSON.stringify(report, null, 2)); + if (options.output !== undefined) { + writeJson(resolve(options.output), report); + } + if (report.status === "fail") { + printValidation({ valid: false, issues: reportIssues(report) }); + } + } + ); + + program + .command("passport") + .description("Create a canonical in-toto Execution Passport without embedding external content.") + .argument("") + .requiredOption("--report ", "sealed ConformanceReport bound to the RunBundle") + .requiredOption("--oasf-record ", "opaque external OASF JSON record to bind by digest") + .requiredOption("--oasf-media-type ", "media type declared by the OASF record owner") + .option("--bundle-uri ", "stable HTTPS or URN RunBundle reference") + .option("--report-uri ", "stable HTTPS or URN ConformanceReport reference") + .option("--oasf-uri ", "stable HTTPS or URN OASF record reference") + .option( + "--execution-evidence ", + "content-free ExecutionEvidenceBinding file; repeat for multiple providers", + collectPath, + [] + ) + .option("--issued-at ", "self-asserted passport issuance time") + .option("-o, --output ", "write exact RFC 8785 Statement bytes") + .action((file: string, options: PassportCommandOptions) => { + const passport = createExecutionPassport({ + bundle: requireBundle(resolve(file)), + report: requireReport(resolve(options.report)), + oasfRecord: readDocument(resolve(options.oasfRecord)), + oasfMediaType: options.oasfMediaType, + subjectUris: { + ...(options.bundleUri === undefined ? {} : { runBundle: options.bundleUri }), + ...(options.reportUri === undefined + ? {} + : { conformanceReport: options.reportUri }), + ...(options.oasfUri === undefined ? {} : { oasfRecord: options.oasfUri }) + }, + executionEvidence: options.executionEvidence.map((path) => + requireExecutionEvidence(resolve(path)) + ), + ...(options.issuedAt === undefined ? {} : { issuedAt: options.issuedAt }) + }); + if (options.output === undefined) { + process.stdout.write(canonicalize(passport)); + } else { + writeCanonicalJson(resolve(options.output), passport); + } + }); + + program + .command("replay") + .description("Replay the hash-linked runtime receipt chain and detect tampering or missing evidence.") + .argument("") + .action((file: string) => { + const bundle = requireBundle(resolve(file)); + const report = verifyTraceChain(bundle.traceEvents, evidenceIndex(bundle)); + console.log(JSON.stringify(report, null, 2)); + if (!report.valid) { + process.exitCode = 2; + } + }); + + program + .command("explain") + .description("Render a human-readable account of request, decisions, authority, effects, and evidence.") + .argument("") + .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") + .action((file: string, options: { profile: ConformanceProfile }) => { + const bundle = requireBundle(resolve(file)); + console.log(explainRun(bundle, runConformance(bundle, options.profile))); + }); + + program + .command("demo") + .description("Execute a neutral prepare/commit/read-back workflow with approval and evidence.") + .option("-o, --output ", "artifact directory", ".tmp/agentic-strata-demo") + .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") + .action((options: { output: string; profile: ConformanceProfile }) => { + const result = runDemo(); + const report = runConformance(result.bundle, options.profile, { + generatedAt: "2026-07-17T12:00:00.000Z" + }); + const output = resolve(options.output); + const explanation = explainRun(result.bundle, report); + writeJson(`${output}/run.bundle.json`, result.bundle); + writeJson(`${output}/conformance-report.json`, report); + writeText(`${output}/explanation.md`, explanation); + console.log(explanation); + if (report.status === "fail" || !result.duplicateWasSuppressed) { + process.exitCode = 2; + } + }); + + return program; +} + +export async function runCli(argv: readonly string[] = process.argv): Promise { + await createCliProgram().parseAsync([...argv]); +} diff --git a/src/cli.ts b/src/cli.ts index 0f70594..6b15fba 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,151 +1,7 @@ #!/usr/bin/env node -import { resolve } from "node:path"; +import { runCli } from "./cli-program.js"; -import { Command, InvalidArgumentError } from "commander"; - -import { readDocument, writeJson, writeText } from "./adapters/documents.js"; -import { validateAs, validateDocument } from "./contracts/registry.js"; -import { CONFORMANCE_PROFILES, EVALUATOR_VERSION } from "./contracts/types.js"; -import type { - ApplicationManifest, - ConformanceProfile, - RunBundle, - ValidationResult -} from "./contracts/types.js"; -import { loadProfileConfiguration } from "./conformance/config.js"; -import { reportIssues, runConformance } from "./conformance/engine.js"; -import { lintManifest } from "./conformance/manifest.js"; -import { runDemo } from "./demo/demo.js"; -import { explainRun } from "./core/explain.js"; -import { evidenceIndex, verifyTraceChain } from "./core/receipts.js"; - -function printValidation(result: ValidationResult): void { - if (result.valid) { - console.log("valid"); - return; - } - for (const issue of result.issues) { - console.error(`${issue.code} ${issue.path}: ${issue.message}`); - } - process.exitCode = 2; -} - -function parseProfile(value: string): ConformanceProfile { - if (!(CONFORMANCE_PROFILES as readonly string[]).includes(value)) { - throw new InvalidArgumentError(`Profile must be one of: ${CONFORMANCE_PROFILES.join(", ")}.`); - } - return value as ConformanceProfile; -} - -function requireBundle(path: string): RunBundle { - const document = readDocument(path); - const validation = validateAs("RunBundle", document); - if (!validation.valid) { - printValidation(validation); - throw new Error("The input is not a valid RunBundle."); - } - return document as RunBundle; -} - -const program = new Command() - .name("agentic-strata") - .description("Validate and explain executable AgenticStrata contracts.") - .version(EVALUATOR_VERSION); - -program - .command("validate") - .description("Validate one JSON or YAML contract against the language-neutral schema.") - .argument("") - .action((file: string) => { - printValidation(validateDocument(readDocument(resolve(file)))); - }); - -program - .command("lint") - .description("Check stratum completeness, ordinals, planes, and dependency direction.") - .argument("") - .action((file: string) => { - const document = readDocument(resolve(file)); - const schema = validateAs("ApplicationManifest", document); - if (!schema.valid) { - printValidation(schema); - return; - } - printValidation(lintManifest(document as ApplicationManifest)); - }); - -program - .command("conformance") - .description("Evaluate a manifest plus real runtime artifacts and receipt chain.") - .argument("") - .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") - .option("--profiles ", "extend the non-removable packaged profile baseline") - .option("-o, --output ", "write the report as JSON") - .action( - ( - file: string, - options: { profile: ConformanceProfile; profiles?: string; output?: string } - ) => { - const bundle = requireBundle(resolve(file)); - const configuration = loadProfileConfiguration( - options.profiles === undefined ? undefined : resolve(options.profiles) - ); - const report = runConformance(bundle, options.profile, { configuration }); - console.log(JSON.stringify(report, null, 2)); - if (options.output !== undefined) { - writeJson(resolve(options.output), report); - } - if (report.status === "fail") { - printValidation({ valid: false, issues: reportIssues(report) }); - } - } - ); - -program - .command("replay") - .description("Replay the hash-linked runtime receipt chain and detect tampering or missing evidence.") - .argument("") - .action((file: string) => { - const bundle = requireBundle(resolve(file)); - const report = verifyTraceChain(bundle.traceEvents, evidenceIndex(bundle)); - console.log(JSON.stringify(report, null, 2)); - if (!report.valid) { - process.exitCode = 2; - } - }); - -program - .command("explain") - .description("Render a human-readable account of request, decisions, authority, effects, and evidence.") - .argument("") - .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") - .action((file: string, options: { profile: ConformanceProfile }) => { - const bundle = requireBundle(resolve(file)); - console.log(explainRun(bundle, runConformance(bundle, options.profile))); - }); - -program - .command("demo") - .description("Execute a neutral prepare/commit/read-back workflow with approval and evidence.") - .option("-o, --output ", "artifact directory", ".tmp/agentic-strata-demo") - .option("-p, --profile ", "conformance profile", parseProfile, "enterprise") - .action((options: { output: string; profile: ConformanceProfile }) => { - const result = runDemo(); - const report = runConformance(result.bundle, options.profile, { - generatedAt: "2026-07-17T12:00:00.000Z" - }); - const output = resolve(options.output); - const explanation = explainRun(result.bundle, report); - writeJson(`${output}/run.bundle.json`, result.bundle); - writeJson(`${output}/conformance-report.json`, report); - writeText(`${output}/explanation.md`, explanation); - console.log(explanation); - if (report.status === "fail" || !result.duplicateWasSuppressed) { - process.exitCode = 2; - } - }); - -program.parseAsync().catch((error: unknown) => { +runCli().catch((error: unknown) => { console.error(error instanceof Error ? error.message : String(error)); process.exitCode ??= 1; }); diff --git a/src/contracts/registry.ts b/src/contracts/registry.ts index 39922e4..57a1974 100644 --- a/src/contracts/registry.ts +++ b/src/contracts/registry.ts @@ -5,11 +5,14 @@ import { Ajv2020, type ErrorObject, type ValidateFunction } from "ajv/dist/2020. import addFormatsImport from "ajv-formats"; import type { FormatsPlugin } from "ajv-formats"; -import type { - CapabilityContract, - ContractType, - ValidationIssue, - ValidationResult +import { + EXECUTION_PASSPORT_PREDICATE_TYPE, + IN_TOTO_STATEMENT_V1_TYPE, + type CapabilityContract, + type ContractType, + type SchemaDefinition, + type ValidationIssue, + type ValidationResult } from "./types.js"; const schemaPath = fileURLToPath( @@ -30,14 +33,14 @@ ajv.addSchema(schema, schemaId); const validators = new Map(); -function validatorFor(contractType: ContractType): ValidateFunction { - const cached = validators.get(contractType); +function validatorFor(schemaDefinition: SchemaDefinition): ValidateFunction { + const cached = validators.get(schemaDefinition); if (cached !== undefined) { return cached; } - const validator = ajv.compile({ $ref: `${schemaId}#/$defs/${contractType}` }); - validators.set(contractType, validator); + const validator = ajv.compile({ $ref: `${schemaId}#/$defs/${schemaDefinition}` }); + validators.set(schemaDefinition, validator); return validator; } @@ -49,8 +52,11 @@ function toIssues(errors: ErrorObject[] | null | undefined): ValidationIssue[] { })); } -export function validateAs(contractType: ContractType, document: unknown): ValidationResult { - const validator = validatorFor(contractType); +export function validateAs( + schemaDefinition: SchemaDefinition, + document: unknown +): ValidationResult { + const validator = validatorFor(schemaDefinition); const valid = validator(document); return { valid, issues: valid ? [] : toIssues(validator.errors) }; } @@ -63,7 +69,40 @@ export function validateDocument(document: unknown): ValidationResult { }; } - const contractType = (document as { contractType?: unknown }).contractType; + const candidate = document as { + _type?: unknown; + predicateType?: unknown; + contractType?: unknown; + }; + if ("_type" in candidate || "predicateType" in candidate) { + if (candidate._type !== IN_TOTO_STATEMENT_V1_TYPE) { + return { + valid: false, + issues: [ + { + path: "/_type", + code: "const", + message: "The in-toto Statement type is missing or unsupported." + } + ] + }; + } + if (candidate.predicateType !== EXECUTION_PASSPORT_PREDICATE_TYPE) { + return { + valid: false, + issues: [ + { + path: "/predicateType", + code: "const", + message: "The in-toto predicate type is missing or unsupported." + } + ] + }; + } + return validateAs("ExecutionPassport", document); + } + + const contractType = candidate.contractType; if (typeof contractType !== "string" || !contractTypes.has(contractType as ContractType)) { return { valid: false, diff --git a/src/contracts/types.ts b/src/contracts/types.ts index ed8a7f0..ba71bbd 100644 --- a/src/contracts/types.ts +++ b/src/contracts/types.ts @@ -1,8 +1,21 @@ export const API_VERSION = "agenticstrata.dev/v1" as const; export const EVALUATOR_NAME = "agentic-strata" as const; -export const EVALUATOR_VERSION = "0.1.0-alpha.1" as const; +export const EVALUATOR_VERSION = "0.2.0-alpha.1" as const; export const EVALUATOR_REVISION = "conformance-2026-07-17.3" as const; +export const IN_TOTO_STATEMENT_V1_TYPE = "https://in-toto.io/Statement/v1" as const; +export const EXECUTION_PASSPORT_PREDICATE_TYPE = + "https://github.com/aantenore/AgenticStrata/tree/main/docs/spec/attestations/execution-passport/v1" as const; +export const EXECUTION_PASSPORT_SUBJECTS = { + runBundle: "agentic-strata-run-bundle", + conformanceReport: "agentic-strata-conformance-report", + oasfRecord: "oasf-agent-record" +} as const; +export const EXECUTION_PASSPORT_MEDIA_TYPES = { + runBundle: "application/vnd.aantenore.agenticstrata.run-bundle+json", + conformanceReport: "application/vnd.aantenore.agenticstrata.conformance-report+json" +} as const; + export const STRATA = [ "interaction", "intent-outcome", @@ -376,11 +389,73 @@ export interface ConformanceReport { digest: Digest; } +export interface ResourceDescriptor { + name: string; + digest: { + sha256: Digest; + }; + mediaType: string; + uri?: string; +} + +export interface RunBundleResourceDescriptor extends ResourceDescriptor { + name: typeof EXECUTION_PASSPORT_SUBJECTS.runBundle; + mediaType: typeof EXECUTION_PASSPORT_MEDIA_TYPES.runBundle; +} + +export interface ConformanceReportResourceDescriptor extends ResourceDescriptor { + name: typeof EXECUTION_PASSPORT_SUBJECTS.conformanceReport; + mediaType: typeof EXECUTION_PASSPORT_MEDIA_TYPES.conformanceReport; +} + +export interface OasfRecordResourceDescriptor extends ResourceDescriptor { + name: typeof EXECUTION_PASSPORT_SUBJECTS.oasfRecord; +} + +export interface ExecutionEvidenceBinding { + role: "execution-placement"; + producer: string; + resource: ResourceDescriptor; + disclosure: "content-free"; +} + +export interface ExecutionPassportPredicate { + apiVersion: ApiVersion; + kind: "ExecutionPassport"; + runId: string; + profile: ConformanceProfile; + runStatus: ConformanceReport["runStatus"]; + conformanceStatus: ConformanceReport["status"]; + evaluatorDigest: Digest; + terminalReceiptDigest: Digest | null; + issuedAt: string; + executionEvidence: ExecutionEvidenceBinding[]; +} + +export interface ExecutionPassport { + _type: typeof IN_TOTO_STATEMENT_V1_TYPE; + subject: [ + RunBundleResourceDescriptor, + ConformanceReportResourceDescriptor, + OasfRecordResourceDescriptor + ]; + predicateType: typeof EXECUTION_PASSPORT_PREDICATE_TYPE; + predicate: ExecutionPassportPredicate; +} + export interface AdapterMapping { contractType: "AdapterMapping"; apiVersion: ApiVersion; mappingId: string; - protocol: "ag-ui" | "a2a" | "mcp" | "opentelemetry-genai" | "cloudevents" | "policy-engine"; + protocol: + | "ag-ui" + | "a2a" + | "mcp" + | "opentelemetry-genai" + | "cloudevents" + | "policy-engine" + | "oasf" + | "stagefabric"; protocolVersion: string; direction: "inbound" | "outbound" | "bidirectional"; contractBindings: Array<{ @@ -432,6 +507,12 @@ export type ContractType = | AdapterMapping["contractType"] | RunBundle["contractType"]; +export type SchemaDefinition = + | ContractType + | "ResourceDescriptor" + | "ExecutionEvidenceBinding" + | "ExecutionPassport"; + export interface ValidationIssue { path: string; code: string; diff --git a/src/core/passport.ts b/src/core/passport.ts new file mode 100644 index 0000000..66cebed --- /dev/null +++ b/src/core/passport.ts @@ -0,0 +1,290 @@ +import { validateAs } from "../contracts/registry.js"; +import { + API_VERSION, + EXECUTION_PASSPORT_MEDIA_TYPES, + EXECUTION_PASSPORT_PREDICATE_TYPE, + EXECUTION_PASSPORT_SUBJECTS, + IN_TOTO_STATEMENT_V1_TYPE, + type ConformanceReport, + type ConformanceReportResourceDescriptor, + type ExecutionEvidenceBinding, + type ExecutionPassport, + type OasfRecordResourceDescriptor, + type ResourceDescriptor, + type RunBundle, + type RunBundleResourceDescriptor, + type ValidationResult +} from "../contracts/types.js"; +import { digestValue, verifySeal } from "./canonical.js"; +import { evidenceIndex, verifyTraceChain } from "./receipts.js"; + +export interface ExecutionPassportSubjectUris { + runBundle?: string; + conformanceReport?: string; + oasfRecord?: string; +} + +export interface CreateExecutionPassportInput { + bundle: RunBundle; + report: ConformanceReport; + oasfRecord: unknown; + oasfMediaType: string; + subjectUris?: ExecutionPassportSubjectUris; + executionEvidence?: readonly ExecutionEvidenceBinding[]; + issuedAt?: string; +} + +function validationMessage(result: ValidationResult): string { + return result.issues + .map((issue) => `${issue.path} ${issue.code}: ${issue.message}`) + .join(", "); +} + +function requireValid(label: string, result: ValidationResult): void { + if (!result.valid) { + throw new Error(`${label} is invalid: ${validationMessage(result)}`); + } +} + +function copyResourceDescriptor(resource: ResourceDescriptor): ResourceDescriptor { + requireValid( + `Execution evidence resource ${resource.name}`, + validateAs("ResourceDescriptor", resource) + ); + return { + name: resource.name, + digest: { sha256: resource.digest.sha256 }, + mediaType: resource.mediaType, + ...(resource.uri === undefined ? {} : { uri: resource.uri }) + }; +} + +function copyExecutionEvidence( + bindings: readonly ExecutionEvidenceBinding[] +): ExecutionEvidenceBinding[] { + const seenDigests = new Set(); + return bindings.map((binding, index) => { + requireValid( + `Execution evidence binding ${index}`, + validateAs("ExecutionEvidenceBinding", binding) + ); + const digest = binding.resource.digest.sha256; + if (seenDigests.has(digest)) { + throw new Error(`Execution evidence digest ${digest} is duplicated.`); + } + seenDigests.add(digest); + return { + role: binding.role, + producer: binding.producer, + resource: copyResourceDescriptor(binding.resource), + disclosure: binding.disclosure + }; + }); +} + +function requireOpaqueJsonRecord(value: unknown): asserts value is Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error("The external OASF record must be a JSON object."); + } +} + +function requireConsistentRunIdentity(bundle: RunBundle): string { + const runId = bundle.execution.runId; + const observedRunIds = [ + bundle.budgetUsage.runId, + ...bundle.runtimeBoundaries.map((item) => item.runId), + ...bundle.criterionResults.map((item) => item.runId), + ...bundle.traceEvents.map((item) => item.runId) + ]; + if (observedRunIds.some((candidate) => candidate !== runId)) { + throw new Error("The RunBundle contains inconsistent run identifiers."); + } + return runId; +} + +function terminalBinding( + bundle: RunBundle, + report: ConformanceReport +): string | null { + const terminalEvents = bundle.traceEvents.filter( + (event) => event.eventType === "run.completed" || event.eventType === "run.failed" + ); + const terminal = bundle.traceEvents.at(-1); + const hasOneFinalTerminal = terminalEvents.length === 1 && terminalEvents[0] === terminal; + const expectedRunStatus = !hasOneFinalTerminal + ? "incomplete" + : terminal?.eventType === "run.completed" + ? "completed" + : "failed"; + if (report.runStatus !== expectedRunStatus) { + throw new Error( + `The ConformanceReport run status ${report.runStatus} does not match ${expectedRunStatus}.` + ); + } + return hasOneFinalTerminal ? (terminal?.digest ?? null) : null; +} + +function requireBoundReport(bundle: RunBundle, report: ConformanceReport): void { + requireValid("RunBundle", validateAs("RunBundle", bundle)); + requireValid("ConformanceReport", validateAs("ConformanceReport", report)); + + if (!verifySeal(report)) { + throw new Error("The ConformanceReport seal does not match its content."); + } + if (!verifySeal(report.evaluator)) { + throw new Error("The ConformanceReport evaluator seal does not match its content."); + } + + const checkIds = report.checks.map((check) => check.id); + if (new Set(checkIds).size !== checkIds.length) { + throw new Error("The ConformanceReport contains duplicate check identifiers."); + } + if (report.rulesDigest !== digestValue(checkIds)) { + throw new Error("The ConformanceReport rules digest does not match its ordered checks."); + } + const expectedStatus = report.checks.some((check) => check.status === "fail") + ? "fail" + : "pass"; + if (report.status !== expectedStatus) { + throw new Error( + `The ConformanceReport status ${report.status} does not match ${expectedStatus}.` + ); + } + + const bundleDigest = digestValue(bundle); + if (report.runBundleDigest !== bundleDigest) { + throw new Error("The ConformanceReport is not bound to the supplied RunBundle."); + } + if (report.manifestDigest !== digestValue(bundle.manifest)) { + throw new Error("The ConformanceReport manifest binding does not match the RunBundle."); + } + + const replay = verifyTraceChain(bundle.traceEvents, evidenceIndex(bundle)); + if (!replay.valid) { + throw new Error(`The RunBundle receipt chain is invalid: ${validationMessage(replay)}`); + } +} + +interface ObservedTimestamp { + label: string; + value: string; +} + +function observedTimestamps(bundle: RunBundle): ObservedTimestamp[] { + return [ + { label: "budget usage", value: bundle.budgetUsage.observedAt }, + ...bundle.traceEvents.map((event) => ({ + label: `trace event ${event.eventId}`, + value: event.occurredAt + })), + ...bundle.runtimeBoundaries.map((boundary) => ({ + label: `runtime boundary ${boundary.boundaryEvidenceId}`, + value: boundary.observedAt + })), + ...bundle.criterionResults.map((criterion) => ({ + label: `criterion result ${criterion.resultId}`, + value: criterion.observedAt + })), + ...bundle.decisions.map((decision) => ({ + label: `decision ${decision.decisionId}`, + value: decision.createdAt + })), + ...bundle.attestations.map((attestation) => ({ + label: `artifact attestation ${attestation.attestationId}`, + value: attestation.createdAt + })), + ...bundle.approvals.map((approval) => ({ + label: `approval ${approval.approvalId}`, + value: approval.issuedAt + })) + ]; +} + +function requireReportAfterObservedEvidence( + bundle: RunBundle, + report: ConformanceReport +): void { + const reportMillis = Date.parse(report.generatedAt); + if (!Number.isFinite(reportMillis)) { + throw new Error("ConformanceReport generatedAt must be a valid date-time."); + } + for (const observed of observedTimestamps(bundle)) { + const observedMillis = Date.parse(observed.value); + if (!Number.isFinite(observedMillis)) { + throw new Error(`${observed.label} must have a valid observed date-time.`); + } + if (reportMillis < observedMillis) { + throw new Error(`The ConformanceReport cannot precede ${observed.label}.`); + } + } +} + +function requireIssuedAt(issuedAt: string, report: ConformanceReport): void { + const issuedAtMillis = Date.parse(issuedAt); + const reportMillis = Date.parse(report.generatedAt); + if (!Number.isFinite(issuedAtMillis)) { + throw new Error("Execution Passport issuedAt must be a valid date-time."); + } + if (issuedAtMillis < reportMillis) { + throw new Error("Execution Passport issuedAt cannot precede the ConformanceReport."); + } +} + +export function createExecutionPassport( + input: CreateExecutionPassportInput +): ExecutionPassport { + requireBoundReport(input.bundle, input.report); + requireReportAfterObservedEvidence(input.bundle, input.report); + requireOpaqueJsonRecord(input.oasfRecord); + + const runId = requireConsistentRunIdentity(input.bundle); + const terminalReceiptDigest = terminalBinding(input.bundle, input.report); + const issuedAt = input.issuedAt ?? new Date().toISOString(); + requireIssuedAt(issuedAt, input.report); + + const runBundleSubject: RunBundleResourceDescriptor = { + name: EXECUTION_PASSPORT_SUBJECTS.runBundle, + digest: { sha256: digestValue(input.bundle) }, + mediaType: EXECUTION_PASSPORT_MEDIA_TYPES.runBundle, + ...(input.subjectUris?.runBundle === undefined + ? {} + : { uri: input.subjectUris.runBundle }) + }; + const conformanceReportSubject: ConformanceReportResourceDescriptor = { + name: EXECUTION_PASSPORT_SUBJECTS.conformanceReport, + digest: { sha256: digestValue(input.report) }, + mediaType: EXECUTION_PASSPORT_MEDIA_TYPES.conformanceReport, + ...(input.subjectUris?.conformanceReport === undefined + ? {} + : { uri: input.subjectUris.conformanceReport }) + }; + const oasfRecordSubject: OasfRecordResourceDescriptor = { + name: EXECUTION_PASSPORT_SUBJECTS.oasfRecord, + digest: { sha256: digestValue(input.oasfRecord) }, + mediaType: input.oasfMediaType, + ...(input.subjectUris?.oasfRecord === undefined + ? {} + : { uri: input.subjectUris.oasfRecord }) + }; + + const passport: ExecutionPassport = { + _type: IN_TOTO_STATEMENT_V1_TYPE, + subject: [runBundleSubject, conformanceReportSubject, oasfRecordSubject], + predicateType: EXECUTION_PASSPORT_PREDICATE_TYPE, + predicate: { + apiVersion: API_VERSION, + kind: "ExecutionPassport", + runId, + profile: input.report.profile, + runStatus: input.report.runStatus, + conformanceStatus: input.report.status, + evaluatorDigest: input.report.evaluator.digest, + terminalReceiptDigest, + issuedAt, + executionEvidence: copyExecutionEvidence(input.executionEvidence ?? []) + } + }; + + requireValid("Execution Passport", validateAs("ExecutionPassport", passport)); + return passport; +} diff --git a/src/index.ts b/src/index.ts index b92d102..0ddad89 100644 --- a/src/index.ts +++ b/src/index.ts @@ -7,6 +7,7 @@ export * from "./core/authority.js"; export * from "./core/canonical.js"; export * from "./core/explain.js"; export * from "./core/fingerprint.js"; +export * from "./core/passport.js"; export * from "./core/receipts.js"; export * from "./demo/demo.js"; export * from "./demo/mock-change.js"; diff --git a/test/execution-passport-cli.test.ts b/test/execution-passport-cli.test.ts new file mode 100644 index 0000000..5a80c61 --- /dev/null +++ b/test/execution-passport-cli.test.ts @@ -0,0 +1,128 @@ +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { describe, expect, it, vi } from "vitest"; + +import { writeJson } from "../src/adapters/documents.js"; +import { createCliProgram } from "../src/cli-program.js"; +import { + canonicalize, + digestValue, + runConformance, + runDemo, + validateDocument +} from "../src/index.js"; +import type { ExecutionEvidenceBinding } from "../src/index.js"; + +function evidenceBinding(): ExecutionEvidenceBinding { + return { + role: "execution-placement", + producer: "placement-provider", + resource: { + name: "execution-placement-evidence", + digest: { sha256: digestValue({ contentFreeFixture: true }) }, + mediaType: "application/vnd.example.execution-placement+json" + }, + disclosure: "content-free" + }; +} + +describe("passport CLI", () => { + it("writes exact canonical Statement bytes without embedding source documents or paths", async () => { + const directory = mkdtempSync(join(tmpdir(), "agentic-strata-passport-cli-")); + try { + const bundle = runDemo().bundle; + const report = runConformance(bundle, "enterprise", { + generatedAt: "2026-07-17T12:00:00.000Z" + }); + const bundlePath = join(directory, "bundle.json"); + const reportPath = join(directory, "report.json"); + const oasfPath = join(directory, "external-oasf.json"); + const evidencePath = join(directory, "execution-evidence-binding.json"); + const outputPath = join(directory, "passport.json"); + writeJson(bundlePath, bundle); + writeJson(reportPath, report); + writeJson(oasfPath, { opaqueExternalRecord: "must-not-be-embedded" }); + writeJson(evidencePath, evidenceBinding()); + + await createCliProgram().parseAsync( + [ + "passport", + bundlePath, + "--report", + reportPath, + "--oasf-record", + oasfPath, + "--oasf-media-type", + "application/json", + "--execution-evidence", + evidencePath, + "--issued-at", + "2026-07-17T12:00:01.000Z", + "--output", + outputPath + ], + { from: "user" } + ); + + const raw = readFileSync(outputPath, "utf8"); + const passport = JSON.parse(raw) as unknown; + expect(raw).toBe(canonicalize(passport)); + expect(validateDocument(passport)).toEqual({ valid: true, issues: [] }); + expect(raw).not.toContain("must-not-be-embedded"); + expect(raw).not.toContain(directory); + expect(raw.endsWith("\n")).toBe(false); + } finally { + rmSync(directory, { recursive: true, force: true }); + } + }); + + it("rejects content-bearing execution evidence at the CLI boundary", async () => { + const directory = mkdtempSync(join(tmpdir(), "agentic-strata-passport-cli-invalid-")); + const previousExitCode = process.exitCode; + const errorOutput = vi.spyOn(console, "error").mockImplementation(() => undefined); + try { + const bundle = runDemo().bundle; + const report = runConformance(bundle, "enterprise", { + generatedAt: "2026-07-17T12:00:00.000Z" + }); + const bundlePath = join(directory, "bundle.json"); + const reportPath = join(directory, "report.json"); + const oasfPath = join(directory, "external-oasf.json"); + const evidencePath = join(directory, "raw-execution.json"); + const binding = evidenceBinding(); + writeJson(bundlePath, bundle); + writeJson(reportPath, report); + writeJson(oasfPath, { opaqueExternalRecord: true }); + writeJson(evidencePath, { + ...binding, + resource: { ...binding.resource, content: "sensitive-output" } + }); + + await expect( + createCliProgram().parseAsync( + [ + "passport", + bundlePath, + "--report", + reportPath, + "--oasf-record", + oasfPath, + "--oasf-media-type", + "application/json", + "--execution-evidence", + evidencePath, + "--issued-at", + "2026-07-17T12:00:01.000Z" + ], + { from: "user" } + ) + ).rejects.toThrow("not a content-free ExecutionEvidenceBinding"); + } finally { + process.exitCode = previousExitCode; + errorOutput.mockRestore(); + rmSync(directory, { recursive: true, force: true }); + } + }); +}); diff --git a/test/execution-passport.test.ts b/test/execution-passport.test.ts new file mode 100644 index 0000000..9909571 --- /dev/null +++ b/test/execution-passport.test.ts @@ -0,0 +1,418 @@ +import { describe, expect, it } from "vitest"; + +import { + API_VERSION, + EXECUTION_PASSPORT_PREDICATE_TYPE, + EXECUTION_PASSPORT_SUBJECTS, + appendTraceEvent, + createExecutionPassport, + digestValue, + omitDigest, + runConformance, + runDemo, + seal, + validateAs, + validateDocument +} from "../src/index.js"; +import type { + ConformanceReport, + ExecutionEvidenceBinding, + RunBundle, + RuntimeBoundaryEvidence +} from "../src/index.js"; + +const GENERATED_AT = "2026-07-17T12:00:00.000Z"; +const ISSUED_AT = "2026-07-17T12:00:01.000Z"; +const OASF_MEDIA_TYPE = "application/json"; +const LATE_OBSERVATION = "2026-07-17T13:00:00.000Z"; +const INVALID_EXTERNAL_URIS = [ + "data:text/plain;base64,U0VDUkVU", + "blob:https://example.test/opaque", + "javascript:alert(1)", + "file:///tmp/private.json", + "filesystem:https://example.test/private.json", + "c:/Users/antonio/private.json", + "https://user:password@example.test/private.json", + "https://example.test/private.json?token=secret", + "https://example.test/private.json#secret", + `urn:example:${"a".repeat(2048)}` +] as const; + +function reportFor(bundle: RunBundle): ConformanceReport { + return runConformance(bundle, "enterprise", { generatedAt: GENERATED_AT }); +} + +function resealReport( + report: ConformanceReport, + patch: Partial> +): ConformanceReport { + return seal({ ...omitDigest(report), ...patch }); +} + +function contentFreeEvidence(): ExecutionEvidenceBinding { + return { + role: "execution-placement", + producer: "placement-provider", + resource: { + name: "execution-placement-evidence", + digest: { sha256: digestValue({ opaqueEvidenceFixture: true }) }, + mediaType: "application/vnd.example.execution-placement+json", + uri: "https://example.test/evidence/execution-placement.json" + }, + disclosure: "content-free" + }; +} + +function create( + bundle: RunBundle, + report: ConformanceReport = reportFor(bundle), + options: { + oasfRecord?: unknown; + executionEvidence?: ExecutionEvidenceBinding[]; + } = {} +) { + return createExecutionPassport({ + bundle, + report, + oasfRecord: options.oasfRecord ?? { opaqueExternalRecord: true }, + oasfMediaType: OASF_MEDIA_TYPE, + executionEvidence: options.executionEvidence ?? [], + issuedAt: ISSUED_AT + }); +} + +describe("Execution Passport", () => { + it("emits one strict in-toto Statement with exactly three ordered subjects", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const oasfRecord = { opaqueExternalRecord: true, owner: "external-validator" }; + const passport = create(bundle, report, { oasfRecord }); + + expect(passport._type).toBe("https://in-toto.io/Statement/v1"); + expect(passport.predicateType).toBe(EXECUTION_PASSPORT_PREDICATE_TYPE); + expect(passport.subject.map((subject) => subject.name)).toEqual([ + EXECUTION_PASSPORT_SUBJECTS.runBundle, + EXECUTION_PASSPORT_SUBJECTS.conformanceReport, + EXECUTION_PASSPORT_SUBJECTS.oasfRecord + ]); + expect(passport.subject).toHaveLength(3); + expect(passport.subject[0].digest.sha256).toBe(digestValue(bundle)); + expect(passport.subject[1].digest.sha256).toBe(digestValue(report)); + expect(passport.subject[1].digest.sha256).not.toBe(report.digest); + expect(passport.subject[2].digest.sha256).toBe(digestValue(oasfRecord)); + expect(validateAs("ExecutionPassport", passport)).toEqual({ valid: true, issues: [] }); + expect(validateDocument(passport)).toEqual({ valid: true, issues: [] }); + }); + + it("treats OASF as opaque canonical JSON rather than claiming semantic validity", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const first = create(bundle, report, { + oasfRecord: { z: "external", a: { second: 2, first: 1 } } + }); + const second = create(bundle, report, { + oasfRecord: { a: { first: 1, second: 2 }, z: "external" } + }); + + expect(first).toEqual(second); + }); + + it("records conformance and run state without implying success", () => { + const failedConformanceBundle = structuredClone(runDemo().bundle); + failedConformanceBundle.manifest.policies.approvalRequiredForHighRisk = false; + const failedConformanceReport = reportFor(failedConformanceBundle); + const failedConformancePassport = create( + failedConformanceBundle, + failedConformanceReport + ); + expect(failedConformancePassport.predicate.conformanceStatus).toBe("fail"); + expect(failedConformancePassport.predicate.runStatus).toBe("completed"); + + const incompleteBundle = structuredClone(runDemo().bundle); + incompleteBundle.traceEvents.pop(); + const incompletePassport = create(incompleteBundle); + expect(incompletePassport.predicate.runStatus).toBe("incomplete"); + expect(incompletePassport.predicate.terminalReceiptDigest).toBeNull(); + + const failedRunBundle = structuredClone(runDemo().bundle); + const terminal = failedRunBundle.traceEvents.at(-1); + const previous = failedRunBundle.traceEvents.at(-2); + if (terminal === undefined || previous === undefined) throw new Error("demo receipts missing"); + failedRunBundle.traceEvents[failedRunBundle.traceEvents.length - 1] = appendTraceEvent( + previous, + { + eventId: terminal.eventId, + runId: terminal.runId, + sequence: terminal.sequence, + eventType: "run.failed", + stratum: terminal.stratum, + summary: "The run terminated with an explicit failure.", + attributes: terminal.attributes, + evidenceRefs: terminal.evidenceRefs, + occurredAt: terminal.occurredAt + } + ); + const failedRunPassport = create(failedRunBundle); + expect(failedRunPassport.predicate.runStatus).toBe("failed"); + expect(failedRunPassport.predicate.terminalReceiptDigest).toBe( + failedRunBundle.traceEvents.at(-1)?.digest + ); + }); + + it("admits only strict content-free execution evidence bindings", () => { + const bundle = runDemo().bundle; + const evidence = contentFreeEvidence(); + const passport = create(bundle, reportFor(bundle), { executionEvidence: [evidence] }); + expect(passport.predicate.executionEvidence).toEqual([evidence]); + expect(JSON.stringify(passport)).not.toContain("opaqueEvidenceFixture"); + + const contentBearing = { + ...evidence, + resource: { ...evidence.resource, content: "sensitive-output" } + } as unknown as ExecutionEvidenceBinding; + expect(() => + create(bundle, reportFor(bundle), { executionEvidence: [contentBearing] }) + ).toThrow("Execution evidence binding 0 is invalid"); + expect(() => + create(bundle, reportFor(bundle), { executionEvidence: [evidence, evidence] }) + ).toThrow("is duplicated"); + }); + + it("rejects inline, credentialed, unstable, oversized, and local subject URIs", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + for (const uri of INVALID_EXTERNAL_URIS) { + expect(() => + createExecutionPassport({ + bundle, + report, + oasfRecord: { opaqueExternalRecord: true }, + oasfMediaType: OASF_MEDIA_TYPE, + subjectUris: { oasfRecord: uri }, + issuedAt: ISSUED_AT + }) + ).toThrow("Execution Passport is invalid"); + } + + const stable = createExecutionPassport({ + bundle, + report, + oasfRecord: { opaqueExternalRecord: true }, + oasfMediaType: OASF_MEDIA_TYPE, + subjectUris: { + runBundle: "https://example.test/runs/bundle.json", + oasfRecord: "urn:example:oasf:record-1" + }, + issuedAt: ISSUED_AT + }); + expect(stable.subject[0].uri).toBe("https://example.test/runs/bundle.json"); + expect(stable.subject[2].uri).toBe("urn:example:oasf:record-1"); + }); + + it("rejects inline, credentialed, unstable, oversized, and local evidence URIs", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const evidence = contentFreeEvidence(); + for (const uri of INVALID_EXTERNAL_URIS) { + const invalid = { + ...evidence, + resource: { ...evidence.resource, uri } + }; + expect(() => create(bundle, report, { executionEvidence: [invalid] })).toThrow( + "Execution evidence binding 0 is invalid" + ); + } + }); + + it("rejects non-canonical descriptor extensions", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + + const passport = create(bundle, report); + expect(validateAs("ExecutionPassport", { ...passport, signatures: [] }).valid).toBe(false); + }); + + it("rejects tampered or mismatched report seals and bindings", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const tampered = structuredClone(report); + tampered.status = "fail"; + expect(() => create(bundle, tampered)).toThrow("ConformanceReport seal does not match"); + + const mismatched = resealReport(report, { runBundleDigest: "0".repeat(64) }); + expect(() => create(bundle, mismatched)).toThrow("not bound to the supplied RunBundle"); + + const invalidEvaluator = resealReport(report, { + evaluator: { ...report.evaluator, digest: "0".repeat(64) } + }); + expect(() => create(bundle, invalidEvaluator)).toThrow("evaluator seal does not match"); + }); + + it("rejects a passing report that contains a failed check", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const checks = structuredClone(report.checks); + const first = checks[0]; + if (first === undefined) throw new Error("conformance checks missing"); + checks[0] = { + ...first, + status: "fail", + severity: "error", + message: "Deliberate failed check." + }; + const inconsistent = resealReport(report, { status: "pass", checks }); + expect(() => create(bundle, inconsistent)).toThrow("status pass does not match fail"); + }); + + it("binds the ordered unique check identifiers to the rules digest", () => { + const bundle = runDemo().bundle; + const report = reportFor(bundle); + const first = report.checks[0]; + if (first === undefined) throw new Error("conformance checks missing"); + + const duplicateChecks = [...report.checks, structuredClone(first)]; + const duplicate = resealReport(report, { + checks: duplicateChecks, + rulesDigest: digestValue(duplicateChecks.map((check) => check.id)) + }); + expect(() => create(bundle, duplicate)).toThrow("duplicate check identifiers"); + + for (const checks of [report.checks.slice(1), [...report.checks].reverse()]) { + const inconsistent = resealReport(report, { checks }); + expect(() => create(bundle, inconsistent)).toThrow( + "rules digest does not match its ordered checks" + ); + } + }); + + it("rejects a report whose stated run status differs from the receipt history", () => { + const bundle = runDemo().bundle; + const report = resealReport(reportFor(bundle), { runStatus: "failed" }); + expect(() => create(bundle, report)).toThrow("does not match completed"); + }); + + it("rejects an invalid receipt chain even when a report is freshly bound to it", () => { + const bundle = structuredClone(runDemo().bundle); + const event = bundle.traceEvents[1]; + if (event === undefined) throw new Error("demo receipt missing"); + event.summary = "tampered"; + const report = reportFor(bundle); + expect(() => create(bundle, report)).toThrow("receipt chain is invalid"); + }); + + it("rejects inconsistent run identities and issuance before the report", () => { + const bundle = structuredClone(runDemo().bundle); + bundle.budgetUsage.runId = "different-run"; + expect(() => create(bundle, reportFor(bundle))).toThrow("inconsistent run identifiers"); + + const validBundle = runDemo().bundle; + expect(() => + createExecutionPassport({ + bundle: validBundle, + report: reportFor(validBundle), + oasfRecord: { opaqueExternalRecord: true }, + oasfMediaType: OASF_MEDIA_TYPE, + issuedAt: "2026-07-17T11:59:59.000Z" + }) + ).toThrow("cannot precede"); + }); + + it("requires report generation after every observed evidence timestamp", () => { + const cases: Array<{ + label: string; + mutate: (bundle: RunBundle) => void; + }> = [ + { + label: "trace event", + mutate: (bundle) => { + const terminal = bundle.traceEvents.at(-1); + if (terminal === undefined) throw new Error("terminal receipt missing"); + bundle.traceEvents[bundle.traceEvents.length - 1] = seal({ + ...omitDigest(terminal), + occurredAt: LATE_OBSERVATION + }); + } + }, + { + label: "budget usage", + mutate: (bundle) => { + bundle.budgetUsage = seal({ + ...omitDigest(bundle.budgetUsage), + observedAt: LATE_OBSERVATION + }); + } + }, + { + label: "runtime boundary", + mutate: (bundle) => { + const boundary: Omit = { + contractType: "RuntimeBoundaryEvidence", + apiVersion: API_VERSION, + boundaryEvidenceId: "boundary-late-observation", + runId: bundle.execution.runId, + modelHosting: "managed", + dataBoundary: "global", + networkEgressObserved: true, + endpointDigest: digestValue({ endpoint: "redacted" }), + producer: "boundary-observer", + observationStartedAt: GENERATED_AT, + observedAt: LATE_OBSERVATION + }; + bundle.runtimeBoundaries.push(seal(boundary)); + } + }, + { + label: "criterion result", + mutate: (bundle) => { + const criterion = bundle.criterionResults[0]; + if (criterion === undefined) throw new Error("criterion result missing"); + bundle.criterionResults[0] = seal({ + ...omitDigest(criterion), + observedAt: LATE_OBSERVATION + }); + } + }, + { + label: "decision", + mutate: (bundle) => { + const decision = bundle.decisions[0]; + if (decision === undefined) throw new Error("decision evidence missing"); + bundle.decisions[0] = seal({ + ...omitDigest(decision), + createdAt: LATE_OBSERVATION + }); + } + }, + { + label: "artifact attestation", + mutate: (bundle) => { + const attestation = bundle.attestations[0]; + if (attestation === undefined) throw new Error("artifact attestation missing"); + bundle.attestations[0] = seal({ + ...omitDigest(attestation), + createdAt: LATE_OBSERVATION + }); + } + }, + { + label: "approval", + mutate: (bundle) => { + const approval = bundle.approvals[0]; + if (approval === undefined) throw new Error("approval receipt missing"); + bundle.approvals[0] = seal({ + ...omitDigest(approval), + issuedAt: LATE_OBSERVATION + }); + } + } + ]; + + for (const testCase of cases) { + const bundle = structuredClone(runDemo().bundle); + testCase.mutate(bundle); + expect(() => create(bundle, reportFor(bundle)), testCase.label).toThrow( + "The ConformanceReport cannot precede" + ); + } + }); +}); diff --git a/test/registry-manifest-config.test.ts b/test/registry-manifest-config.test.ts index 7a73200..fa068a6 100644 --- a/test/registry-manifest-config.test.ts +++ b/test/registry-manifest-config.test.ts @@ -1,15 +1,18 @@ import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { join, resolve } from "node:path"; import { describe, expect, it } from "vitest"; +import { readDocument } from "../src/adapters/documents.js"; import { + createExecutionPassport, defaultProfileConfigPath, getSchemaPath, lintManifest, loadProfileConfiguration, resolveProfileRules, + runConformance, validateAs, validateDocument } from "../src/index.js"; @@ -39,6 +42,36 @@ describe("language-neutral contract registry", () => { expect(result.valid).toBe(false); expect(result.issues.some((issue) => issue.path === "/profile")).toBe(true); }); + + it("dispatches the supported in-toto predicate without a contractType", () => { + const bundle = runDemo().bundle; + const report = runConformance(bundle, "enterprise", { + generatedAt: "2026-07-17T12:00:00.000Z" + }); + const passport = createExecutionPassport({ + bundle, + report, + oasfRecord: { opaqueExternalRecord: true }, + oasfMediaType: "application/json", + issuedAt: "2026-07-17T12:00:01.000Z" + }); + expect(validateDocument(passport)).toEqual({ valid: true, issues: [] }); + expect( + validateDocument({ ...passport, predicateType: "https://example.test/unknown" }) + .issues[0] + ).toMatchObject({ path: "/predicateType", code: "const" }); + expect(validateDocument({ ...passport, _type: "https://example.test/unknown" }).issues[0]) + .toMatchObject({ path: "/_type", code: "const" }); + }); + + it("validates the OASF and placement-provider loss mappings", () => { + for (const path of ["adapters/oasf.mapping.yaml", "adapters/stagefabric.mapping.yaml"]) { + expect(validateAs("AdapterMapping", readDocument(resolve(path)))).toEqual({ + valid: true, + issues: [] + }); + } + }); }); describe("manifest dependency lint", () => {