diff --git a/README.md b/README.md index 4d1ed7c..44c42cd 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # SourceOS/SociOS Typed Contracts -**SourceOS/SociOS Typed Contracts** is the canonical, machine-readable specification for the SourceOS metadata governance platform and the SociOS agent plane. It defines the full set of JSON Schemas, an OpenAPI REST surface, an AsyncAPI event spine, and a JSON-LD / Hydra semantic overlay that together make up the "contract layer" every implementation component must satisfy. +**SourceOS/SociOS Typed Contracts** is the canonical, machine-readable specification for the SourceOS metadata governance platform and the SociOS agent plane. It defines the full set of JSON Schemas, additive OpenAPI / AsyncAPI patch fragments, and a JSON-LD / Hydra semantic overlay that together make up the "contract layer" every implementation component must satisfy. > **Spec version:** `2.0.0`  |  **License:** see [LICENSE](LICENSE) @@ -8,19 +8,19 @@ ## Why this repository exists -A metadata governance platform can only unify data meaning, policy, provenance, and agent execution if every component agrees on the *shape* of the objects it exchanges. This repository is that shared agreement. Downstream consumers include: +A metadata governance platform can only unify data meaning, policy, provenance, and agent execution if every component agrees on the *shape* of the objects it exchanges. This repository is that shared agreement. Downstream consumers include: -- **API services** — scaffolded from `openapi.yaml` (metadata plane) and `openapi.agent-plane.patch.yaml` (agent plane). -- **Event consumers** — Kafka topics declared in `asyncapi.yaml` + `asyncapi.agent-plane.patch.yaml`. +- **API services** — scaffolded from `openapi.yaml` plus additive patch fragments such as `openapi.agent-plane.patch.yaml` and `openapi.fog.patch.yaml`. +- **Event consumers** — Kafka topics declared in `asyncapi.yaml` plus additive channel fragments such as `asyncapi.agent-plane.patch.yaml` and `asyncapi.fog.patch.yaml`. - **Validators** — AJV (Node.js) or `jsonschema` (Python) loaded from `schemas/`. - **Code generators** — TypeScript types via [quicktype](https://quicktype.io); Python models via [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator). -- **Semantic tooling** — JSON-LD context + Hydra API documentation in `semantic/`. +- **Semantic tooling** — JSON-LD context + Hydra API documentation in `semantic/`, including additive vocabulary seeds such as `semantic/fog-vocabulary.jsonld`. --- ## Repository layout -``` +```text sourceos-spec/ ├── README.md # This file ├── ARCHITECTURE.md # Two-plane architecture, schema families, lifecycle @@ -30,27 +30,31 @@ sourceos-spec/ │ ├── openapi.yaml # Metadata-plane REST API (v2) ├── openapi.agent-plane.patch.yaml # Additive agent-plane REST endpoints +├── openapi.fog.patch.yaml # Additive fog-layer REST endpoints ├── asyncapi.yaml # Metadata-plane event channels ├── asyncapi.agent-plane.patch.yaml # Agent-plane event channels +├── asyncapi.fog.patch.yaml # Fog-layer event channels │ -├── schemas/ # 54 JSON Schema (draft 2020-12) files +├── schemas/ # Top-level JSON Schema files (draft 2020-12) │ └── README.md # Schema catalog and URN patterns │ ├── examples/ # Conforming example payloads (one per type) │ └── README.md │ ├── semantic/ # JSON-LD context + Hydra API documentation -│ └── README.md +│ ├── README.md +│ └── fog-vocabulary.jsonld # Additive fog vocabulary seed │ └── docs/ - └── adr/ # Architecture Decision Records + ├── adr/ # Architecture Decision Records + └── contract-additions/ # Discoverability notes for additive families ``` --- ## Schema families -The 54 schemas are organised into six families that map directly to the Open Metadata Types taxonomy areas: +The schemas are organised into domain-oriented families that map to the SourceOS / SociOS contract surface: | # | Family | Key schemas | |---|--------|-------------| @@ -63,6 +67,7 @@ The 54 schemas are organised into six families that map directly to the Open Met | + | **Execution / Provenance** | `Dataset`, `RunRecord`, `WorkflowSpec`, `WorkflowNode`, `WorkflowEdge`, `WorkloadSpec`, `DataSphere`, `ProvenanceRecord`, `EventEnvelope`, `MappingSpec` | | + | **Agent Plane** | `AgentSession`, `ExecutionDecision`, `ExecutionSurface`, `SkillManifest`, `MemoryEntry`, `SessionReceipt`, `SessionReview`, `TelemetryEvent`, `FrustrationSignal` | | + | **Release / Experiments** | `ExperimentFlag`, `RolloutPolicy`, `ReleaseReceipt` | +| + | **Fog Layer** | `Topic`, `TopicEnvelope`, `ReplicationPolicy`, `ContentRef`, `Offer`, `WorkOrder`, `UsageReceipt`, `SettlementEvent` | --- @@ -128,4 +133,3 @@ fastapi-codegen --input openapi.yaml --output app/ ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md) for schema authoring conventions, the URN naming guide, and the pull-request checklist. - diff --git a/examples/README.md b/examples/README.md index 90ef4ef..404934c 100644 --- a/examples/README.md +++ b/examples/README.md @@ -6,9 +6,9 @@ This directory contains one conforming JSON example payload for each top-level s ## What the examples show -The examples are designed to tell a coherent end-to-end story: a personal health dataset is catalogued, governed, transformed by an obfuscation workload, and released — all within an agent session. They share cross-reference URNs so you can trace the full lifecycle: +The examples are designed to tell a coherent end-to-end story: a personal health dataset is catalogued, governed, transformed by an obfuscation workload, and released — all within an agent session. They share cross-reference URNs so you can trace the full lifecycle: -``` +```text connector.json ──► asset.json │ ▼ @@ -41,6 +41,23 @@ agent_session.json ──► execution_decision.json ──► session_recei --- +## Recent additions — Fog Layer examples + +The fog example set illustrates the new FogVault / FogCompute contract family: + +| File | Schema type | Description | +|------|------------|-------------| +| `content_ref.json` | ContentRef | Content-addressed reference for fog blobs / manifests | +| `topic.json` | Topic | FogVault topic definition with replication and encryption metadata | +| `topic_envelope.json` | TopicEnvelope | Append-only topic entry envelope with integrity and encryption metadata | +| `replication_policy.json` | ReplicationPolicy | Replica / ack / retention / compaction policy example | +| `offer.json` | Offer | FogCompute provider offer advertising local resources | +| `workorder.json` | WorkOrder | FogCompute request with image, inputs, outputs, and verification mode | +| `usage_receipt.json` | UsageReceipt | Worker-emitted execution receipt with metered resource usage | +| `settlement_event.json` | SettlementEvent | Optional mapping from receipt to settlement backend / credits | + +--- + ## File index | File | Schema type | Description | @@ -52,30 +69,37 @@ agent_session.json ──► execution_decision.json ──► session_recei | `comment.json` | Comment | A review note on a field mapping | | `community.json` | Community | The data-governance team community | | `connector.json` | Connector | A local S3 connector | +| `content_ref.json` | ContentRef | Content-addressed reference for fog blobs / manifests | | `dataset.json` | Dataset | Personal health observations dataset | | `data_sphere.json` | DataSphere | The personal-curated execution environment | -| `policy_decision.json` | PolicyDecision | An `export` permit decision with an obfuscation obligation | | `execution_decision.json` | ExecutionDecision | Agent allow-decision for a tool invocation | -| `experiment_flag.json` | ExperimentFlag | A feature flag for the new obfuscation algorithm | | `event_envelope.json` | EventEnvelope | Event published when the run completes | +| `experiment_flag.json` | ExperimentFlag | A feature flag for the new obfuscation algorithm | | `field.json` | Field | The `patient.dateOfBirth` field with PII tags and quality metrics | | `frustration_signal.json` | FrustrationSignal | A frustration signal from a repeated-failure condition | | `glossary.json` | GlossaryTerm | Glossary term for "Date of Birth" | | `mapping.json` | MappingSpec | A field mapping between two dataset fields | | `memory_entry.json` | MemoryEntry | A learned memory entry from an agent session | +| `offer.json` | Offer | FogCompute provider offer | | `policy.json` | Policy | Health export must be obfuscated | +| `policy_decision.json` | PolicyDecision | An `export` permit decision with an obfuscation obligation | | `provenance.json` | ProvenanceRecord | Provenance record for the obfuscation run | | `rating.json` | Rating | A 5-star rating on the health observations dataset | | `release_receipt.json` | ReleaseReceipt | Release receipt for spec version 2.0.0 | +| `replication_policy.json` | ReplicationPolicy | Fog topic replication/retention policy example | | `rollout_policy.json` | RolloutPolicy | Rollout rules for the obfuscation experiment flag | | `run.json` | RunRecord | The obfuscation workload run record | | `schema.json` | SchemaDefinition | The schema for health observations | | `session_receipt.json` | SessionReceipt | Receipt for the completed agent session | | `session_review.json` | SessionReview | Post-session learning review | +| `settlement_event.json` | SettlementEvent | Optional receipt-to-settlement mapping | | `skill_manifest.json` | SkillManifest | The obfuscation skill manifest | | `telemetry_event.json` | TelemetryEvent | An informational telemetry event from the agent session | -| `capability_token.json` | CapabilityToken | Capability token for the export operation | +| `topic.json` | Topic | FogVault topic definition | +| `topic_envelope.json` | TopicEnvelope | FogVault append-only entry envelope | | `workflow_spec.json` | WorkflowSpec | The health-data obfuscation workflow | +| `workorder.json` | WorkOrder | FogCompute work order | +| `usage_receipt.json` | UsageReceipt | FogCompute execution receipt | --- @@ -92,14 +116,12 @@ ajv validate -s ../schemas/Dataset.json -d dataset.json cd .. for example in examples/*.json; do name=$(basename "$example" .json) - # Find matching schema (PascalCase title) python3 - <` IRIs -- Selected cross-vocabulary terms: `prov:wasGeneratedBy`, `prov:used`, `prov:wasAssociatedWith` +- schema type names to `srcos:` IRIs +- selected cross-vocabulary terms such as `prov:wasGeneratedBy`, `prov:used`, `prov:wasAssociatedWith` --- ## `hydra.jsonld` — Hydra API documentation -The Hydra document is a machine-readable API description that extends the JSON-LD context. It can be served at `/.well-known/hydra` to make the API self-describing. +The Hydra document is a machine-readable API description that extends the JSON-LD context. It can be served at `/.well-known/hydra` to make the API self-describing. The document lists every resource class supported by the API with its supported operations (HTTP methods), expected inputs, and returned types. @@ -82,7 +98,7 @@ When adding a new first-class schema type: 1. Add a mapping to `context.jsonld`: `"NewType": "srcos:NewType"` 2. Add a `hydra:supportedClass` entry to `hydra.jsonld` with at minimum `hydra:title`, `hydra:description`, and the applicable `hydra:supportedOperation` entries. -3. Update this README's file table. +3. Update this README's file table and any additive vocabulary seeds if the new type is being staged incrementally. See [CONTRIBUTING.md](../CONTRIBUTING.md#6-update-the-semantic-context-if-needed) for the full guide.