Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
# SourceOS/SociOS Typed Contracts

**SourceOS/SociOS Typed Contracts** is the canonical, machine-readable specification for the SourceOS metadata governance platform and the SociOS agent plane. It defines the full set of JSON Schemas, an OpenAPI REST surface, an AsyncAPI event spine, and a JSON-LD / Hydra semantic overlay that together make up the "contract layer" every implementation component must satisfy.
**SourceOS/SociOS Typed Contracts** is the canonical, machine-readable specification for the SourceOS metadata governance platform and the SociOS agent plane. It defines the full set of JSON Schemas, additive OpenAPI / AsyncAPI patch fragments, and a JSON-LD / Hydra semantic overlay that together make up the "contract layer" every implementation component must satisfy.

> **Spec version:** `2.0.0`  |  **License:** see [LICENSE](LICENSE)

---

## Why this repository exists

A metadata governance platform can only unify data meaning, policy, provenance, and agent execution if every component agrees on the *shape* of the objects it exchanges. This repository is that shared agreement. Downstream consumers include:
A metadata governance platform can only unify data meaning, policy, provenance, and agent execution if every component agrees on the *shape* of the objects it exchanges. This repository is that shared agreement. Downstream consumers include:

- **API services** — scaffolded from `openapi.yaml` (metadata plane) and `openapi.agent-plane.patch.yaml` (agent plane).
- **Event consumers** — Kafka topics declared in `asyncapi.yaml` + `asyncapi.agent-plane.patch.yaml`.
- **API services** — scaffolded from `openapi.yaml` plus additive patch fragments such as `openapi.agent-plane.patch.yaml` and `openapi.fog.patch.yaml`.
- **Event consumers** — Kafka topics declared in `asyncapi.yaml` plus additive channel fragments such as `asyncapi.agent-plane.patch.yaml` and `asyncapi.fog.patch.yaml`.
- **Validators** — AJV (Node.js) or `jsonschema` (Python) loaded from `schemas/`.
- **Code generators** — TypeScript types via [quicktype](https://quicktype.io); Python models via [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator).
- **Semantic tooling** — JSON-LD context + Hydra API documentation in `semantic/`.
- **Semantic tooling** — JSON-LD context + Hydra API documentation in `semantic/`, including additive vocabulary seeds such as `semantic/fog-vocabulary.jsonld`.

---

## Repository layout

```
```text
sourceos-spec/
├── README.md # This file
├── ARCHITECTURE.md # Two-plane architecture, schema families, lifecycle
Expand All @@ -30,27 +30,31 @@ sourceos-spec/
├── openapi.yaml # Metadata-plane REST API (v2)
├── openapi.agent-plane.patch.yaml # Additive agent-plane REST endpoints
├── openapi.fog.patch.yaml # Additive fog-layer REST endpoints
├── asyncapi.yaml # Metadata-plane event channels
├── asyncapi.agent-plane.patch.yaml # Agent-plane event channels
├── asyncapi.fog.patch.yaml # Fog-layer event channels
├── schemas/ # 54 JSON Schema (draft 2020-12) files
├── schemas/ # Top-level JSON Schema files (draft 2020-12)
│ └── README.md # Schema catalog and URN patterns
├── examples/ # Conforming example payloads (one per type)
│ └── README.md
├── semantic/ # JSON-LD context + Hydra API documentation
│ └── README.md
│ ├── README.md
│ └── fog-vocabulary.jsonld # Additive fog vocabulary seed
└── docs/
└── adr/ # Architecture Decision Records
├── adr/ # Architecture Decision Records
└── contract-additions/ # Discoverability notes for additive families
```

---

## Schema families

The 54 schemas are organised into six families that map directly to the Open Metadata Types taxonomy areas:
The schemas are organised into domain-oriented families that map to the SourceOS / SociOS contract surface:

| # | Family | Key schemas |
|---|--------|-------------|
Expand All @@ -63,6 +67,7 @@ The 54 schemas are organised into six families that map directly to the Open Met
| + | **Execution / Provenance** | `Dataset`, `RunRecord`, `WorkflowSpec`, `WorkflowNode`, `WorkflowEdge`, `WorkloadSpec`, `DataSphere`, `ProvenanceRecord`, `EventEnvelope`, `MappingSpec` |
| + | **Agent Plane** | `AgentSession`, `ExecutionDecision`, `ExecutionSurface`, `SkillManifest`, `MemoryEntry`, `SessionReceipt`, `SessionReview`, `TelemetryEvent`, `FrustrationSignal` |
| + | **Release / Experiments** | `ExperimentFlag`, `RolloutPolicy`, `ReleaseReceipt` |
| + | **Fog Layer** | `Topic`, `TopicEnvelope`, `ReplicationPolicy`, `ContentRef`, `Offer`, `WorkOrder`, `UsageReceipt`, `SettlementEvent` |

---

Expand Down Expand Up @@ -128,4 +133,3 @@ fastapi-codegen --input openapi.yaml --output app/
## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for schema authoring conventions, the URN naming guide, and the pull-request checklist.

46 changes: 34 additions & 12 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ This directory contains one conforming JSON example payload for each top-level s

## What the examples show

The examples are designed to tell a coherent end-to-end story: a personal health dataset is catalogued, governed, transformed by an obfuscation workload, and released — all within an agent session. They share cross-reference URNs so you can trace the full lifecycle:
The examples are designed to tell a coherent end-to-end story: a personal health dataset is catalogued, governed, transformed by an obfuscation workload, and released — all within an agent session. They share cross-reference URNs so you can trace the full lifecycle:

```
```text
connector.json ──► asset.json
Expand Down Expand Up @@ -41,6 +41,23 @@ agent_session.json ──► execution_decision.json ──► session_recei

---

## Recent additions — Fog Layer examples

The fog example set illustrates the new FogVault / FogCompute contract family:

| File | Schema type | Description |
|------|------------|-------------|
| `content_ref.json` | ContentRef | Content-addressed reference for fog blobs / manifests |
| `topic.json` | Topic | FogVault topic definition with replication and encryption metadata |
| `topic_envelope.json` | TopicEnvelope | Append-only topic entry envelope with integrity and encryption metadata |
| `replication_policy.json` | ReplicationPolicy | Replica / ack / retention / compaction policy example |
| `offer.json` | Offer | FogCompute provider offer advertising local resources |
| `workorder.json` | WorkOrder | FogCompute request with image, inputs, outputs, and verification mode |
| `usage_receipt.json` | UsageReceipt | Worker-emitted execution receipt with metered resource usage |
| `settlement_event.json` | SettlementEvent | Optional mapping from receipt to settlement backend / credits |

---

## File index

| File | Schema type | Description |
Expand All @@ -52,30 +69,37 @@ agent_session.json ──► execution_decision.json ──► session_recei
| `comment.json` | Comment | A review note on a field mapping |
| `community.json` | Community | The data-governance team community |
| `connector.json` | Connector | A local S3 connector |
| `content_ref.json` | ContentRef | Content-addressed reference for fog blobs / manifests |
| `dataset.json` | Dataset | Personal health observations dataset |
| `data_sphere.json` | DataSphere | The personal-curated execution environment |
| `policy_decision.json` | PolicyDecision | An `export` permit decision with an obfuscation obligation |
| `execution_decision.json` | ExecutionDecision | Agent allow-decision for a tool invocation |
| `experiment_flag.json` | ExperimentFlag | A feature flag for the new obfuscation algorithm |
| `event_envelope.json` | EventEnvelope | Event published when the run completes |
| `experiment_flag.json` | ExperimentFlag | A feature flag for the new obfuscation algorithm |
| `field.json` | Field | The `patient.dateOfBirth` field with PII tags and quality metrics |
| `frustration_signal.json` | FrustrationSignal | A frustration signal from a repeated-failure condition |
| `glossary.json` | GlossaryTerm | Glossary term for "Date of Birth" |
| `mapping.json` | MappingSpec | A field mapping between two dataset fields |
| `memory_entry.json` | MemoryEntry | A learned memory entry from an agent session |
| `offer.json` | Offer | FogCompute provider offer |
| `policy.json` | Policy | Health export must be obfuscated |
| `policy_decision.json` | PolicyDecision | An `export` permit decision with an obfuscation obligation |
| `provenance.json` | ProvenanceRecord | Provenance record for the obfuscation run |
| `rating.json` | Rating | A 5-star rating on the health observations dataset |
| `release_receipt.json` | ReleaseReceipt | Release receipt for spec version 2.0.0 |
| `replication_policy.json` | ReplicationPolicy | Fog topic replication/retention policy example |
| `rollout_policy.json` | RolloutPolicy | Rollout rules for the obfuscation experiment flag |
| `run.json` | RunRecord | The obfuscation workload run record |
| `schema.json` | SchemaDefinition | The schema for health observations |
| `session_receipt.json` | SessionReceipt | Receipt for the completed agent session |
| `session_review.json` | SessionReview | Post-session learning review |
| `settlement_event.json` | SettlementEvent | Optional receipt-to-settlement mapping |
| `skill_manifest.json` | SkillManifest | The obfuscation skill manifest |
| `telemetry_event.json` | TelemetryEvent | An informational telemetry event from the agent session |
| `capability_token.json` | CapabilityToken | Capability token for the export operation |
| `topic.json` | Topic | FogVault topic definition |
| `topic_envelope.json` | TopicEnvelope | FogVault append-only entry envelope |
| `workflow_spec.json` | WorkflowSpec | The health-data obfuscation workflow |
| `workorder.json` | WorkOrder | FogCompute work order |
| `usage_receipt.json` | UsageReceipt | FogCompute execution receipt |

---

Expand All @@ -92,14 +116,12 @@ ajv validate -s ../schemas/Dataset.json -d dataset.json
cd ..
for example in examples/*.json; do
name=$(basename "$example" .json)
# Find matching schema (PascalCase title)
python3 - <<EOF
import json, sys, glob
import json
ex = json.load(open("$example"))
t = ex.get("type") or ex.get("title")
if t:
f = f"schemas/{t}.json"
print(f)
print(f"schemas/{t}.json")
EOF
done
```
Expand All @@ -108,8 +130,8 @@ done

## Adding a new example

See [CONTRIBUTING.md](../CONTRIBUTING.md#writing-examples) for the full guide. Key rules:
1. The filename must be the schema `title` lowercased, e.g. `AgentSession` → `agent_session.json`.
See [CONTRIBUTING.md](../CONTRIBUTING.md#writing-examples) for the full guide. Key rules:
1. The filename should match the schema title in the repository’s naming convention.
2. All required fields must be present and valid.
3. Use existing cross-reference URNs from this directory so the example set stays coherent.
3. Use coherent cross-reference URNs where possible so the example set stays navigable.
4. Run `ajv validate` before committing.
48 changes: 46 additions & 2 deletions schemas/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,29 @@
# Schema Catalog

This directory contains all 54 JSON Schema (draft 2020-12) files that make up the SourceOS/SociOS Typed Contracts specification.
This directory contains the JSON Schema (draft 2020-12) files that make up the SourceOS/SociOS Typed Contracts specification.

---

## Recent additions — Fog Layer

The FogVault / FogCompute contract family adds the following top-level schemas:

| File | Type | URN prefix |
|------|------|-----------|
| `ContentRef.json` | ContentRef | _(digest-based content reference)_ |
| `Offer.json` | Offer | `urn:srcos:offer:` |
| `ReplicationPolicy.json` | ReplicationPolicy | _(top-level policy object)_ |
| `SettlementEvent.json` | SettlementEvent | `urn:srcos:settlement:` |
| `Topic.json` | Topic | `urn:srcos:topic:` |
| `TopicEnvelope.json` | TopicEnvelope | `urn:srcos:topic-entry:` |
| `UsageReceipt.json` | UsageReceipt | `urn:srcos:receipt:usage:` |
| `WorkOrder.json` | WorkOrder | `urn:srcos:workorder:` |

These types support:
- append-only Merkle-log topics
- explicit replication/retention policy
- content-addressed storage references
- compute offers, work orders, receipts, and optional settlement mappings

---

Expand All @@ -15,6 +38,7 @@ This directory contains all 54 JSON Schema (draft 2020-12) files that make up th
| `Comment.json` | Comment | `urn:srcos:comment:` |
| `Community.json` | Community | `urn:srcos:community:` |
| `Connector.json` | Connector | `urn:srcos:connector:` |
| `ContentRef.json` | ContentRef | _(digest-based content reference)_ |
| `DataRef.json` | DataRef | _(sub-object, no top-level id)_ |
| `DataSphere.json` | DataSphere | `urn:srcos:sphere:` |
| `Dataset.json` | Dataset | `urn:srcos:dataset:` |
Expand All @@ -34,6 +58,7 @@ This directory contains all 54 JSON Schema (draft 2020-12) files that make up th
| `ObjectContext.json` | ObjectContext | _(sub-object, no id)_ |
| `ObjectSelector.json` | ObjectSelector | _(sub-object inside Policy scope)_ |
| `Obligation.json` | Obligation | _(sub-object, no id)_ |
| `Offer.json` | Offer | `urn:srcos:offer:` |
| `Party.json` | Party | `urn:srcos:party:` |
| `PhysicalAsset.json` | PhysicalAsset | `urn:srcos:asset:` |
| `Policy.json` | Policy | `urn:srcos:policy:` |
Expand All @@ -45,23 +70,29 @@ This directory contains all 54 JSON Schema (draft 2020-12) files that make up th
| `QualityMetric.json` | QualityMetric | _(sub-object inside Field.quality)_ |
| `Rating.json` | Rating | `urn:srcos:rating:` |
| `ReleaseReceipt.json` | ReleaseReceipt | `urn:srcos:release-receipt:` |
| `ReplicationPolicy.json` | ReplicationPolicy | _(top-level policy object)_ |
| `RolloutPolicy.json` | RolloutPolicy | `urn:srcos:rollout:` |
| `Rule.json` | Rule | _(sub-object inside Policy)_ |
| `RunRecord.json` | RunRecord | `urn:srcos:run:` |
| `SchemaDefinition.json` | SchemaDefinition | `urn:srcos:schema:` |
| `SessionReceipt.json` | SessionReceipt | `urn:srcos:receipt:session:` |
| `SessionReview.json` | SessionReview | `urn:srcos:session-review:` |
| `SettlementEvent.json` | SettlementEvent | `urn:srcos:settlement:` |
| `SkillManifest.json` | SkillManifest | `urn:srcos:skill:` |
| `SubjectContext.json` | SubjectContext | _(sub-object, no id)_ |
| `SubjectSelector.json` | SubjectSelector | _(sub-object inside Policy scope)_ |
| `TagAssignment.json` | TagAssignment | _(sub-object inside Field/GlossaryTerm)_ |
| `TelemetryEvent.json` | TelemetryEvent | `urn:srcos:telemetry:` |
| `Topic.json` | Topic | `urn:srcos:topic:` |
| `TopicEnvelope.json` | TopicEnvelope | `urn:srcos:topic-entry:` |
| `Trigger.json` | Trigger | _(sub-object inside WorkflowSpec)_ |
| `UsageReceipt.json` | UsageReceipt | `urn:srcos:receipt:usage:` |
| `ValidValues.json` | ValidValues | _(sub-object inside EntityField)_ |
| `WorkflowEdge.json` | WorkflowEdge | _(sub-object inside WorkflowSpec)_ |
| `WorkflowNode.json` | WorkflowNode | _(sub-object inside WorkflowSpec)_ |
| `WorkflowSpec.json` | WorkflowSpec | `urn:srcos:workflow:` |
| `WorkloadSpec.json` | WorkloadSpec | `urn:srcos:workload:` |
| `WorkOrder.json` | WorkOrder | `urn:srcos:workorder:` |

---

Expand Down Expand Up @@ -163,6 +194,19 @@ This directory contains all 54 JSON Schema (draft 2020-12) files that make up th
| `RolloutPolicy` | Audience-based percentage rollout rules for an `ExperimentFlag` |
| `ReleaseReceipt` | A verified release record with artifact hashes and gate check results |

### Fog Layer

| Schema | Description |
|--------|-------------|
| `Topic` | An append-only topic contract for FogVault channels |
| `TopicEnvelope` | A typed append-only entry envelope for topic events and payloads |
| `ReplicationPolicy` | Replication, retention, and compaction policy for a fog topic |
| `ContentRef` | A content-addressed reference to blobs, chunks, or manifests |
| `Offer` | A FogCompute provider offer advertising resources and constraints |
| `WorkOrder` | A FogCompute request describing workload, inputs, outputs, and verification mode |
| `UsageReceipt` | A worker-emitted usage and output receipt for a completed work order |
| `SettlementEvent` | An optional settlement mapping from receipt to credits/tokens/backend |

---

## Validation
Expand All @@ -183,4 +227,4 @@ done

## Versioning

Schema evolution follows [Semantic Versioning](https://semver.org/). See [CONTRIBUTING.md](../CONTRIBUTING.md#breaking-vs-additive-changes) for the full policy and [CHANGELOG.md](../CHANGELOG.md) for the history.
Schema evolution follows [Semantic Versioning](https://semver.org/). See [CONTRIBUTING.md](../CONTRIBUTING.md#breaking-vs-additive-changes) for the full policy and [CHANGELOG.md](../CHANGELOG.md) for the history.
Loading
Loading