Skip to content

Commit 9a01749

Browse files
dmealingclaude
andcommitted
spec: FR-021/022/023 design sketches — api metadata, contract emitters, metadata packages
FR-021: declared api surfaces (operations + protocol bindings) over the derived-CRUD default; payloads are object.value projections (origin.*); wire- stable wireId numbering on the contract projection, never the entity. FR-022: Tier-2 contract emitters — JSON Schema 2020-12 (canonical + strict structured-output profile) -> OpenAPI 3.1 -> proto3 (AIP CRUD services, locked type mappings, buf breaking as the wire-compat CI gate). FR-023: metadata packages — code-free versioned artifacts via npm/Maven/PyPI/ NuGet, metadataDependencies config, cross-package overlay/extends with provenance attribution. All three added to roadmap Planned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 0424db8 commit 9a01749

4 files changed

Lines changed: 314 additions & 0 deletions
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# FR-021 — `api` metadata type + contract projections (design)
2+
3+
_Status: PROPOSED (design sketch — needs brainstorming + plan before implementation)._
4+
_Date: 2026-06-11._
5+
6+
## Problem
7+
8+
The generated REST surface is **convention-derived**: FR-008/FR-009 define a cross-port
9+
CRUD contract (routes, filter grammar, pagination) computed from each entity. That covers
10+
CRUD well, but three needs have no declared home:
11+
12+
1. **Non-CRUD operations** (RPC-shaped actions, batch ops, domain verbs) — today they are
13+
hand-written outside the metadata, invisible to `verify` and to doc/contract emitters.
14+
2. **Versioned wire contracts** — an API surface that must stay stable while the domain
15+
entities evolve (v1 and v2 served simultaneously over the same entities).
16+
3. **Contract emission** (FR-022: JSON Schema / OpenAPI / protobuf) — emitters need a
17+
declared operation surface and, for protobuf, **wire-stable field numbering**, which
18+
must not live on domain entities (it would couple domain evolution to wire compat).
19+
20+
## Design direction
21+
22+
### 1. API payloads are projections (reuse, don't invent)
23+
24+
An operation's input/output references an `object.value` VO whose fields carry the
25+
existing `origin.*` provenance (`passthrough` / `aggregate` / `collection`) from entities
26+
— the **same projection machinery** that already drives prompt payloads (FR-004) and DB
27+
views (FR-003). The entity stays domain truth; the API projection is the contract surface.
28+
Renaming an entity field re-points (or breaks) the projection's `origin.*`, which `meta
29+
verify` catches at build time — the wire shape never silently changes.
30+
31+
### 2. New metamodel vocabulary (provider-registered, ADR-0023)
32+
33+
Slim sketch (names to be settled in brainstorming):
34+
35+
```yaml
36+
api: # new top-level type; protocol-neutral surface
37+
name: ProgramApi
38+
version: v1 # bare attr; versioned surfaces = sibling api nodes
39+
children:
40+
- operation: # subtypes: query | command (read vs write semantics)
41+
name: getProgram
42+
inputRef: ProgramIdParam # object.value projection (or none)
43+
outputRef: ProgramSummary # object.value projection
44+
children:
45+
- binding.rest: { method: GET, path: "/programs/{id}" }
46+
# future: binding.grpc, binding.messaging — per-protocol bindings
47+
```
48+
49+
- **Derived CRUD stays the zero-config default.** With no `api` declared, FR-008/009
50+
behavior is unchanged. A declared `api` *extends* (or, per entity opt-in, replaces)
51+
the derived surface. Emitters consume the union.
52+
- `binding.*` keeps the surface protocol-neutral with per-protocol attrs where they
53+
belong (REST method/path now; gRPC service/method naming later).
54+
55+
### 3. Wire-stable numbering lives on the contract projection
56+
57+
protobuf field numbers identify fields **on the wire**; renumbering silently misdecodes
58+
(it is equivalent to delete+add). Numbers therefore must be **authored, append-only
59+
metadata** — they genuinely cannot be computed from the model across schema evolution,
60+
which is the ADR-0023 "cannot be computed" justification recorded here.
61+
62+
- `wireId: <n>` on fields of an `object.value` used as a contract projection (and
63+
`wireId` on `field.enum` members where the enum crosses the wire). Protocol-neutral
64+
name on purpose: protobuf consumes it now; Avro/Thrift-style emitters could later.
65+
- `reservedWireIds: [..]` + `reservedWireNames: [..]` on the VO — emitted as proto
66+
`reserved` so deleted fields can never be reused.
67+
- Validation (loader, own-only): duplicate `wireId` in one VO → `ERR_BAD_ATTR_VALUE`;
68+
`wireId` outside 1..536870911 or in 19000..19999 → `ERR_BAD_ATTR_VALUE`; a VO
69+
referenced by a proto-bound operation with any field missing `wireId` → emit-time
70+
error in FR-022 (not a load error — VOs without proto bindings don't need numbers).
71+
- Precedent: TypeSpec's protobuf emitter requires `@field(n)`; entproto requires
72+
`entproto.Field(n)`. Auto-numbering from declaration order is explicitly rejected
73+
(insert/rename/delete renumbers the tail = wire break).
74+
75+
### 4. Relationship to the enterprise tier
76+
77+
Application/ecosystem/dependency-level modeling (which org unit owns which API, what
78+
consumes what) is intentionally **out of scope** for this library — `api` here is the
79+
slim contract vocabulary only. An organization-level metadata tier can layer on top via
80+
the provider SPI without changes here.
81+
82+
## Conformance + verify
83+
84+
- New `fixtures/conformance/` fixtures: api/operation/binding loading, wireId validation
85+
errors, projection-ref resolution errors (`format: "resolved"` envelopes).
86+
- `meta verify --templates`-style gate extended: operation input/output refs must
87+
resolve; projections' `origin.*` must resolve against entities (existing machinery).
88+
- Registry-conformance manifest gains the new vocabulary in all 5 ports (atomic, like
89+
the `@responseRef` carve-out close).
90+
91+
## Open questions (settle in brainstorming)
92+
93+
1. Operation vocabulary: AIP-style five standard methods + custom, or free-form
94+
query/command only?
95+
2. Pagination style per surface: the FR-008 `limit/offset` contract vs AIP
96+
`page_size/page_token` when emitting proto services — per-binding attr?
97+
3. Does `endpoint` (a node per HTTP route) deserve first-class existence, or is
98+
`operation` + `binding.rest` sufficient? (Lean: the latter — fewer types.)
99+
4. Versioning semantics: sibling `api` nodes (v1, v2) vs `@version` on operations.
100+
5. Can a contract projection ALSO be a DB view projection (one VO, two consumers), or
101+
should that be discouraged by validation?
102+
103+
## Dependencies / consumers
104+
105+
- Consumes: `object.value` + `origin.*` (shipped), provider SPI (shipped).
106+
- Consumed by: FR-022 contract emitters; generated API docs (`meta docs` api surface);
107+
future MCP exposure (declared operations become discoverable tools).
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# FR-022 — Contract emitters: JSON Schema → OpenAPI 3.1 → protobuf (design)
2+
3+
_Status: PROPOSED (design sketch — needs brainstorming + plan before implementation)._
4+
_Date: 2026-06-11._
5+
6+
## Problem
7+
8+
Adopters keep their existing contract ecosystems: validation tooling wants JSON Schema,
9+
REST consumers want OpenAPI, gRPC shops want `.proto`. Today MetaObjects generates the
10+
*implementation* (routes, models) but not the *neutral contract artifacts*, so adopters
11+
hand-write them — a drift surface the metadata fully describes (raison-d'être violation).
12+
13+
## Design direction
14+
15+
All three emitters are **Tier-2** per ADR-0020 (output bytes don't depend on the
16+
implementing language → ONE shared TS engine + golden-file conformance fixtures), wired
17+
as ordinary generators (stable names in the generator registry, `gen --list`). They are
18+
**one layered investment**, built in this order:
19+
20+
### Phase 1 — JSON Schema 2020-12, two profiles
21+
22+
- `jsonSchemaFile()` — per-entity / per-projection `<Name>.schema.json`:
23+
- **Canonical profile**: full fidelity. `$defs` for shared abstracts, `format:
24+
uuid/date-time/...`, `field.enum @values``enum`, currency → integer minor units
25+
(description carries the ISO code), wire-contract-faithful types (decimal as string,
26+
int64 as string — must match `normalization.md` exactly or the emitter itself becomes
27+
a drift source; conformance-gated).
28+
- **Strict profile** (`profile: "strict"`): the cross-provider structured-output
29+
intersection — root `type: object`; every property `required` with `["T","null"]`
30+
unions for optionals; `additionalProperties: false` everywhere; **no recursion**; no
31+
numeric/length constraint keywords (`@maxChars` etc. become `description` prose).
32+
This one artifact serves OpenAI strict mode, Anthropic structured outputs, and MCP
33+
hosts with restricted schema subsets.
34+
- Consumers unlocked: validators in every language, MCP `inputSchema`, LLM
35+
structured-output schemas, OpenAPI components (Phase 2), schema-registry JSON mode,
36+
quicktype-class codegen for languages without a MetaObjects port.
37+
38+
### Phase 2 — OpenAPI 3.1
39+
40+
- `openapiFile()` — one document describing the generated REST surface
41+
(accurate-by-construction: derived-CRUD routes + FR-021 declared operations), with
42+
`components.schemas` = Phase 1 canonical output (OpenAPI 3.1 schemas ARE JSON Schema
43+
2020-12). Covers: CRUD paths, the bracketed filter grammar + operator-per-subtype
44+
gating, sort/limit/offset (+`withCount`), M:N traversal routes, error envelopes.
45+
- `meta verify` gains a spec↔routes drift check (regen-to-temp diff, like `--codegen`).
46+
- Emit 3.1 canonical; optional 3.0-downlevel flag (consumer tooling lag is real —
47+
evaluate per demand).
48+
- Multiplier: openapi-generator / Kiota / orval turn the document into client SDKs in
49+
languages MetaObjects doesn't port.
50+
51+
### Phase 3 — protobuf (proto3)
52+
53+
- `protoFile()``.proto` per package + AIP-style CRUD services (Get/List/Create/
54+
Update/Delete + FR-021 custom operations) with `google.api.http` annotations (REST
55+
transcoding / gRPC-Gateway / Connect-compatible).
56+
- **Field numbers come from FR-021 `wireId`** (required on proto-bound projections;
57+
emit-time error if missing). `reservedWireIds/Names``reserved` statements.
58+
- Type mapping (locked by research; conformance-gated):
59+
long→`int64` (ProtoJSON already strings 64-bit — matches our wire contract) ·
60+
decimal→`google.type.Decimal` (string-valued) · currency→`int64` minor units (NOT
61+
`google.type.Money` — Money is units+nanos, a different contract; offer Money only as
62+
an opt-in with explicit conversion) · uuid→`string` (never `bytes`: endianness
63+
hazards) · timestamp→`google.protobuf.Timestamp` (UTC) · date/time→`google.type.Date`
64+
/ `TimeOfDay` · nullable→proto3 `optional` (never wrapper types) · enum→synthesized
65+
`<ENUM>_UNSPECIFIED = 0` zero value + prefixed member names + member `wireId`s ·
66+
`field.object @storage: jsonb``google.protobuf.Struct`, flattened/subdocument→nested
67+
message · arrays→`repeated`.
68+
- **CI gate**: document (and scaffold in `meta init` recipes) `buf breaking
69+
--against` on the emitted, committed protos (WIRE_JSON category) — wire-compat
70+
regression detection is inherited from buf rather than re-implemented.
71+
- Explicit non-goals: no runtime descriptor construction in this FR (per-language
72+
descriptor APIs are uneven — notably no dynamic message support in one major port
73+
ecosystem) and no gRPC *server* codegen — adopters run protoc/buf on the emitted
74+
contract with their existing toolchains.
75+
76+
## Conformance
77+
78+
New `fixtures/contract-conformance/` (or per-emitter fixture trees): golden emitted
79+
artifacts byte-gated from shared input metadata; strict-profile outputs validated
80+
against provider schema-subset rules; proto outputs additionally `buf lint`/`buf
81+
breaking`-clean in CI.
82+
83+
## Open questions
84+
85+
1. Package/file layout for emitted artifacts (per-entity vs per-package files; where in
86+
`targets` they land).
87+
2. OpenAPI security schemes: out of metadata scope (config passthrough?) or modeled?
88+
3. Whether Phase-1 strict profile needs per-provider sub-variants or one intersection
89+
profile suffices (start: one).
90+
4. proto package naming + `option java_package` etc. mapping from `::` packages.
91+
92+
## Dependencies
93+
94+
- FR-021 (declared operations + `wireId`) for Phase 3 and for non-CRUD coverage in
95+
Phase 2; Phases 1–2 are useful with derived CRUD alone and can ship first.
Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# FR-023 — Metadata packages: cross-repo distribution + reuse (design)
2+
3+
_Status: PROPOSED (design sketch — needs brainstorming + plan before implementation)._
4+
_Date: 2026-06-11._
5+
6+
## Problem
7+
8+
The core polyglot workflow is: a **shared base model** (common entities, abstracts,
9+
enums, validation) maintained once, consumed by many downstream apps **in different
10+
languages and different repos**, each augmenting it locally via `extends` and overlays.
11+
12+
The loader mechanics for this already ship — `MetaDataSource` composition
13+
(directory/file/URI/in-memory) and overlay merge by `package`+`name` with per-node
14+
source attribution work across arbitrarily many sources. What's missing is the
15+
**convention**: how a shared model is packaged, versioned, published, resolved, and
16+
declared by a consumer in each language ecosystem. Today it's hand-wired glue (point a
17+
source at a checked-out path); it should be a one-liner.
18+
19+
## Design direction
20+
21+
### 1. The metadata package artifact
22+
23+
A metadata package is a **code-free artifact** containing:
24+
25+
```
26+
metaobjects/ # the standard tree (canonical JSON and/or YAML)
27+
meta.common.json
28+
meta.commerce.json
29+
metaobjects.pkg.json # manifest: name, version, exported metamodel packages,
30+
# provider requirements (custom types it depends on)
31+
```
32+
33+
Published through each ecosystem's normal registry — npm / Maven Central / PyPI / NuGet
34+
— as an ordinary versioned dependency. No runtime code inside; it is data. (One
35+
artifact CAN be published to multiple registries from one source repo — recipe, not
36+
mechanism.)
37+
38+
### 2. Consumer declaration (per-port resolution, one shared semantic)
39+
40+
`.metaobjects/config.json` (the JSON config, parseable by all tooling) gains:
41+
42+
```jsonc
43+
{ "metadataDependencies": [
44+
{ "package": "@acme/common-model", "version": "^2.1.0" } // resolved per ecosystem
45+
]}
46+
```
47+
48+
- **TS/Node**: resolve via `node_modules/<pkg>/metaobjects/`.
49+
- **Java/Kotlin**: a Maven/Gradle dependency whose jar carries `metaobjects/` as a
50+
classpath resource; the Maven plugin resolves and feeds it to the loader (classpath
51+
resource loading is the JVM-native path).
52+
- **Python**: package-data path via `importlib.resources`.
53+
- **C#**: NuGet `contentFiles` path.
54+
- The loader composes: dependency sources first (in declared order), local
55+
`metaobjects/**` last — so local overlays win last-writer-wins attr conflicts,
56+
matching existing overlay semantics. Provenance attribution records the contributing
57+
package+version on every node (extends FR-5c attribution).
58+
59+
### 3. Semantics + guardrails
60+
61+
- `extends` and overlays work across package boundaries exactly as across files
62+
(existing behavior — conformance-gated with new cross-package fixtures).
63+
- **Same-package-name collision** between two dependencies → load error
64+
(`ERR_DUPLICATE_DECLARATION` escalated from warn when sources are different
65+
dependencies and the merge is not a declared overlay) — settle exact rule in
66+
brainstorming.
67+
- Custom-type requirements: the manifest names required providers; loading a package
68+
without its providers registered fails with the existing provider error codes
69+
(ADR-0023 strictness preserved — a metadata package cannot smuggle vocabulary in).
70+
- Version conflicts (two apps want different shared-model versions) are the consuming
71+
ecosystem's resolver problem (npm/Maven semantics), not re-solved here; the manifest
72+
version is recorded in attribution for drift forensics.
73+
- `meta verify` reports per-dependency provenance (which package contributed what) so
74+
a shared-model upgrade that changes downstream codegen is attributable.
75+
76+
### 4. What this is NOT
77+
78+
- Not a hosted registry or remote-fetch protocol — resolution rides each language's
79+
package manager; the loader still only reads local files/resources at build time.
80+
- Not runtime schema distribution — this FR is about build-time model reuse. (Runtime
81+
loading from other sources already exists via `MetaDataSource`; conventions for that
82+
are out of scope here.)
83+
84+
## Conformance
85+
86+
- New cross-package fixtures in `fixtures/conformance/`: dependency-then-local overlay
87+
merge order, cross-package `extends`, collision errors, provider-requirement errors,
88+
attribution shape. All 5 ports load the same fixture "packages" (as plain dirs in the
89+
corpus) — the per-ecosystem resolution is port-specific glue tested per port.
90+
91+
## Open questions
92+
93+
1. Manifest shape + name (`metaobjects.pkg.json`?) and whether the manifest is required
94+
(lean: required for dependencies, absent for apps).
95+
2. Exact collision/override rules between two dependencies (vs dependency→local, which
96+
is settled: local wins).
97+
3. Lockfile story: rely on ecosystem lockfiles only, or record resolved versions in
98+
`.metaobjects/` state for cross-tool reproducibility?
99+
4. YAML vs canonical JSON inside published packages (lean: canonical JSON only —
100+
interchange form, no desugar variance).
101+
5. `meta init` scaffold for *authoring* a metadata package (publish workflows per
102+
registry).
103+
104+
## Dependencies / consumers
105+
106+
- Consumes: `MetaDataSource` composition + overlay/extends + FR-5c attribution (all
107+
shipped).
108+
- Consumed by: every multi-repo adopter; future organization-level tooling can build on
109+
the same manifest.

spec/roadmap.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,9 @@ _Last refreshed 2026-05-30._
7474

7575
- **FR-019 — Shared + externally-provided enums.** Stop redeclaring a `field.enum` inline in every consuming entity: a package-level abstract `field.enum` materializes ONE standalone enum type per port (the existing D6 `extends` reuse vocabulary), and **`@provided: true`** — a provenance flag on the named-type *declaration* (not the field), shared cross-type with value objects — references an existing hand-written type instead of emitting one (per-port namespace via codegen config, never a metadata FQN — retires the C#-only `@csEnumType`). Decision in [ADR-0026](decisions/ADR-0026-shared-and-provided-named-types.md); implementation spec `docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md`. (Generators are now subclass-extensible across all 5 ports, so this lands on open seams.)
7676
- **MCP exposure of declared prompts/tools** — the remaining library-side piece of the prompt-construction pillar. Surface a `template.output` / tool declaration over the Model Context Protocol (model-agnostic) so an LLM host can discover + register it, built on the shipped render / payload / verify / FR-006 / FR-010 primitives. Designed in `docs/superpowers/specs/2026-05-22-fr-004-cross-language-prompt-construction-design.md`.
77+
- **FR-021 — `api` metadata type + contract projections.** Declared API surfaces (operations + per-protocol bindings) over the derived-CRUD default; operation payloads are `object.value` projections (the same `origin.*` machinery as prompts and views); wire-stable `wireId` numbering lives on the contract projection (never the entity) so domain evolution can't break wire compat. Design sketch: `docs/superpowers/specs/2026-06-11-fr-021-api-metadata-and-contract-projections-design.md`.
78+
- **FR-022 — Contract emitters (Tier-2): JSON Schema 2020-12 (canonical + strict structured-output profile) → OpenAPI 3.1 → protobuf.** One shared neutral engine per ADR-0020; strict profile targets the cross-provider structured-output intersection (also MCP input schemas); proto emission consumes FR-021 `wireId`s, maps types per the locked table (Decimal/minor-units/uuid-string/optional/UNSPECIFIED-enum-zero), and inherits wire-compat regression detection via `buf breaking` on the emitted artifacts. Design sketch: `docs/superpowers/specs/2026-06-11-fr-022-contract-emitters-design.md`.
79+
- **FR-023 — Metadata packages: cross-repo distribution + reuse.** A code-free, versioned artifact (the `metaobjects/` tree + manifest) published through each ecosystem's registry (npm/Maven/PyPI/NuGet) and declared via `metadataDependencies` in config; loader source composition + overlay/extends across package boundaries (mechanics already shipped) with dependency-then-local merge order, collision rules, and per-package provenance attribution. Design sketch: `docs/superpowers/specs/2026-06-11-fr-023-metadata-packages-design.md`.
7780

7881
### Tracked outside this library repo (not roadmap work here)
7982

0 commit comments

Comments
 (0)