Skip to content

Commit b52e13e

Browse files
dmealingclaude
andcommitted
docs(#208): ADR-0043 + migration/verify docs for @sql/@Unmanaged
Task 11 — the durable docs for the DDL-ownership escape valves. - spec/decisions/ADR-0043-ddl-ownership-escape-valves.md (new, Nygard): the one-axis / two-attr model, the ADR-0037 attribute rationale, the suppression rule, the six loader-validation rules, the cross-port (register+validate) vs TS-only (lowering) split, the adopt-view ceremony, and the documented adopter caveats + deferred follow-ups (@dependsOn for @SQL, matview managed path, opaque-body column verify). - agent-context/skills/metaobjects-verify/references/migration.md: replaced the "no attribute injects hand-written SQL (by design)" statement with a "DDL-ownership escape valves" section documenting @SQL (tool-managed opaque body + adopt-view) and @Unmanaged (external, managed elsewhere). - docs/features/migrations-and-drift.md: an @sql/@Unmanaged note beside the adopt-view step — an irreducible view is now managed-but-opaque instead of accidentally-unmanaged. - CHANGELOG.md [Unreleased]: the #208 entry. - spec/roadmap.md: marked #208 SHIPPED (unreleased) + filed the three deferred follow-ups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NGQ7oSuNcjhsMHWwZzhBwr
1 parent 234f784 commit b52e13e

5 files changed

Lines changed: 148 additions & 7 deletions

File tree

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
77

88
## [Unreleased]
99

10+
### #208 — DDL-ownership escape valves (`@sql` body + `@unmanaged` marker)
11+
12+
Two mutually-exclusive `source.rdb` attributes express *who owns a DB object's DDL* — the escape from "a projection's view is always synthesized from its `origin.*` children" (ADR-0043):
13+
14+
- **`@sql`** — a hand-written view body the tool **registers, fingerprints, and drift-checks but never authors or parses**. The value is the body inside `CREATE <kind> <name> AS …`. It lets a genuinely-irreducible view (recursive CTE, window function, set operation) carry an `extends`-bound identity/fields for row identity and shape *without* the tool mis-synthesizing a wrong base-table passthrough `SELECT` — the **suppression rule** classifies DDL-ownership *before* the derivation decision, closing a silent-wrong-synthesis hole. The `@sql` view rides the existing emit/fingerprint pipeline; a pre-existing unstamped view at its name is `replace-view`-blocked pending the one-time **`meta migrate --allow adopt-view`** adoption ceremony. `@sql` on a writable kind, combined with `@unmanaged`, empty, or combined with `origin.*`/`@filter` is a load error. v1 migrate lowering is `@kind: view` only (matview/proc → an actionable hard error).
15+
- **`@unmanaged: true`** — this DB object (a view **or a table**) is managed elsewhere (Flyway / a hand-migration owns its DDL). `meta migrate` never creates, drops, or drift-checks it (excluded from both the expected and the introspected-actual sides across the online, offline, D1, and `verify` paths); `meta verify --db` reports it as *external (declared)*. An inbound FK from a managed table still resolves the external table's physical name.
16+
17+
**Registration + the six fail-closed loader-validation rules (R1–R6) ship in all five ports** (TS / C# / Java / Kotlin / Python), resolving `MetaSource` accessors following the `@role`/`effectiveKind` precedent (ADR-0039), conformance-gated by shared `fixtures/conformance/` error-fixtures — every port emits the same code per rule (`ERR_SQL_BODY_WITH_UNMANAGED` / `ERR_SQL_BODY_ON_WRITABLE_KIND` / `ERR_BAD_ATTR_VALUE` / `ERR_ORIGIN_UNDER_SQL_BODY` / `WARN_ORIGIN_UNDER_UNMANAGED`). All migrate/verify **lowering is TS-only** (ADR-0015). An `xhigh` review before merge caught a real offline-`migrate` DROP-for-external-table bug (the offline `planOffline` path did not thread the unmanaged-name set), fixed + regression-tested. Deferred follow-ups: `@dependsOn` for `@sql` views, the matview managed path, and opaque-body column-name verification. Designed in `docs/superpowers/specs/2026-07-17-issue-208-ddl-ownership-escape-valves-design.md`.
18+
1019
### #214 — entity read-view codegen READ half (all five ports)
1120

1221
Completes the read side of the FR-024 §7 entity read-view (#213 shipped the write/schema half — the write table excludes derived `origin.*` fields and the replica view is emitted + migrate-converged). A **write-through entity** (a writable `table` source + a read-only replica `view` source + derived fields) now generates a hybrid read/write surface in TypeScript:

agent-context/skills/metaobjects-verify/references/migration.md

Lines changed: 53 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -129,8 +129,10 @@ A non-unique recency index is `index.lookup`:
129129

130130
The `@where` / `@using` / `@expr` / `@orders` attributes are **index** physical
131131
escapes on `identity.secondary` / `index.lookup` — they are NOT a raw-SQL escape
132-
hatch for views. There is no attribute that injects hand-written SQL into a
133-
projection view body (by design — see below).
132+
hatch for views. For a genuinely-irreducible view body, use the `source.rdb`
133+
**`@sql`** escape (a tool-managed, opaque hand-written body — see the
134+
"DDL-ownership escape valves" section below); for a DB object owned entirely by
135+
Flyway / a hand-migration, use **`@unmanaged`**.
134136

135137
## Projection views (generated view DDL)
136138

@@ -151,10 +153,55 @@ canonical view-SQL emitter shared with drift detection.
151153
`metaobjects-audit` skill, not here.
152154

153155
**Do not hand-author view SQL for a shape origins can express** — model it as a
154-
projection so the view DDL is generated and drift-checked. The only case for
155-
hand-written view DDL is a genuinely irreducible view (recursive CTE, window
156-
function, set operation) that origins can't express; carry that in a hand-edited
157-
migration file.
156+
projection so the view DDL is generated and drift-checked. For a genuinely
157+
irreducible view (recursive CTE, window function, set operation) that origins
158+
can't express, use the `@sql` escape below rather than a hand-edited migration
159+
file — that keeps the view tool-managed (emitted, fingerprinted, drift-checked)
160+
instead of accidentally unmanaged.
161+
162+
## DDL-ownership escape valves (`@sql` / `@unmanaged`) — #208
163+
164+
Two mutually-exclusive `source.rdb` attributes express *who owns a DB object's
165+
DDL* (ADR-0043). They are the escape from "a projection's view is always
166+
synthesized from its `origin.*` children."
167+
168+
**`@sql`** — a hand-written view body the tool **registers, fingerprints, and
169+
drift-checks but never authors or parses**. The value is the body *inside*
170+
`CREATE VIEW <name> AS …` (not the `CREATE` wrapper, not the name). Legal only on
171+
a read-only kind; v1 migrate lowers it on `@kind: view` only (matview/proc → a
172+
hard error, deferred). Authored sigil-free in YAML as a block scalar:
173+
174+
```yaml
175+
object.projection:
176+
name: OrgTree
177+
children:
178+
- source.rdb: { kind: view, view: v_org_tree, sql: |
179+
WITH RECURSIVE t AS (
180+
SELECT id, parent_id FROM org WHERE parent_id IS NULL
181+
UNION ALL SELECT o.id, o.parent_id FROM org o JOIN t ON o.parent_id = t.id)
182+
SELECT * FROM t }
183+
- field.long: { name: id, extends: Org.id }
184+
- identity.primary: { extends: Org.pk }
185+
```
186+
187+
The `extends`-bound identity/fields declare the read model's shape and row
188+
identity *without* triggering wrong synthesis (the suppression rule). The view is
189+
emitted verbatim with a fingerprint COMMENT stamp; a second `meta migrate` is a
190+
no-op. **Adopting a pre-existing hand-written view** at that name: the first diff
191+
reports `replace-view` **blocked** (an unstamped view is indistinguishable from
192+
someone else's SQL) — run **`meta migrate --allow adopt-view`** once to stamp it,
193+
then it converges. `@sql` **forbids** `origin.*` children and a `@filter` on the
194+
same host (two sources of truth → load error).
195+
196+
**`@unmanaged: true`** — this DB object (a view **or a table**) is managed
197+
elsewhere (Flyway / a hand-migration owns its DDL). `meta migrate` never creates,
198+
drops, or drift-checks it; `meta verify --db` reports it as *external (declared)*.
199+
Legal on any `@kind`, including `table` (the Flyway-owned-entity case). An FK from
200+
a managed table into an `@unmanaged` table resolves its physical name, but the
201+
external object must exist before that FK is applied (a documented ordering caveat,
202+
not enforced).
203+
204+
`@sql` and `@unmanaged` are **mutually exclusive** on one source.
158205

159206
## Adopting an existing database (non-destructive)
160207

docs/features/migrations-and-drift.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,21 @@ closes the loop on the doctrine in
183183
sitting where a projection expects one is now **visible to `meta verify --db`** as drift,
184184
instead of being silently invisible to it.
185185

186+
**Genuinely-irreducible views and externally-owned objects (`@sql` / `@unmanaged`, #208).**
187+
Some read models cannot be expressed as `origin.*` (a recursive CTE, a window function, a
188+
set operation). Rather than hand-write such a view *outside* the tool — where it degrades
189+
to accidentally-unmanaged — carry its body in a `source.rdb` **`@sql`** attribute. The tool
190+
registers, fingerprints, and drift-checks that verbatim body (never parsing it) exactly like
191+
a synthesized view: it emits `CREATE VIEW … AS <your body>` with a fingerprint stamp, a
192+
second migrate is a no-op, and a pre-existing hand-written view at that name is adopted via
193+
the same `--allow adopt-view` step above. An `@sql` view may carry an `extends`-bound
194+
`identity.primary` / fields (for row identity and read-model shape) without the tool
195+
mis-synthesizing a body. For a DB object whose DDL is owned entirely elsewhere (Flyway, a
196+
hand-migration) — a view **or a table** — mark its source **`@unmanaged: true`**: `meta
197+
migrate` never creates, drops, or drift-checks it, and `meta verify --db` reports it as
198+
*external (declared)*. The two are mutually exclusive. See
199+
[ADR-0043](../../spec/decisions/ADR-0043-ddl-ownership-escape-valves.md).
200+
186201
## Verified by
187202

188203
The following conformance fixtures gate this feature's behavior across ports:
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# ADR-0043: DDL-ownership escape valves (`@sql` body + `@unmanaged` marker)
2+
3+
## Status
4+
5+
**Accepted** (2026-07-17). Implements FR #208 (chartered as the #195 follow-up (a)). Applies the ADR-0037 vocabulary-expansion framework; composes with the projection/view model (ADR-0028), the source-v2 paradigm (ADR-0007), and the TS-owns-schema split (ADR-0015).
6+
7+
## Context
8+
9+
A projection's view body is **synthesized** from its `origin.*` children through the one canonical view-SQL emitter (`build-projection-views.ts`). This is the right default — a projection's `CREATE VIEW` is derived from the model, so a hand-written view is drift. But two cases have no home:
10+
11+
1. **A genuinely-irreducible view.** Some read models cannot be expressed as `origin.*` (recursive CTEs, window functions, lateral joins, vendor-specific SQL). Today the author's only recourse is to hand-write the view *outside* the tool — where it degrades to **accidentally unmanaged**: `meta migrate` neither owns nor emits it, and `meta verify --db` cannot drift-check it.
12+
13+
2. **The silent-wrong-synthesis hole.** The moment such a hand-bodied view declares its lineage — an `extends`-bound `identity.primary` for row identity, or `extends`-bound fields for shape — `viewIsDerived()` flips **true** and the tool **synthesizes a wrong body** (a trivial base-table passthrough `SELECT`) in place of the author's intended SQL. That silent-wrong-synthesis is worse than an abort.
14+
15+
Separately, an author may want a DB object (a view **or a table**) whose DDL is owned entirely **elsewhere** — a Flyway migration, a hand-run script — so `meta migrate` must never create, drop, or drift-check it.
16+
17+
These are the two non-default states of **one axis: who owns a DB object's DDL**`derived` (default; the tool synthesizes) · `supplied` (the author writes the body, the tool manages its lifecycle) · `external` (someone else owns the DDL).
18+
19+
## Decision
20+
21+
### 1. Two attributes on `source.rdb`, one axis
22+
23+
- **`@sql`** (string) — a hand-written body the tool **registers, fingerprints, and drift-checks but never authors or parses**. The body goes *inside* `CREATE <kind> <physicalName> AS …` — never the `CREATE` wrapper, never the object name.
24+
- **`@unmanaged`** (boolean) — this object's DDL is owned elsewhere; the tool never touches it.
25+
26+
They are **mutually exclusive** on a single source ("emit + fingerprint this" contradicts "never touch this"). Both are attributes on `source.rdb` (ADR-0007 — physical concerns live on the source child), so granularity is **per-source**: a write-through entity can keep its table managed while marking only its read-view source `@unmanaged`.
27+
28+
### 2. ADR-0037 rationale — why attributes, not a subtype or a `@kind`
29+
30+
- `@sql`: **physical-only** (Step 1, decisive) — the body changes no native type and no logical meaning; the projection's declared fields still type the read model and the wire contract is untouched. It is a pure physical-DDL payload, the same category as the existing `@where`/`@expr` raw-SQL escapes. Not a subtype (still an RDB view, no distinct native type/behavior/child vocabulary); not a `@kind` (a hand-bodied view is still `@kind: view` — DDL-provenance is a *second* axis, and folding it into `@kind` makes `@kind` the ADR-0037-forbidden catch-all). Name `@sql` (not `@sqlView`, which is view-specific and self-contradictory on a matview; not `@ddl`, which implies the full `CREATE` wrapper; not `@body`) generalizes by pattern — a future non-RDB paradigm subtype registers its own body attr.
31+
- `@unmanaged`: tool-lifecycle **configuration** → attribute (Step 2c). A boolean exception-flag (the common case is absent). Rejected `@managed: false` (2c forbids a default-true opt-out) and an enum (`@lifecycle`), since the "supplied" state is already carried by `@sql`'s presence.
32+
33+
### 3. The suppression rule (the load-bearing, non-deferrable piece)
34+
35+
An `@sql`- or `@unmanaged`-marked read source **suppresses derivation-classification entirely**: DDL-ownership is classified **before** `viewIsDerived`. This is what lets an escape-valve host legally carry `extends`-bound fields and a borrowed `identity.primary { extends: "Base.id" }` as pure shape / row-identity declarations *without* the tool mis-synthesizing a body. ADR-0028's "self-declared under external assembly" already licenses these declarations — no ADR-0028 change needed.
36+
37+
### 4. Cross-port scope split
38+
39+
- **All five ports** (TS / C# / Java / Kotlin / Python): register the two attrs on `source.rdb`; resolving `MetaSource` accessors (`sqlBody`/`isUnmanaged` etc., following the `@role`/`effectiveKind` *resolving* precedent — **not** the `@dbColumnType` own-only exception, ADR-0039); and the six fail-closed loader-validation rules (§5), conformance-gated by shared `fixtures/conformance/` error-fixtures.
40+
- **TS-only** (ADR-0015 — schema is TS-owned): all migrate/verify *lowering* — the `@sql` verbatim-emit + fingerprint + adopt-view ceremony, and the `@unmanaged` skip + act-side silencing.
41+
42+
### 5. Loader validation — six fail-closed rules (cross-port)
43+
44+
| # | Rule | Code |
45+
|---|------|------|
46+
| R1 | `@sql` **and** `@unmanaged` on the same source | `ERR_SQL_BODY_WITH_UNMANAGED` |
47+
| R2 | `@sql` on a writable `@kind` (`table`) | `ERR_SQL_BODY_ON_WRITABLE_KIND` |
48+
| R3 | `@sql` present but empty/whitespace | `ERR_BAD_ATTR_VALUE` |
49+
| R4 | `origin.*`-bearing field under an `@sql` host | `ERR_ORIGIN_UNDER_SQL_BODY` |
50+
| R5 | `object.projection` `@filter` (#207) + `@sql` host | `ERR_ORIGIN_UNDER_SQL_BODY` |
51+
| R6 | `origin.*`-bearing field under an `@unmanaged` host | **WARN** `WARN_ORIGIN_UNDER_UNMANAGED` |
52+
53+
The R4-error / R6-warn asymmetry is deliberate: `@sql` is a second body (two sources of truth for the *same* data); `@unmanaged` acts on nothing, so a documented-but-unacted-on lineage is benign. `@sql` (R4) takes priority over `@unmanaged` (R6) when a host declares both markers across different sources.
54+
55+
### 6. Migrate lowering (TS-only)
56+
57+
- **`@sql`** rides the *existing* pipeline: push a verbatim `ExpectedView` (`name`=physicalName, `sql`=the verbatim body, `dependsOn`=the `extends`-bound anchor tables, `columns` **omitted**). `buildExpectedSchema` fingerprints `v.sql`; emit stamps the COMMENT marker; an absent `columns` makes the diff fail safe to a gated drop+create instead of a wrong `CREATE OR REPLACE`. A pre-existing **unstamped** view at the name is `replace-view` **blocked pending `migrate --allow adopt-view`** — the one-time adoption ceremony. `@sql` on a non-`view` kind (matview/proc/tableFunction) is an actionable hard error (D4 — not yet migrate-managed).
58+
- **`@unmanaged`** is excluded from both sides: skipped from `expected` (views in `build-projection-views`; tables in `buildExpectedSchema` Pass 1, **but kept in `entityToTable`** so an inbound FK resolves the external table's physical name), and excluded from the introspected `actual` via a `collectUnmanagedNames` set threaded from every diff entry point (online, offline `planOffline`, D1, and `verify`'s `computeDrift`). Net: a declared-external object produces **silence**, not a policy-gated drop. `verify --db` annotates it "external (declared)".
59+
60+
## Consequences
61+
62+
- A genuinely-irreducible view is now **managed-but-opaque**: `meta migrate` owns its lifecycle (emit / fingerprint-drift / gated drop) without ever parsing its body, and the one-time `--allow adopt-view` ceremony adopts a pre-existing hand-written view.
63+
- The "generalizes to any object" half falls out for free: `@unmanaged` on a `@kind: table` is the Flyway-owned-entity case.
64+
- **Documented adopter caveats (not errors):**
65+
- An FK *from a managed table into an `@unmanaged` table* has no migration-**ordering** guarantee versus the external tool — the external object must exist before the managed FK is applied.
66+
- An `@unmanaged` view is **not** drop/recreated around a managed dependency's column-altering `ALTER` — the tool has no body for an external view and cannot recreate it; Postgres blocking the `ALTER` is the adopter's signal to coordinate.
67+
- **Deferred follow-ups (filed on the roadmap):**
68+
- **`@dependsOn` for `@sql` views** — a `@sql` view's `dependsOn` is derived from `extends`-bound anchors only, so a table the opaque body JOINs but does not surface as a bound column is not tracked for auto-recreate around a column `ALTER` (§11 YAGNI — defer the explicit `@dependsOn` attr until an adopter hits it).
69+
- **The matview managed path**`@sql` on `@kind: materializedView` needs `pg_matviews` introspection + COMMENT-on-matview plumbing + a REFRESH story.
70+
- **Opaque-body column-name verification** — PG exposes an `@sql` view's output columns via `information_schema.columns`, so `verify` could eventually column-name-check an opaque body against the declared field set.

0 commit comments

Comments
 (0)