Skip to content

Commit 1e62f64

Browse files
dmealingclaude
dmealing
and
claude
committed
feat(migrate): physical index/constraint attrs + auto schema-scope + adoption fixes
Enables non-destructive adoption of an existing Postgres database (verify --db / migrate --from-db reaching zero drift against a hand-built schema) and richer index modeling. All physical-storage concerns are contributed by the db provider (spec/metamodel/db.json `extends`), not core. db provider — new physical attrs on core identity subtypes: - identity.secondary @orders — per-key sort direction (asc|desc) → DESC index keys - identity.secondary @where — partial-index predicate → WHERE (<pred>) - identity.reference @constraintName — FK constraint-name override (adopt an existing DB's naming without a rename) Wired end-to-end: IndexDescriptor {orders?, where?}, buildSecondaryIndexes, renderCreateIndex (DESC + WHERE), readPgIndexes (indoption + pg_get_expr(indpred); expression-key indexes skipped), indexEquals (positional orders + normalized predicate). FkDescriptor honours @constraintName. diff — auto schema scope (DiffArgs.scopeSchemas): the diff manages only the schemas the expected/metadata side declares; a table in an undeclared schema belongs to another owner and is left untouched. Makes per-owner drift gates clean with no manual config; an explicit set overrides; an empty model preserves prior whole-DB behavior. introspect — gen_random_uuid() classified as an expression default: parsePgDefault now treats any bare function-call default as an expression (mirrors the metadata-side EXPR_DEFAULT_PATTERNS), so a uuid-PK column round-trips without a spurious default diff. fix(metadata) — allowedValues on an isArray attr validates each ELEMENT, not the array as a whole (previously always failed; @orders is the first isArray+allowedValues attr). Tests: index-partial-ordered, diff-schema-scope, expected-schema-fk-constraint-name, isArray-allowedValues cases; updated diff-schema-aware + identity/coverage gates; regenerated the embedded metamodel, registry manifest, metamodel docs, and Python spec copies. metadata + migrate-ts suites green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent f0d2fa6 commit 1e62f64

21 files changed

Lines changed: 727 additions & 110 deletions

File tree

‎fixtures/metamodel-docs/expected/providers.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ DB-domain attributes — @column / @db.indexed / @dbColumnType on every field, @
6161
- `field.time`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType`
6262
- `field.timestamp`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType`
6363
- `field.uuid`: `@column`, `@db.indexed`, `@dbColumnType`
64+
- `identity.reference`: `@constraintName`
65+
- `identity.secondary`: `@orders`, `@where`
6466
- `source.rdb`: `@function`, `@kind`, `@materializedView`, `@parameterRef`, `@proc`, `@role`, `@schema`, `@table`, `@view`
6567

6668
## metaobjects-documentation

‎fixtures/metamodel-docs/expected/types/identity.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ A foreign-key reference to another entity (@references target; @enforce toggles
3737

3838
| Attribute | Type | Required | Default | Allowed values | Provider | Description |
3939
| --- | --- | --- | --- | --- | --- | --- |
40+
| `@constraintName` | string | no | | | metaobjects-db | Physical foreign-key constraint name override. Absent → the backend's auto-derived default (e.g. `<table>_<firstFkColumn>_fk`). Lets a model adopt an existing database whose FK constraints follow a different naming convention without a destructive rename. RDB-physical — contributed by the db provider. |
4041
| `@enforce` | boolean | no | | | metaobjects-core-types | When true (default), the backend physically enforces the reference (SQL FK constraint, document validation rule, graph edge guarantee). Set false to declare a logical reference for navigation/typing/codegen only — the value may dangle at the backend level. |
4142
| `@fields` | string[] | yes | | | metaobjects-core-types | The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite. |
4243
| `@references` | string | yes | | | metaobjects-core-types | Target of the reference. Bare entity name (e.g. 'Program') resolves to that entity's primary identity. Dotted forms ('Program.id' or 'Program.fieldA,fieldB') target an explicit field set on the entity. |
@@ -56,7 +57,9 @@ A secondary index (unique by default via @unique).
5657
| Attribute | Type | Required | Default | Allowed values | Provider | Description |
5758
| --- | --- | --- | --- | --- | --- | --- |
5859
| `@fields` | string[] | yes | | | metaobjects-core-types | The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite. |
60+
| `@orders` | string[] | no | | `asc`, `desc` | metaobjects-db | Physical index-key sort direction, positional to @fields ('asc' \| 'desc'). Omit for all-ascending (the default); a shorter array leaves trailing keys ascending. Drives DESC-ordered index keys (e.g. a recency index on a timestamp). RDB-physical — contributed by the db provider, not core identity. |
5961
| `@unique` | boolean | no | | | metaobjects-core-types | When true (default), the secondary identity is a UNIQUE index; false makes it a plain (non-unique) index. |
62+
| `@where` | string | no | | | metaobjects-db | Partial-index predicate (raw SQL, e.g. "delivered_at IS NULL"). When set, the index covers only rows matching the predicate — smaller and cheaper for queries that always filter on it. Absent = a full index over every row. RDB-physical — contributed by the db provider. |
6063

6164
**Allowed children**
6265

‎fixtures/registry-conformance/coverage-report.json‎

Lines changed: 9 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -24,40 +24,27 @@
2424
{
2525
"key": "field.boolean",
2626
"untestedAttrs": [
27-
"autoSet",
2827
"column",
2928
"dbColumnType",
3029
"example",
3130
"instruction",
32-
"maxLength",
33-
"objectRef",
34-
"precision",
3531
"readOnly",
36-
"scale",
3732
"sortable",
3833
"sortableDefaultOrder",
39-
"storage",
4034
"unique",
4135
"xmlText"
4236
]
4337
},
4438
{
4539
"key": "field.currency",
4640
"untestedAttrs": [
47-
"autoSet",
48-
"column",
4941
"dbColumnType",
5042
"default",
5143
"example",
5244
"instruction",
53-
"maxLength",
54-
"objectRef",
55-
"precision",
5645
"readOnly",
57-
"scale",
5846
"sortable",
5947
"sortableDefaultOrder",
60-
"storage",
6148
"unique",
6249
"xmlText"
6350
]
@@ -71,151 +58,108 @@
7158
"default",
7259
"example",
7360
"instruction",
74-
"maxLength",
75-
"objectRef",
76-
"precision",
7761
"readOnly",
78-
"scale",
7962
"sortable",
8063
"sortableDefaultOrder",
81-
"storage",
8264
"unique",
8365
"xmlText"
8466
]
8567
},
8668
{
8769
"key": "field.decimal",
8870
"untestedAttrs": [
89-
"autoSet",
9071
"column",
9172
"dbColumnType",
9273
"default",
9374
"example",
9475
"instruction",
95-
"maxLength",
96-
"objectRef",
9776
"readOnly",
9877
"sortable",
9978
"sortableDefaultOrder",
100-
"storage",
10179
"unique",
10280
"xmlText"
10381
]
10482
},
10583
{
10684
"key": "field.double",
10785
"untestedAttrs": [
108-
"autoSet",
10986
"column",
11087
"dbColumnType",
11188
"default",
11289
"example",
11390
"instruction",
114-
"maxLength",
115-
"objectRef",
116-
"precision",
11791
"readOnly",
118-
"scale",
11992
"sortable",
12093
"sortableDefaultOrder",
121-
"storage",
12294
"unique",
12395
"xmlText"
12496
]
12597
},
12698
{
12799
"key": "field.enum",
128100
"untestedAttrs": [
129-
"autoSet",
130101
"column",
131102
"dbColumnType",
132103
"example",
133104
"instruction",
134-
"maxLength",
135-
"objectRef",
136-
"precision",
137105
"readOnly",
138-
"scale",
139106
"sortable",
140107
"sortableDefaultOrder",
141-
"storage",
142108
"unique",
143109
"xmlText"
144110
]
145111
},
146112
{
147113
"key": "field.float",
148114
"untestedAttrs": [
149-
"autoSet",
150115
"column",
151116
"db.indexed",
152117
"dbColumnType",
153118
"default",
154119
"example",
155120
"instruction",
156-
"maxLength",
157-
"objectRef",
158-
"precision",
159121
"readOnly",
160-
"scale",
161122
"sortable",
162123
"sortableDefaultOrder",
163-
"storage",
164124
"unique",
165125
"xmlText"
166126
]
167127
},
168128
{
169129
"key": "field.int",
170130
"untestedAttrs": [
171-
"autoSet",
172131
"column",
173132
"dbColumnType",
174133
"example",
175134
"instruction",
176-
"maxLength",
177-
"objectRef",
178-
"precision",
179-
"scale",
180135
"sortableDefaultOrder",
181-
"storage",
182136
"unique",
183137
"xmlText"
184138
]
185139
},
186140
{
187141
"key": "field.long",
188142
"untestedAttrs": [
189-
"autoSet",
190143
"column",
191144
"db.indexed",
192145
"dbColumnType",
193146
"example",
194147
"instruction",
195-
"maxLength",
196-
"objectRef",
197-
"precision",
198-
"scale",
199148
"sortableDefaultOrder",
200-
"storage",
201149
"unique",
202150
"xmlText"
203151
]
204152
},
205153
{
206154
"key": "field.object",
207155
"untestedAttrs": [
208-
"autoSet",
209156
"column",
210157
"dbColumnType",
211158
"default",
212159
"example",
213160
"instruction",
214-
"maxLength",
215-
"precision",
216161
"readOnly",
217162
"required",
218-
"scale",
219163
"sortable",
220164
"sortableDefaultOrder",
221165
"unique",
@@ -225,11 +169,7 @@
225169
{
226170
"key": "field.string",
227171
"untestedAttrs": [
228-
"objectRef",
229-
"precision",
230-
"scale",
231172
"sortableDefaultOrder",
232-
"storage",
233173
"xmlText"
234174
]
235175
},
@@ -243,14 +183,9 @@
243183
"default",
244184
"example",
245185
"instruction",
246-
"maxLength",
247-
"objectRef",
248-
"precision",
249186
"readOnly",
250-
"scale",
251187
"sortable",
252188
"sortableDefaultOrder",
253-
"storage",
254189
"unique",
255190
"xmlText"
256191
]
@@ -263,48 +198,41 @@
263198
"default",
264199
"example",
265200
"instruction",
266-
"maxLength",
267-
"objectRef",
268-
"precision",
269-
"scale",
270201
"sortableDefaultOrder",
271-
"storage",
272202
"unique",
273203
"xmlText"
274204
]
275205
},
276206
{
277207
"key": "field.uuid",
278208
"untestedAttrs": [
279-
"autoSet",
280209
"db.indexed",
281210
"dbColumnType",
282211
"default",
283212
"example",
284213
"instruction",
285-
"maxLength",
286-
"objectRef",
287-
"precision",
288214
"readOnly",
289-
"scale",
290215
"sortable",
291216
"sortableDefaultOrder",
292-
"storage",
293217
"xmlText"
294218
]
295219
},
296220
{
297-
"key": "object.projection",
221+
"key": "identity.reference",
222+
"untestedAttrs": [
223+
"constraintName"
224+
]
225+
},
226+
{
227+
"key": "identity.secondary",
298228
"untestedAttrs": [
299-
"discriminator",
300-
"discriminatorValue"
229+
"orders",
230+
"where"
301231
]
302232
},
303233
{
304234
"key": "object.value",
305235
"untestedAttrs": [
306-
"discriminator",
307-
"discriminatorValue",
308236
"normalize"
309237
]
310238
},

‎fixtures/registry-conformance/expected-registry.json‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2014,6 +2014,13 @@
20142014
"subType": "reference",
20152015
"description": "A foreign-key reference to another entity (@references target; @enforce toggles a physical FK).",
20162016
"attrs": [
2017+
{
2018+
"name": "constraintName",
2019+
"valueType": "string",
2020+
"isArray": false,
2021+
"required": false,
2022+
"description": "Physical foreign-key constraint name override. Absent → the backend's auto-derived default (e.g. `<table>_<firstFkColumn>_fk`). Lets a model adopt an existing database whose FK constraints follow a different naming convention without a destructive rename. RDB-physical — contributed by the db provider."
2023+
},
20172024
{
20182025
"name": "enforce",
20192026
"valueType": "boolean",
@@ -2050,12 +2057,26 @@
20502057
"required": true,
20512058
"description": "The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite."
20522059
},
2060+
{
2061+
"name": "orders",
2062+
"valueType": "string",
2063+
"isArray": true,
2064+
"required": false,
2065+
"description": "Physical index-key sort direction, positional to @fields ('asc' | 'desc'). Omit for all-ascending (the default); a shorter array leaves trailing keys ascending. Drives DESC-ordered index keys (e.g. a recency index on a timestamp). RDB-physical — contributed by the db provider, not core identity."
2066+
},
20532067
{
20542068
"name": "unique",
20552069
"valueType": "boolean",
20562070
"isArray": false,
20572071
"required": false,
20582072
"description": "When true (default), the secondary identity is a UNIQUE index; false makes it a plain (non-unique) index."
2073+
},
2074+
{
2075+
"name": "where",
2076+
"valueType": "string",
2077+
"isArray": false,
2078+
"required": false,
2079+
"description": "Partial-index predicate (raw SQL, e.g. \"delivered_at IS NULL\"). When set, the index covers only rows matching the predicate — smaller and cheaper for queries that always filter on it. Absent = a full index over every row. RDB-physical — contributed by the db provider."
20592080
}
20602081
],
20612082
"children": []

‎server/python/src/metaobjects/spec_metamodel/db.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,21 @@
3838
{ "type": "attr", "subType": "string", "name": "schema", "min": 0, "max": 1, "description": "Optional database schema name (e.g. 'catalog', 'public'). Postgres defaults to 'public'; SQLite rejects any non-default value." },
3939
{ "type": "attr", "subType": "string", "name": "parameterRef", "min": 0, "max": 1, "description": "FR-015: name or FQN of an object.value describing the input shape of this source's callable interface. Permitted on @kind: \"storedProc\" / \"tableFunction\"; rejected on non-callable kinds (table / view / materializedView). Field children of the referenced object.value become the call-site parameter list in declaration order. Symmetric with template.@payloadRef in FR-004 — the typed-input pattern reuses object.value rather than minting a new parameter.* node type." }
4040
]
41+
},
42+
{
43+
"type": "identity",
44+
"subType": "secondary",
45+
"children": [
46+
{ "type": "attr", "subType": "string", "name": "orders", "isArray": true, "min": 0, "max": 1, "allowedValues": ["asc", "desc"], "description": "Physical index-key sort direction, positional to @fields ('asc' | 'desc'). Omit for all-ascending (the default); a shorter array leaves trailing keys ascending. Drives DESC-ordered index keys (e.g. a recency index on a timestamp). RDB-physical — contributed by the db provider, not core identity." },
47+
{ "type": "attr", "subType": "string", "name": "where", "min": 0, "max": 1, "description": "Partial-index predicate (raw SQL, e.g. \"delivered_at IS NULL\"). When set, the index covers only rows matching the predicate — smaller and cheaper for queries that always filter on it. Absent = a full index over every row. RDB-physical — contributed by the db provider." }
48+
]
49+
},
50+
{
51+
"type": "identity",
52+
"subType": "reference",
53+
"children": [
54+
{ "type": "attr", "subType": "string", "name": "constraintName", "min": 0, "max": 1, "description": "Physical foreign-key constraint name override. Absent → the backend's auto-derived default (e.g. `<table>_<firstFkColumn>_fk`). Lets a model adopt an existing database whose FK constraints follow a different naming convention without a destructive rename. RDB-physical — contributed by the db provider." }
55+
]
4156
}
4257
]
4358
}

‎server/typescript/packages/metadata/src/attr-schema-validate.ts‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -251,14 +251,22 @@ function validateNode(
251251
}
252252
}
253253

254-
// Check 3: allowedValues membership (unchanged).
254+
// Check 3: allowedValues membership. For an isArray attr the value is an array,
255+
// so each ELEMENT must be a member (not the array as a whole); a scalar attr
256+
// checks the value directly.
255257
if (spec.allowedValues !== undefined && spec.allowedValues.length > 0) {
256-
if (!spec.allowedValues.includes(value)) {
258+
const allowed = spec.allowedValues;
259+
const offenders = Array.isArray(value)
260+
? value.filter((v) => !allowed.includes(v))
261+
: allowed.includes(value)
262+
? []
263+
: [value];
264+
for (const bad of offenders) {
257265
errors.push(
258266
new ParseError(
259267
`${nodeLabel(node)} attribute '@${inst.name}' has value ` +
260-
`'${String(value)}' which is not one of the allowed values: ` +
261-
`${spec.allowedValues.map((v) => String(v)).join(", ")}`,
268+
`'${String(bad)}' which is not one of the allowed values: ` +
269+
`${allowed.map((v) => String(v)).join(", ")}`,
262270
{ code: "ERR_BAD_ATTR_VALUE", source: node.source },
263271
),
264272
);

0 commit comments

Comments
 (0)