Skip to content

Commit 0518071

Browse files
dmealingclaude
dmealing
and
claude
authored
feat(migrate): physical index/constraint attrs + auto schema-scope + DB-adoption fixes (#42)
* feat(migrate): physical index/constraint attrs + auto schema-scope + adoption fixes Enables non-destructive adoption of an existing Postgres database (verify --db / migrate --from-db reaching zero drift against a hand-built schema) and richer index modeling. All physical-storage concerns are contributed by the db provider (spec/metamodel/db.json `extends`), not core. db provider — new physical attrs on core identity subtypes: - identity.secondary @orders — per-key sort direction (asc|desc) → DESC index keys - identity.secondary @where — partial-index predicate → WHERE (<pred>) - identity.reference @constraintName — FK constraint-name override (adopt an existing DB's naming without a rename) Wired end-to-end: IndexDescriptor {orders?, where?}, buildSecondaryIndexes, renderCreateIndex (DESC + WHERE), readPgIndexes (indoption + pg_get_expr(indpred); expression-key indexes skipped), indexEquals (positional orders + normalized predicate). FkDescriptor honours @constraintName. diff — auto schema scope (DiffArgs.scopeSchemas): the diff manages only the schemas the expected/metadata side declares; a table in an undeclared schema belongs to another owner and is left untouched. Makes per-owner drift gates clean with no manual config; an explicit set overrides; an empty model preserves prior whole-DB behavior. introspect — gen_random_uuid() classified as an expression default: parsePgDefault now treats any bare function-call default as an expression (mirrors the metadata-side EXPR_DEFAULT_PATTERNS), so a uuid-PK column round-trips without a spurious default diff. fix(metadata) — allowedValues on an isArray attr validates each ELEMENT, not the array as a whole (previously always failed; @orders is the first isArray+allowedValues attr). Tests: index-partial-ordered, diff-schema-scope, expected-schema-fk-constraint-name, isArray-allowedValues cases; updated diff-schema-aware + identity/coverage gates; regenerated the embedded metamodel, registry manifest, metamodel docs, and Python spec copies. metadata + migrate-ts suites green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * conformance: reconcile Python/Java/Kotlin/C# ports with the new db identity attrs Registers the three db-provider identity attrs (identity.secondary @orders + @where, identity.reference @constraintName) in each non-TS port so every port's emitted registry manifest byte-matches the canonical fixtures/registry-conformance/ expected-registry.json (canonical/TS is the source of truth). - Python: db_provider.py extends TYPE_IDENTITY secondary/reference (db_constants.py constants); AttrSchema is_array + descriptions sourced byte-exact from db.json. - Java: SecondaryIdentity / ReferenceIdentity register the attrs via the fluent builder; descriptions land via applySpecDescriptions from the bundled spec/metamodel/db.json (Kotlin reuses the JVM registration — no separate change). - C#: DbProvider extends the identity subtypes with DbSchema schemas; SpecMetamodel/ db.json mirror synced. The manifest carries no allowedValues, so @orders' asc/desc set is validation-only. Verified locally: registry-conformance passes for all four ports (python 3, java 3, kotlin 1, csharp 3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent a29e5f6 commit 0518071

30 files changed

Lines changed: 885 additions & 112 deletions

File tree

‎fixtures/metamodel-docs/expected/providers.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ DB-domain attributes — @column / @db.indexed / @dbColumnType on every field, @
6161
- `field.time`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType`
6262
- `field.timestamp`: `@autoSet`, `@column`, `@db.indexed`, `@dbColumnType`
6363
- `field.uuid`: `@column`, `@db.indexed`, `@dbColumnType`
64+
- `identity.reference`: `@constraintName`
65+
- `identity.secondary`: `@orders`, `@where`
6466
- `source.rdb`: `@function`, `@kind`, `@materializedView`, `@parameterRef`, `@proc`, `@role`, `@schema`, `@table`, `@view`
6567

6668
## metaobjects-documentation

‎fixtures/metamodel-docs/expected/types/identity.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ A foreign-key reference to another entity (@references target; @enforce toggles
3737

3838
| Attribute | Type | Required | Default | Allowed values | Provider | Description |
3939
| --- | --- | --- | --- | --- | --- | --- |
40+
| `@constraintName` | string | no | | | metaobjects-db | Physical foreign-key constraint name override. Absent → the backend's auto-derived default (e.g. `<table>_<firstFkColumn>_fk`). Lets a model adopt an existing database whose FK constraints follow a different naming convention without a destructive rename. RDB-physical — contributed by the db provider. |
4041
| `@enforce` | boolean | no | | | metaobjects-core-types | When true (default), the backend physically enforces the reference (SQL FK constraint, document validation rule, graph edge guarantee). Set false to declare a logical reference for navigation/typing/codegen only — the value may dangle at the backend level. |
4142
| `@fields` | string[] | yes | | | metaobjects-core-types | The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite. |
4243
| `@references` | string | yes | | | metaobjects-core-types | Target of the reference. Bare entity name (e.g. 'Program') resolves to that entity's primary identity. Dotted forms ('Program.id' or 'Program.fieldA,fieldB') target an explicit field set on the entity. |
@@ -56,7 +57,9 @@ A secondary index (unique by default via @unique).
5657
| Attribute | Type | Required | Default | Allowed values | Provider | Description |
5758
| --- | --- | --- | --- | --- | --- | --- |
5859
| `@fields` | string[] | yes | | | metaobjects-core-types | The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite. |
60+
| `@orders` | string[] | no | | `asc`, `desc` | metaobjects-db | Physical index-key sort direction, positional to @fields ('asc' \| 'desc'). Omit for all-ascending (the default); a shorter array leaves trailing keys ascending. Drives DESC-ordered index keys (e.g. a recency index on a timestamp). RDB-physical — contributed by the db provider, not core identity. |
5961
| `@unique` | boolean | no | | | metaobjects-core-types | When true (default), the secondary identity is a UNIQUE index; false makes it a plain (non-unique) index. |
62+
| `@where` | string | no | | | metaobjects-db | Partial-index predicate (raw SQL, e.g. "delivered_at IS NULL"). When set, the index covers only rows matching the predicate — smaller and cheaper for queries that always filter on it. Absent = a full index over every row. RDB-physical — contributed by the db provider. |
6063

6164
**Allowed children**
6265

‎fixtures/registry-conformance/coverage-report.json‎

Lines changed: 9 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -24,40 +24,27 @@
2424
{
2525
"key": "field.boolean",
2626
"untestedAttrs": [
27-
"autoSet",
2827
"column",
2928
"dbColumnType",
3029
"example",
3130
"instruction",
32-
"maxLength",
33-
"objectRef",
34-
"precision",
3531
"readOnly",
36-
"scale",
3732
"sortable",
3833
"sortableDefaultOrder",
39-
"storage",
4034
"unique",
4135
"xmlText"
4236
]
4337
},
4438
{
4539
"key": "field.currency",
4640
"untestedAttrs": [
47-
"autoSet",
48-
"column",
4941
"dbColumnType",
5042
"default",
5143
"example",
5244
"instruction",
53-
"maxLength",
54-
"objectRef",
55-
"precision",
5645
"readOnly",
57-
"scale",
5846
"sortable",
5947
"sortableDefaultOrder",
60-
"storage",
6148
"unique",
6249
"xmlText"
6350
]
@@ -71,151 +58,108 @@
7158
"default",
7259
"example",
7360
"instruction",
74-
"maxLength",
75-
"objectRef",
76-
"precision",
7761
"readOnly",
78-
"scale",
7962
"sortable",
8063
"sortableDefaultOrder",
81-
"storage",
8264
"unique",
8365
"xmlText"
8466
]
8567
},
8668
{
8769
"key": "field.decimal",
8870
"untestedAttrs": [
89-
"autoSet",
9071
"column",
9172
"dbColumnType",
9273
"default",
9374
"example",
9475
"instruction",
95-
"maxLength",
96-
"objectRef",
9776
"readOnly",
9877
"sortable",
9978
"sortableDefaultOrder",
100-
"storage",
10179
"unique",
10280
"xmlText"
10381
]
10482
},
10583
{
10684
"key": "field.double",
10785
"untestedAttrs": [
108-
"autoSet",
10986
"column",
11087
"dbColumnType",
11188
"default",
11289
"example",
11390
"instruction",
114-
"maxLength",
115-
"objectRef",
116-
"precision",
11791
"readOnly",
118-
"scale",
11992
"sortable",
12093
"sortableDefaultOrder",
121-
"storage",
12294
"unique",
12395
"xmlText"
12496
]
12597
},
12698
{
12799
"key": "field.enum",
128100
"untestedAttrs": [
129-
"autoSet",
130101
"column",
131102
"dbColumnType",
132103
"example",
133104
"instruction",
134-
"maxLength",
135-
"objectRef",
136-
"precision",
137105
"readOnly",
138-
"scale",
139106
"sortable",
140107
"sortableDefaultOrder",
141-
"storage",
142108
"unique",
143109
"xmlText"
144110
]
145111
},
146112
{
147113
"key": "field.float",
148114
"untestedAttrs": [
149-
"autoSet",
150115
"column",
151116
"db.indexed",
152117
"dbColumnType",
153118
"default",
154119
"example",
155120
"instruction",
156-
"maxLength",
157-
"objectRef",
158-
"precision",
159121
"readOnly",
160-
"scale",
161122
"sortable",
162123
"sortableDefaultOrder",
163-
"storage",
164124
"unique",
165125
"xmlText"
166126
]
167127
},
168128
{
169129
"key": "field.int",
170130
"untestedAttrs": [
171-
"autoSet",
172131
"column",
173132
"dbColumnType",
174133
"example",
175134
"instruction",
176-
"maxLength",
177-
"objectRef",
178-
"precision",
179-
"scale",
180135
"sortableDefaultOrder",
181-
"storage",
182136
"unique",
183137
"xmlText"
184138
]
185139
},
186140
{
187141
"key": "field.long",
188142
"untestedAttrs": [
189-
"autoSet",
190143
"column",
191144
"db.indexed",
192145
"dbColumnType",
193146
"example",
194147
"instruction",
195-
"maxLength",
196-
"objectRef",
197-
"precision",
198-
"scale",
199148
"sortableDefaultOrder",
200-
"storage",
201149
"unique",
202150
"xmlText"
203151
]
204152
},
205153
{
206154
"key": "field.object",
207155
"untestedAttrs": [
208-
"autoSet",
209156
"column",
210157
"dbColumnType",
211158
"default",
212159
"example",
213160
"instruction",
214-
"maxLength",
215-
"precision",
216161
"readOnly",
217162
"required",
218-
"scale",
219163
"sortable",
220164
"sortableDefaultOrder",
221165
"unique",
@@ -225,11 +169,7 @@
225169
{
226170
"key": "field.string",
227171
"untestedAttrs": [
228-
"objectRef",
229-
"precision",
230-
"scale",
231172
"sortableDefaultOrder",
232-
"storage",
233173
"xmlText"
234174
]
235175
},
@@ -243,14 +183,9 @@
243183
"default",
244184
"example",
245185
"instruction",
246-
"maxLength",
247-
"objectRef",
248-
"precision",
249186
"readOnly",
250-
"scale",
251187
"sortable",
252188
"sortableDefaultOrder",
253-
"storage",
254189
"unique",
255190
"xmlText"
256191
]
@@ -263,48 +198,41 @@
263198
"default",
264199
"example",
265200
"instruction",
266-
"maxLength",
267-
"objectRef",
268-
"precision",
269-
"scale",
270201
"sortableDefaultOrder",
271-
"storage",
272202
"unique",
273203
"xmlText"
274204
]
275205
},
276206
{
277207
"key": "field.uuid",
278208
"untestedAttrs": [
279-
"autoSet",
280209
"db.indexed",
281210
"dbColumnType",
282211
"default",
283212
"example",
284213
"instruction",
285-
"maxLength",
286-
"objectRef",
287-
"precision",
288214
"readOnly",
289-
"scale",
290215
"sortable",
291216
"sortableDefaultOrder",
292-
"storage",
293217
"xmlText"
294218
]
295219
},
296220
{
297-
"key": "object.projection",
221+
"key": "identity.reference",
222+
"untestedAttrs": [
223+
"constraintName"
224+
]
225+
},
226+
{
227+
"key": "identity.secondary",
298228
"untestedAttrs": [
299-
"discriminator",
300-
"discriminatorValue"
229+
"orders",
230+
"where"
301231
]
302232
},
303233
{
304234
"key": "object.value",
305235
"untestedAttrs": [
306-
"discriminator",
307-
"discriminatorValue",
308236
"normalize"
309237
]
310238
},

‎fixtures/registry-conformance/expected-registry.json‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2014,6 +2014,13 @@
20142014
"subType": "reference",
20152015
"description": "A foreign-key reference to another entity (@references target; @enforce toggles a physical FK).",
20162016
"attrs": [
2017+
{
2018+
"name": "constraintName",
2019+
"valueType": "string",
2020+
"isArray": false,
2021+
"required": false,
2022+
"description": "Physical foreign-key constraint name override. Absent → the backend's auto-derived default (e.g. `<table>_<firstFkColumn>_fk`). Lets a model adopt an existing database whose FK constraints follow a different naming convention without a destructive rename. RDB-physical — contributed by the db provider."
2023+
},
20172024
{
20182025
"name": "enforce",
20192026
"valueType": "boolean",
@@ -2050,12 +2057,26 @@
20502057
"required": true,
20512058
"description": "The field name(s) composing this identity. Single-element for a simple PK/index, multiple for a composite."
20522059
},
2060+
{
2061+
"name": "orders",
2062+
"valueType": "string",
2063+
"isArray": true,
2064+
"required": false,
2065+
"description": "Physical index-key sort direction, positional to @fields ('asc' | 'desc'). Omit for all-ascending (the default); a shorter array leaves trailing keys ascending. Drives DESC-ordered index keys (e.g. a recency index on a timestamp). RDB-physical — contributed by the db provider, not core identity."
2066+
},
20532067
{
20542068
"name": "unique",
20552069
"valueType": "boolean",
20562070
"isArray": false,
20572071
"required": false,
20582072
"description": "When true (default), the secondary identity is a UNIQUE index; false makes it a plain (non-unique) index."
2073+
},
2074+
{
2075+
"name": "where",
2076+
"valueType": "string",
2077+
"isArray": false,
2078+
"required": false,
2079+
"description": "Partial-index predicate (raw SQL, e.g. \"delivered_at IS NULL\"). When set, the index covers only rows matching the predicate — smaller and cheaper for queries that always filter on it. Absent = a full index over every row. RDB-physical — contributed by the db provider."
20592080
}
20602081
],
20612082
"children": []

‎server/csharp/MetaObjects/Persistence/Db/DbConstants.cs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,19 @@ public static class DbConstants
2020
/// <summary>When true, suppress the @filterable-without-index Loader warning (Project D drift check).</summary>
2121
public const string FIELD_ATTR_DB_INDEXED = "db.indexed";
2222

23+
// -----------------------------------------------------------------------
24+
// Physical RDB index/constraint attrs the db provider adds to identity.*
25+
// (TypeRegistry.Extend on identity subtypes) — pure physical-storage
26+
// concerns, NOT core identity. Mirror the TS db.json identity extends.
27+
// -----------------------------------------------------------------------
28+
29+
/// <summary>identity.secondary: per-field index-key sort direction array ('asc' | 'desc'), positional to @fields.</summary>
30+
public const string IDENTITY_ATTR_ORDERS = "orders";
31+
/// <summary>identity.secondary: partial-index predicate (raw SQL). When set, the index covers only matching rows.</summary>
32+
public const string IDENTITY_ATTR_WHERE = "where";
33+
/// <summary>identity.reference: physical FK constraint-name override. Absent =&gt; the auto-derived default.</summary>
34+
public const string IDENTITY_ATTR_CONSTRAINT_NAME = "constraintName";
35+
2336
// -----------------------------------------------------------------------
2437
// R6 Plan 2b — @dbColumnType physical column-type attribute (ADR-0013).
2538
//

‎server/csharp/MetaObjects/Persistence/Db/DbProvider.cs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
// as Java and TS, keeping all domain field-attrs in domain providers.
99

1010
using MetaObjects.Core.Field;
11+
using MetaObjects.Core.Identity;
1112

1213
namespace MetaObjects.Persistence.Db;
1314

@@ -40,5 +41,17 @@ public void RegisterTypes(TypeRegistry registry)
4041
subType,
4142
attributes: [DbSchema.ColumnSchema, DbSchema.DbIndexedSchema, DbSchema.DbColumnTypeSchema]);
4243
}
44+
45+
// Physical RDB index/constraint attrs on identity subtypes — DB-domain
46+
// concerns (index ordering / partial predicate / FK constraint naming),
47+
// NOT core identity. Mirror the TS db.json identity extends.
48+
registry.Extend(
49+
MetaObjects.Shared.BaseTypes.TYPE_IDENTITY,
50+
IdentityConstants.IDENTITY_SUBTYPE_SECONDARY,
51+
attributes: [DbSchema.OrdersSchema, DbSchema.WhereSchema]);
52+
registry.Extend(
53+
MetaObjects.Shared.BaseTypes.TYPE_IDENTITY,
54+
IdentityConstants.IDENTITY_SUBTYPE_REFERENCE,
55+
attributes: [DbSchema.ConstraintNameSchema]);
4356
}
4457
}

0 commit comments

Comments
 (0)