Skip to content

Commit 7a813d3

Browse files
dmealingclaude
andcommitted
Merge plan/field-object-storage — relocate @storage into per-concern modules
The constants-colocation refactor (ADR-0003/0004) landed on main while this branch was in flight, deleting constants.ts and core-attr-schemas.ts. The merge relocates this branch's additions into the new per-concern structure: * FIELD_ATTR_STORAGE + STORAGE_FLATTENED/JSONB/SUBDOCUMENT + STORAGE_VALUES + StorageValue type → core/field/field-constants.ts (alongside the other FIELD_ATTR_* and AUTO_SET_* constants) * @storage AttrSchema entry → core/field/field-schema.ts (alongside the @objectref entry in commonFieldAttrs) * validateFieldObjectStorage validator → unchanged location at loader/validation-passes.ts; imports redirected to per-concern files The barrel re-exports in metadata/src/index.ts preserve bare-name imports, so migrate-ts code and tests need no changes — they import FIELD_ATTR_STORAGE from "@metaobjectsdev/metadata" which resolves through the new per-concern path transparently. Plus a related .gitignore broadening: csharp build outputs (bin/, obj/, *.dll, *.pdb, etc.) are now ignored under any **/csharp/** subtree, cleaning up the stale root-level csharp/ directory left by the layout migration. Verification: 2168 pass / 0 fail (TS), 168 pass / 0 fail (C# conformance, +6 for the four new @storage fixtures), 14/14 packages build + typecheck exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 parents f153493 + e4d2b82 commit 7a813d3

19 files changed

Lines changed: 1740 additions & 18 deletions

File tree

.gitignore

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,23 @@ server/java/**/target/
2929
server/typescript/**/dist/
3030
server/typescript/**/node_modules/
3131

32-
# C# build output
33-
server/csharp/**/bin/
34-
server/csharp/**/obj/
32+
# C# / .NET build output. The `bin/` and `obj/` patterns are scoped to csharp
33+
# subtrees only (TS packages use `bin/` for executable scripts like
34+
# `server/typescript/packages/cli/bin/meta.ts` — must not be ignored).
35+
# The file-extension patterns below are safe anywhere — no non-.NET project
36+
# produces these.
37+
**/csharp/**/bin/
38+
**/csharp/**/obj/
39+
*.dll
40+
*.pdb
41+
*.exe
42+
*.suo
43+
*.user
44+
.vs/
45+
[Tt]est[Rr]esults/
46+
[Bb]uild[Ll]og.*
47+
*.nupkg
48+
*.snupkg
3549

3650
# Python
3751
__pycache__/

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -394,6 +394,7 @@ Preserve the following contracts exactly across all language ports:
394394
- Layout subtypes: `dataGrid`
395395
- Currency attrs: `@currency` (ISO 4217), `@locale` (BCP 47)
396396
- Schema attrs: `@schema` on `source[dbTable]` and `source[dbView]` (DB schema name; Postgres default `public`, SQLite rejects non-default values)
397+
- Storage attrs: `@storage` on `field.object` (with `@objectRef`) — values `flattened` / `jsonb` / `subdocument`. Unifies "owned types" (flattened storage) and "structured JSONB" (jsonb storage). Defaults to single-jsonb-column when absent (back-compat).
397398

398399
**Wire format:**
399400
- Currency: integer minor units on the wire always. Float arithmetic for money is forbidden.

docs/superpowers/plans/2026-05-22-field-object-storage.md

Lines changed: 1053 additions & 0 deletions
Large diffs are not rendered by default.

docs/superpowers/specs/2026-05-20-csharp-tool-and-metamodel-extensions-design.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,7 @@ These additions are Tier 1 cross-language metamodel changes — TS, Java (future
7575

7676
| Addition | Shape | Notes |
7777
|---|---|---|
78-
| **JSONB column subtype** | `field.subType = "jsonb"` | Postgres-native. Column maps to `jsonb`. C# codegen emits the property as `string` or a typed model + value converter (configurable via `@csJsonbType`). |
79-
| **Schema namespacing** | A `package` segment maps to a DB schema (e.g., `acme::api``acme_api`). Default schema: `public`. | Necessary for projects with curated read-model schemas alongside the canonical OLTP schema. |
78+
| **Schema namespacing** | `@schema` attr on `source[dbTable]` / `source[dbView]`. Default `public` for Postgres; SQLite rejects non-default values. | **Shipped** 2026-05-20 — see plan `2026-05-20-schema-namespacing.md`. |
8079
| **Partial indexes** | `index.@filter "IsOpen = true"` | Postgres `CREATE INDEX ... WHERE ...`. Common in production schemas. |
8180
| **Computed columns** | `field.@computed "(\"ParentId\" IS NULL)"` | Postgres `GENERATED ... STORED`. Codegen emits `HasComputedColumnSql(...)` on the Fluent side. |
8281

@@ -85,7 +84,7 @@ These additions are Tier 1 cross-language metamodel changes — TS, Java (future
8584
| Addition | Shape | Notes |
8685
|---|---|---|
8786
| **TPH discriminator inheritance** | `object.@discriminator "Type"` on the base; `object.@discriminatorValue "Bridge"` on each child object that `extends` it | Maps to EF `HasDiscriminator`. Discriminator column type: int / string / enum. Conformance must cover: nullable subtypes, FK pointing at base vs. subtype, EF skip-navigation onto subtypes. |
88-
| **Owned types** | A field of subtype `ownedComplex` references a separate `object` declared with `object.@owned true`; columns from the owned object flatten into the owner's table with optional column-name remap | Maps to EF `OwnsOne` / `OwnsMany`. Cross-language ports represent the same flattening; non-.NET languages may emit a nested struct or a flattened record. |
87+
| **Structured nested values via `object.value` + `@storage` hint** | A field declared `field.object @objectRef "Address"` references an existing `object.value`. The `@storage` attr — `"flattened"` \| `"jsonb"` \| `"subdocument"` — tells codegen + introspect how the nested value lands in storage. `isArray true` supported with `"jsonb"` / `"subdocument"`. | **Unifies the original Tier-2 "JSONB subtype" and Tier-3 "Owned types".** One concept: the metadata declares the structured shape, the storage hint picks the strategy. `flattened` → multiple columns prefixed by field name (EF `OwnsOne` pattern). `jsonb` → single Postgres jsonb column holding the structured value (and an array of values when `isArray true`). `subdocument` → nested document for Mongo-shaped stores. Cross-language ports interpret the same metadata against their target dialect; the metamodel does not encode any dialect-specific behavior. |
8988
| **External-SQL function references** | `function[externalSql]` declaration: name, parameters (typed), return type, and a path to a `.sql` file holding the body | For Postgres functions whose body is too large or imperative to author in metadata. The tool tracks the file's checksum; when it changes, the next `migrate emit` produces a `CREATE OR REPLACE FUNCTION` statement. |
9089
| **External-SQL view references** | `view[externalSql]` declaration: name, optional column list, and a path to a `.sql` file holding the SELECT body | Same pattern for views that aren't projections (legacy hand-written views, complex CTEs that resist projection authoring). |
9190

@@ -113,21 +112,22 @@ Both extensions are general-purpose and benefit every TypeScript consumer that d
113112

114113
Required new fixtures under `fixtures/conformance/`:
115114

116-
- `field-subtype-jsonb-basic/`
117-
- `package-to-schema-mapping/`
118115
- `index-partial-filter/`
119116
- `field-computed-column/`
120117
- `tph-discriminator-int-base-only/`
121118
- `tph-discriminator-string-base-only/`
122119
- `tph-discriminator-with-subtypes/`
123120
- `tph-discriminator-nullable-subtypes/`
124-
- `owned-type-single/`
125-
- `owned-type-with-column-rename/`
126-
- `owned-type-nullable/`
121+
- `field-object-storage-flattened/`
122+
- `field-object-storage-flattened-nullable/`
123+
- `field-object-storage-jsonb-single/`
124+
- `field-object-storage-jsonb-array/`
127125
- `function-externalsql-basic/`
128126
- `view-externalsql-basic/`
129127
- `view-externalsql-with-dependency-tracking/`
130128

129+
(Schema namespacing fixtures already shipped: `source-db-table-with-schema/`, `source-db-view-with-schema/`, `source-db-table-default-schema-omitted/`.)
130+
131131
Each fixture pairs input metadata with the canonical serializer output. TS implementation establishes the canonical form; C# (and future Java, Python) implementations must match.
132132

133133
## Estimated effort
@@ -153,19 +153,20 @@ Each fixture pairs input metadata with the canonical serializer output. TS imple
153153
## Risks
154154

155155
1. **EF model reflection at design time has historically been fiddly** — assembly loading, design-time DI, runtime-only configuration. Mitigation: the `ModelSnapshot.cs` parsing fallback. Validate which is robust in the first 2 weeks.
156-
2. **Owned-type codegen has edge cases** the conformance fixtures might not catch on first pass (shadow FKs, nullable owned blocks, nested owned types). Mitigation: 2-3 week buffer in the estimate for first-real-project iteration.
156+
2. **`field.object + @storage` codegen has edge cases** the conformance fixtures might not catch on first pass — column-name prefixing for `flattened` (`PRPAddress_Street` vs `Address_Street`?), nullable owned blocks, nested objects inside JSONB, arrays of jsonb-stored values. Mitigation: 2-3 week buffer in the estimate for first-real-project iteration; cover all four shapes (single+flattened, single+jsonb, array+jsonb, array+subdocument) with dedicated fixtures.
157157
3. **TPH inheritance is the gnarliest codegen pattern.** Subtype-pointing FKs, EF skip-navigation, discriminator-as-enum vs. discriminator-as-int. Mitigation: cover 5+ TPH conformance fixtures before C# codegen ships.
158158
4. **`view[externalSql]` checksum tracking format must be stable across language ports.** A naive whitespace-sensitive checksum is fragile. Mitigation: define a canonical normalization (strip trailing whitespace, normalize line endings, optionally strip SQL comments) — and conformance-test it.
159159
5. **`migrate emit` SQL output divergence between TS and C# implementations.** The cross-language conformance corpus must enforce byte-equality of emitted DDL for the same metadata + dialect. Mitigation: a `fixtures/migrations/` corpus parallel to the loader corpus, validating both implementations against the same canonical script.
160160

161161
## Cross-references
162162

163163
- **Superseded:** `docs/superpowers/specs/2026-05-20-csharp-rdb-persistence-design.md`
164-
- **TS reference implementation:** `typescript/packages/migrate-ts/`
165-
- **TS codegen architecture:** `typescript/packages/codegen-ts/`
166-
- **C# loader (foundation):** `csharp/MetaObjects/`
164+
- **TS reference implementation:** `server/typescript/packages/migrate-ts/`
165+
- **TS codegen architecture:** `server/typescript/packages/codegen-ts/`
166+
- **C# loader (foundation):** `server/csharp/MetaObjects/`
167167
- **C# loader plan:** `docs/superpowers/plans/2026-05-19-csharp-conformance-port.md`
168168
- **Existing conformance corpus:** `fixtures/conformance/`
169+
- **Shipped Tier-2 — schema namespacing:** `docs/superpowers/plans/2026-05-20-schema-namespacing.md`
169170

170171
## Open decisions for the implementation plan
171172

fixtures/conformance/ERROR-CODES.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@
2222
"ERR_MALFORMED_YAML": "The YAML metadata input is not valid YAML, or cannot be desugared into canonical metadata.",
2323
"ERR_INVALID_ORIGIN": "A field origin (passthrough/aggregate) declares an invalid path or attribute.",
2424
"ERR_BAD_ATTR_FILTER": "A dataGrid @filter references a non-filterable field or uses an op disallowed for the field's subtype.",
25+
"ERR_STORAGE_FLATTENED_ARRAY": "@storage \"flattened\" cannot be combined with isArray=true.",
26+
"ERR_STORAGE_WITHOUT_OBJECT_REF": "@storage was set on a field that has no @objectRef.",
2527
"ERR_UNKNOWN": "An internal loader error with no stable error code."
2628
}
2729
}
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"object.value": {
7+
"name": "AddressOptional",
8+
"children": [
9+
{
10+
"field.string": {
11+
"name": "street"
12+
}
13+
},
14+
{
15+
"field.string": {
16+
"name": "city"
17+
}
18+
},
19+
{
20+
"field.string": {
21+
"name": "postalCode"
22+
}
23+
}
24+
]
25+
}
26+
},
27+
{
28+
"object.entity": {
29+
"name": "Person",
30+
"children": [
31+
{
32+
"source.dbTable": {
33+
"@name": "people"
34+
}
35+
},
36+
{
37+
"field.long": {
38+
"name": "id"
39+
}
40+
},
41+
{
42+
"field.object": {
43+
"name": "homeAddress",
44+
"@objectRef": "AddressOptional",
45+
"@storage": "flattened"
46+
}
47+
},
48+
{
49+
"identity.primary": {
50+
"@fields": [
51+
"id"
52+
]
53+
}
54+
}
55+
]
56+
}
57+
}
58+
]
59+
}
60+
}
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"object.value": {
7+
"name": "AddressOptional",
8+
"children": [
9+
{ "field.string": { "name": "street" } },
10+
{ "field.string": { "name": "city" } },
11+
{ "field.string": { "name": "postalCode" } }
12+
]
13+
}
14+
},
15+
{
16+
"object.entity": {
17+
"name": "Person",
18+
"children": [
19+
{ "source.dbTable": { "@name": "people" } },
20+
{ "field.long": { "name": "id" } },
21+
{ "field.object": { "name": "homeAddress", "@objectRef": "AddressOptional", "@storage": "flattened" } },
22+
{ "identity.primary": { "@fields": "id" } }
23+
]
24+
}
25+
}
26+
]
27+
}
28+
}
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"object.value": {
7+
"name": "Address",
8+
"children": [
9+
{
10+
"field.string": {
11+
"name": "street",
12+
"@required": true
13+
}
14+
},
15+
{
16+
"field.string": {
17+
"name": "city",
18+
"@required": true
19+
}
20+
},
21+
{
22+
"field.string": {
23+
"name": "postalCode"
24+
}
25+
}
26+
]
27+
}
28+
},
29+
{
30+
"object.entity": {
31+
"name": "Customer",
32+
"children": [
33+
{
34+
"source.dbTable": {
35+
"@name": "customers"
36+
}
37+
},
38+
{
39+
"field.long": {
40+
"name": "id"
41+
}
42+
},
43+
{
44+
"field.object": {
45+
"name": "shippingAddress",
46+
"@objectRef": "Address",
47+
"@storage": "flattened"
48+
}
49+
},
50+
{
51+
"identity.primary": {
52+
"@fields": [
53+
"id"
54+
]
55+
}
56+
}
57+
]
58+
}
59+
}
60+
]
61+
}
62+
}
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"object.value": {
7+
"name": "Address",
8+
"children": [
9+
{ "field.string": { "name": "street", "@required": true } },
10+
{ "field.string": { "name": "city", "@required": true } },
11+
{ "field.string": { "name": "postalCode" } }
12+
]
13+
}
14+
},
15+
{
16+
"object.entity": {
17+
"name": "Customer",
18+
"children": [
19+
{ "source.dbTable": { "@name": "customers" } },
20+
{ "field.long": { "name": "id" } },
21+
{ "field.object": { "name": "shippingAddress", "@objectRef": "Address", "@storage": "flattened" } },
22+
{ "identity.primary": { "@fields": "id" } }
23+
]
24+
}
25+
}
26+
]
27+
}
28+
}
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"object.value": {
7+
"name": "ContactInfo",
8+
"children": [
9+
{
10+
"field.string": {
11+
"name": "phone"
12+
}
13+
},
14+
{
15+
"field.string": {
16+
"name": "email"
17+
}
18+
},
19+
{
20+
"field.string": {
21+
"name": "preferredMethod"
22+
}
23+
}
24+
]
25+
}
26+
},
27+
{
28+
"object.entity": {
29+
"name": "Patient",
30+
"children": [
31+
{
32+
"source.dbTable": {
33+
"@name": "patients"
34+
}
35+
},
36+
{
37+
"field.long": {
38+
"name": "id"
39+
}
40+
},
41+
{
42+
"field.object": {
43+
"name": "contactInfos",
44+
"isArray": true,
45+
"@objectRef": "ContactInfo",
46+
"@storage": "jsonb"
47+
}
48+
},
49+
{
50+
"identity.primary": {
51+
"@fields": [
52+
"id"
53+
]
54+
}
55+
}
56+
]
57+
}
58+
}
59+
]
60+
}
61+
}

0 commit comments

Comments
 (0)