|
| 1 | +# R6 — RDB-Fidelity Field Types (float contract · `field.uuid` · `@dbColumnType`) |
| 2 | + |
| 3 | +_Date: 2026-05-29. Status: **Approved — ready for implementation planning.**_ |
| 4 | + |
| 5 | +> This spec **re-layers** the deferred |
| 6 | +> [`2026-05-29-rdb-fidelity-field-type-additions-design.md`](2026-05-29-rdb-fidelity-field-type-additions-design.md) |
| 7 | +> against [ADR-0013](../../../spec/decisions/ADR-0013-logical-field-types-vs-physical-column-attributes.md). |
| 8 | +> The deferred spec stays as the historical reference ADR-0013 critiques; **this** is the |
| 9 | +> spec to implement. It is item **R6** in the conformance-hardening backlog |
| 10 | +> ([`2026-05-29-conformance-hardening-review.md`](2026-05-29-conformance-hardening-review.md)). |
| 11 | +
|
| 12 | +## Why this exists |
| 13 | + |
| 14 | +A real downstream consumer's relational schema needs floating-point columns, `uuid` |
| 15 | +columns, timezone-aware timestamps, and opaque (genuinely-open) jsonb. Investigating that |
| 16 | +need surfaced a deeper conformance gap (review finding #11): `field.float`/`double` map to |
| 17 | +`DOUBLE PRECISION` in every port, but `fixtures/persistence-conformance/normalization.md` |
| 18 | +serializes `REAL`/`DOUBLE` as a **raw JSON number** with no canonicalization rule — driver- |
| 19 | +native double formatting differs across Node/Bun/.NET/Python/JVM, so the contract is |
| 20 | +cross-port-divergent and **zero persistence rows return a float**. The `uuid` lowercase- |
| 21 | +canonical rule (`normalization.md:39`) is likewise dead contract — no fixture uses a uuid. |
| 22 | + |
| 23 | +R6 closes the float contract, fixes a standing float fidelity loss before fixtures lock it |
| 24 | +in, adds the one genuinely-new logical type (`field.uuid`), promotes Kotlin's physical |
| 25 | +`@dbColumnType` escape hatch to a cross-port attribute, and locks all of it with shared |
| 26 | +fixtures. |
| 27 | + |
| 28 | +## Governing constraints |
| 29 | + |
| 30 | +- **ADR-0013 (logical vs physical):** a logical field subtype fixes a value's *semantic |
| 31 | + type + idiomatic native binding*; physical column-storage concerns live on a `dbProvider`- |
| 32 | + registered attribute (`@dbColumnType`), never on the logical field. `field.uuid` is |
| 33 | + logical; timezone-awareness and opaque-jsonb are physical (`@dbColumnType`). |
| 34 | +- **ADR-0001 (cross-language type binding):** binding resolves at codegen time via static |
| 35 | + generated mappings, never runtime reflection. |
| 36 | +- **ADR-0002 (open-closed typed nodes):** a new field subtype is one class + one |
| 37 | + registration line per port. C# is still central-dispatch (higher-touch — a known risk, |
| 38 | + not redesigned here). |
| 39 | +- **ADR-0014 (loader-scoped registry):** unaffected — no change to registry resolution. |
| 40 | +- **Constants discipline:** add subtype/attr names to TS constants first |
| 41 | + (`server/typescript/packages/metadata/src/core/field/field-constants.ts`), then the |
| 42 | + Java/Python/C#/Kotlin parallels. Never inline metamodel strings. |
| 43 | +- **Conformance-first / TDD:** every addition lands as shared fixtures; tests precede |
| 44 | + implementation. |
| 45 | + |
| 46 | +## Layering map |
| 47 | + |
| 48 | +| Concern | Layer | Conformance gate | |
| 49 | +|---|---|---| |
| 50 | +| Float/double wire-normalization rule | wire contract | persistence-conformance | |
| 51 | +| `field.float` → `REAL` fidelity fix | logical→physical DDL mapping | persistence-conformance | |
| 52 | +| `field.uuid` subtype | **logical** (core metamodel) | metamodel (canonical) **+** persistence | |
| 53 | +| `@dbColumnType` (`uuid`/`jsonb`/`timestamp_with_tz`) | **physical** (dbProvider attr) | metamodel (attr round-trip) **+** persistence (effect) | |
| 54 | + |
| 55 | +--- |
| 56 | + |
| 57 | +## Section 1 — Float/double wire-normalization contract |
| 58 | + |
| 59 | +### Decision |
| 60 | + |
| 61 | +`REAL` and `DOUBLE PRECISION` result values serialize as a **plain-decimal string** with |
| 62 | +trailing zeros (and a bare trailing decimal point) stripped — **identical** to the existing |
| 63 | +`NUMERIC`/`DECIMAL` rule. They are no longer raw JSON numbers. |
| 64 | + |
| 65 | +### Research basis (empirical, 5 runtimes: Bun/JSC, Node/V8, Python 3.10, Java 21, .NET 8) |
| 66 | + |
| 67 | +Each runtime's shortest float formatter was measured against candidate values at single and |
| 68 | +double precision, then run through a strip-trailing-zeros pass: |
| 69 | + |
| 70 | +1. **Stringify is mandatory.** As *JSON numbers*, integer-valued floats diverge: Python and |
| 71 | + Java emit `100.0`; JS and .NET emit `100`. Stringify + strip-zeros canonicalizes to `100` |
| 72 | + uniformly, and reuses the strip-zeros code path every runner already has for `NUMERIC`. |
| 73 | +2. **`DOUBLE PRECISION` is safe unconstrained (within the band below).** Shortest-round-trip |
| 74 | + + strip is byte-identical across all five runtimes for every value tested, including |
| 75 | + non-dyadic `0.1`, `3.14`, π (`3.141592653589793`), and `1/3` — because all five have a |
| 76 | + native float64 and a correct shortest formatter (Java's `Double.toString` is shortest as |
| 77 | + of JDK 19; the toolchain is Java 21). |
| 78 | +3. **`REAL` has two divergences that the fixture-authoring constraint removes:** |
| 79 | + - *No native float32 in JS/Python.* For a non-dyadic value they render the widened-double |
| 80 | + tail (`0.1` → `0.10000000149011612`) while Java/.NET render shortest-single (`0.1`). |
| 81 | + - *Exponential notation.* Java is the most aggressive, switching to `E`-notation below |
| 82 | + `1e-3` and at/above `1e7` (dyadic `0.0009765625` → Java `9.765625E-4`, plain decimal |
| 83 | + elsewhere). |
| 84 | + |
| 85 | +### The contract (added to `normalization.md`) |
| 86 | + |
| 87 | +- **Serialization:** `REAL`/`DOUBLE` → decimal **string**, **plain (never exponential) |
| 88 | + notation**, trailing zeros and bare decimal point stripped (e.g. `1.5`, `100`, `-3.25`). |
| 89 | +- **Fixture-authoring constraint:** `REAL`/`DOUBLE` test values MUST be **exact dyadic |
| 90 | + rationals** (terminating binary fraction) within the plain-decimal band |
| 91 | + `|x| ∈ {0} ∪ [0.001, 1 000 000)`; `REAL` values additionally ≤ 6 significant decimal |
| 92 | + digits (single-precision exactness). With this constraint the widened double **is** the |
| 93 | + short decimal, so all five ports agree under plain shortest formatting. |
| 94 | + - **Safe:** `1.5`, `0.125`, `1234.5`, `-3.25`, `100`, `0.5`, `0.0625`. |
| 95 | + - **Forbidden (with the reason):** `0.1`/`3.14`/π (non-dyadic → JS/Python tail on `REAL`); |
| 96 | + `0.0009765625` (dyadic but `< 1e-3` → Java `E`-notation); `12345678` (`≥ 1e7`). |
| 97 | +- **Runner guard (recommended):** each port formats from the value *as returned by the |
| 98 | + driver for the column's SQL type* (a `REAL` formats from the native single where the |
| 99 | + language has one), forces plain decimal, then strips zeros — so an accidentally out-of-band |
| 100 | + value **fails loudly** on diff rather than silently diverging. |
| 101 | + |
| 102 | +This activates the dead `UUID` and `TIMESTAMPTZ` rows as a side effect (Sections 3–4). |
| 103 | + |
| 104 | +--- |
| 105 | + |
| 106 | +## Section 2 — `field.float` → `REAL` (fidelity fix) |
| 107 | + |
| 108 | +### Decision |
| 109 | + |
| 110 | +`field.float` emits **`REAL`** (single precision); `field.double` emits **`DOUBLE |
| 111 | +PRECISION`**; `field.decimal` stays **`NUMERIC`**. Today all ports collapse float and double |
| 112 | +onto a single internal "real" kind that emits `DOUBLE PRECISION` (TS `emit/postgres.ts:149`; |
| 113 | +C# `ExpectedSchema.cs:143`; Python `postgres_emit.py:127`) — a silent fidelity loss this |
| 114 | +fixes before fixtures cement it. |
| 115 | + |
| 116 | +### Per-port work |
| 117 | + |
| 118 | +- **TS:** introduce a distinct single-precision `SqlType` kind in `migrate-ts/src/sql-type.ts`; |
| 119 | + map `field.double` → double-precision kind and `field.float` → the new single kind in |
| 120 | + `expected-schema.ts`; emit `REAL` vs `DOUBLE PRECISION` in `emit/postgres.ts`; **round-trip |
| 121 | + the distinction in `introspect/postgres.ts`** so `meta migrate` sees no phantom |
| 122 | + `REAL`↔`DOUBLE PRECISION` diff. SQLite: both remain `REAL` (SQLite has one float type) — |
| 123 | + document that the float/double distinction is Postgres-only. |
| 124 | +- **C#:** add a single-precision `SqlType` variant (peer of `SqlType.Real`/`Numeric`) in |
| 125 | + `MetaObjects.Codegen/Migrate`; route `field.float` to it in `ExpectedSchema.cs`; emit + |
| 126 | + introspect in `PostgresSchema.cs`. |
| 127 | +- **Python:** add the single-precision SQL-type in `migrate/sql_type.py`; route in |
| 128 | + `migrate/expected_schema.py`; emit in `migrate/postgres_emit.py`; introspection round-trip. |
| 129 | +- **Java (`codegen-spring`) + Kotlin (Exposed):** confirm/adjust the type mappers so |
| 130 | + `field.float` → `REAL`/Exposed `float()` and `field.double` → `DOUBLE PRECISION`/Exposed |
| 131 | + `double()`. |
| 132 | + |
| 133 | +### Native binding |
| 134 | + |
| 135 | +Java/C#/Kotlin bind a 4-byte float for `field.float`. TS (`number`) and Python (`float`) |
| 136 | +have only a 64-bit float and read the `REAL` column losslessly **for in-band dyadic values** |
| 137 | +(the Section 1 constraint guarantees this in fixtures). |
| 138 | + |
| 139 | +### Risk |
| 140 | + |
| 141 | +Existing port-local snapshot tests that asserted `DOUBLE PRECISION` for `field.float` will |
| 142 | +change; update them as part of this section. |
| 143 | + |
| 144 | +--- |
| 145 | + |
| 146 | +## Section 3 — `field.uuid` logical subtype |
| 147 | + |
| 148 | +### Decision |
| 149 | + |
| 150 | +Add `field.uuid` as a first-class logical field subtype (ADR-0002: one class + one |
| 151 | +registration line per port). It is the only genuinely-new *type* in R6. |
| 152 | + |
| 153 | +### Native + DDL binding (ADR-0001, static) |
| 154 | + |
| 155 | +| Port | Native type | Postgres | SQLite | |
| 156 | +|---|---|---|---| |
| 157 | +| TypeScript | `string` | `uuid` | `text` | |
| 158 | +| Java | `java.util.UUID` | `uuid` | `TEXT` | |
| 159 | +| C# | `System.Guid` | `uuid` | `TEXT` | |
| 160 | +| Python | `uuid.UUID` | `uuid` | `TEXT` | |
| 161 | +| Kotlin | `java.util.UUID` | `uuid` (Exposed `uuid(...)`) | `TEXT` | |
| 162 | + |
| 163 | +The DataType is **STRING-backed** (the value's canonical wire form is a string), mirroring |
| 164 | +the `enum`/`currency` precedent. Persistence normalization is the already-pinned lowercase- |
| 165 | +canonical form — this fixes C#'s `Guid` casing/brace-format never being caught (review |
| 166 | +finding #12). |
| 167 | + |
| 168 | +### Per-port touch-points |
| 169 | + |
| 170 | +- **TS (reference, constants-first):** `field-constants.ts` (`FIELD_SUBTYPE_UUID` + |
| 171 | + `FIELD_SUBTYPES`), `meta-field.ts` (`FIELD_DATA_TYPE` STRING entry), `core-types.ts` |
| 172 | + registration; codegen `codegen-ts/src/column-mapper.ts` + `templates/inferred-types.ts`; |
| 173 | + DDL `migrate-ts` emit + `introspect/postgres.ts`. |
| 174 | +- **Java:** `UuidField` class + `FieldTypesMetaDataProvider` registration; `SpringTypeMapper`. |
| 175 | +- **Python:** `field_constants.py` + `meta_field.py` + `loader/validation_passes.py`; |
| 176 | + `migrate` emit. |
| 177 | +- **C#:** `FieldConstants.cs` + (central-dispatch) `CoreTypes.cs`, `Loader/ValidationPasses.cs`, |
| 178 | + `EntityGenerator.cs`/`PostgresSchema.cs`/`ExpectedSchema.cs`. **Highest-touch port.** |
| 179 | +- **Kotlin:** `KotlinTypeMapper.kt` — map `field.uuid` (the logical subtype) → `java.util.UUID` |
| 180 | + + Exposed `uuid(...)`, distinct from the legacy `@dbColumnType:uuid`-on-`string` path |
| 181 | + (which ADR-0013 notes yields `String`). Persistence wire form is identical (lowercase). |
| 182 | + |
| 183 | +### Generation — out of scope for R6 |
| 184 | + |
| 185 | +Fixtures use **assigned** uuid values to exercise the wire form. PK generation stays on the |
| 186 | +existing `@generation:uuid` identity path (ADR-0013 "one resolver"); the `@default:"uuid"` |
| 187 | +non-PK token is deferred to a later item. R6 fixtures keep `field.long` PKs and add `uuid` |
| 188 | +as a non-PK column. |
| 189 | + |
| 190 | +--- |
| 191 | + |
| 192 | +## Section 4 — `@dbColumnType` physical attribute |
| 193 | + |
| 194 | +### Decision |
| 195 | + |
| 196 | +Promote Kotlin's existing `@dbColumnType` escape hatch (`KotlinTypeMapper.kt`) to a |
| 197 | +**`dbProvider`-registered** field attribute in TS/Java/Python/C# (Kotlin already ships it). |
| 198 | +It selects the **physical DB column type while leaving the logical field's native binding |
| 199 | +untouched** (ADR-0013 rejects native-binding-suppressing raw passthrough). |
| 200 | + |
| 201 | +### Value set + validation |
| 202 | + |
| 203 | +Closed set **`{ uuid, jsonb, timestamp_with_tz }`** (parity with Kotlin's current tokens). |
| 204 | +An unknown value → **`ERR_BAD_ATTR_VALUE`**. No raw-dialect-SQL passthrough in R6 (YAGNI; |
| 205 | +ADR-0013 sanctions a bounded form, and the bounded set covers the consumer's need). |
| 206 | + |
| 207 | +### Effect (persistence/codegen layer only) |
| 208 | + |
| 209 | +| Attr value | Applies to | Physical column | Native binding | Wire normalization | |
| 210 | +|---|---|---|---|---| |
| 211 | +| `uuid` | `field.string` | PG `uuid` | unchanged (string) | lowercase canonical | |
| 212 | +| `jsonb` | `field.string` | PG `jsonb` (opaque, raw JSON text) | unchanged (string) | JSONB sorted-keys | |
| 213 | +| `timestamp_with_tz` | `field.timestamp` | PG `timestamptz` | unchanged | `…Z` (UTC) branch | |
| 214 | + |
| 215 | +`uuid` here is the *physical* path for an existing string column; **structured** identity is |
| 216 | +the logical `field.uuid` (Section 3). `jsonb` is the deliberately-narrow opaque escape hatch |
| 217 | +(arbitrary third-party payloads); **typed** jsonb stays `field.object` + `@objectRef` + |
| 218 | +`@storage: jsonb` (the preferred, drift-checked path — unchanged). `timestamp_with_tz` |
| 219 | +drives the `TIMESTAMPTZ` normalization branch that has never been exercised at runtime |
| 220 | +(review finding #16). |
| 221 | + |
| 222 | +### Loader behavior |
| 223 | + |
| 224 | +As an `@`-attr registered by the `dbProvider`, `@dbColumnType` round-trips through the |
| 225 | +canonical serializer like `@column`/`@kind`. One loader fixture confirms the canonical |
| 226 | +round-trip; the physical effect is exercised by persistence fixtures. Registration must be |
| 227 | +active in the registry the persistence-corpus loader composes (verify during implementation |
| 228 | +— the persistence fixtures already use `dbProvider` attrs such as `@table`/`@column`). |
| 229 | + |
| 230 | +--- |
| 231 | + |
| 232 | +## Section 5 — Conformance fixtures |
| 233 | + |
| 234 | +### Loader corpus (`fixtures/conformance/`) |
| 235 | + |
| 236 | +Mirroring the `enum` precedent (happy + extends + array, plus negatives). Loader-behavior |
| 237 | +only — canonical `expected.json` or FR5a-envelope `expected-errors.json`: |
| 238 | + |
| 239 | +- `field-uuid-basic`, `field-uuid-extends` (abstract base + `extends`), `field-uuid-array` |
| 240 | +- `field-double-basic`, `field-float-basic`, `field-decimal-precision`, `field-double-array` |
| 241 | + (cheap round-trips locking the existing subtypes — currently zero loader coverage) |
| 242 | +- `attr-dbcolumntype-uuid`, `attr-dbcolumntype-jsonb`, `attr-dbcolumntype-timestamptz` |
| 243 | + (canonical attr round-trip) |
| 244 | +- `error-field-dbcolumntype-bad-value` (`ERR_BAD_ATTR_VALUE` negative) |
| 245 | + |
| 246 | +Register any new code in `fixtures/conformance/ERROR-CODES.json`; update `CAPABILITIES.json`. |
| 247 | + |
| 248 | +### Persistence corpus (`fixtures/persistence-conformance/`) |
| 249 | + |
| 250 | +A **new dedicated entity** — generically named (e.g. `Measurement`) — added to the corpus, |
| 251 | +**not** bolted onto `Program`/`meta.fitness.json` (which would churn every existing |
| 252 | +scenario's expected rows). It carries: |
| 253 | + |
| 254 | +- `id` `field.long` PK (assigned), |
| 255 | +- a `field.float` column and a `field.double` column (dyadic-band values), |
| 256 | +- a `field.uuid` column (assigned, lowercase wire form), |
| 257 | +- a `field.timestamp` + `@dbColumnType: timestamp_with_tz` column (the `…Z` branch), |
| 258 | +- a `field.string` + `@dbColumnType: jsonb` opaque column (sorted-keys branch). |
| 259 | + |
| 260 | +Plus a bootstrap migration and `get`/`list` query scenarios asserting each normalized wire |
| 261 | +form. This doubles as the missing runtime "kitchen-sink" (review finding #16). |
| 262 | + |
| 263 | +`normalization.md` updates: REAL/DOUBLE row → the Section 1 string rule with a worked example |
| 264 | +and forbidden-values note; the existing UUID, TIMESTAMPTZ, and JSONB rows are now genuinely |
| 265 | +exercised. |
| 266 | + |
| 267 | +--- |
| 268 | + |
| 269 | +## Sequencing & verification |
| 270 | + |
| 271 | +The implementation plan (writing-plans, next step) will stage roughly as: |
| 272 | + |
| 273 | +1. **Float contract + `field.float`→`REAL`** (Sections 1–2) — wire rule + DDL/introspection, |
| 274 | + port by port; verify migrate sees no phantom diff. |
| 275 | +2. **`field.uuid` logical subtype** (Section 3) — TS-first, then the four ports; loader |
| 276 | + fixtures green before persistence. |
| 277 | +3. **`@dbColumnType` physical attr** (Section 4) — dbProvider registration + persistence |
| 278 | + effect; reconcile Kotlin to the canonical name. |
| 279 | +4. **Fixtures** (Section 5) — loader corpus then the persistence `Measurement` entity; run |
| 280 | + the Docker persistence + api-contract suites across all five ports. |
| 281 | + |
| 282 | +Stages 1–2 may be one plan doc and 3–4 a second if the combined plan is unwieldy. |
| 283 | + |
| 284 | +Per-stage commands (verify before claiming done): |
| 285 | + |
| 286 | +``` |
| 287 | +TS: cd server/typescript && bun test |
| 288 | +Python: cd server/python && uv run --extra dev pytest |
| 289 | +Java meta: cd server/java && mvn -q -pl metadata test |
| 290 | +Java mod: cd server/java && mvn -q -pl <module> -am test |
| 291 | +Kotlin: mvn -f server/java/integration-tests-kotlin/pom.xml test |
| 292 | +Persist./API (Docker): scripts/integration-test.sh |
| 293 | +``` |
| 294 | + |
| 295 | +After each unit: `/code-review` + `/simplify`, fix findings, then mark R6 resolved in the |
| 296 | +hardening-review backlog and the `conformance-suite-gaps` memory with the commit ref. |
| 297 | + |
| 298 | +## Out of scope (YAGNI) |
| 299 | + |
| 300 | +- `@default:"uuid"` non-PK token and any change to the `@generation:uuid` PK path. |
| 301 | +- Raw-dialect-SQL `@dbColumnType` passthrough beyond the three named values. |
| 302 | +- New numeric types beyond the existing `double`/`float`/`decimal`. |
| 303 | +- Native Postgres `enum`, int-backed enums (separate deferred work). |
| 304 | +- Migrating C# off central-dispatch (tracked separately; R6 accepts the higher C# touch). |
| 305 | + |
| 306 | +## Risks & open questions |
| 307 | + |
| 308 | +1. **C# central-dispatch (ADR-0002 not yet realized in C#)** — `field.uuid` touches more C# |
| 309 | + files; risk to effort, not correctness (the open-closed proof test guards the other ports). |
| 310 | +2. **Introspection round-trip for `REAL` vs `DOUBLE PRECISION`** — must be exact or `meta |
| 311 | + migrate` shows a phantom diff. Confirm each port's introspector distinguishes `float4` |
| 312 | + from `float8`. |
| 313 | +3. **SQLite float/uuid** — SQLite collapses float/double to one `REAL` and stores uuid as |
| 314 | + `TEXT`; document the Postgres-only distinction; confirm introspection round-trips it. |
| 315 | +4. **`@dbColumnType` registry visibility** — confirm the attribute is registered in whatever |
| 316 | + registry the persistence-corpus loader composes, so fixtures load without |
| 317 | + `ERR_UNKNOWN_ATTR`. |
| 318 | +5. **Kotlin dual uuid paths** — logical `field.uuid` (→`UUID`) and legacy |
| 319 | + `@dbColumnType:uuid`-on-`string` (→`String`) coexist; both normalize to the same lowercase |
| 320 | + wire form, but the binding differs. Documented intentionally; revisit only if it confuses. |
0 commit comments