Merge worktree-fr5e-wrapup-2026-05-27 — FR5e wrap-up (envelope schema lock + per-port shape tests + design closure)

Author: dmealing

docs/superpowers/specs/2026-05-25-fr5e-database-source-errors.md

@@ -1,15 +1,33 @@
-# FR5e — Database-source loader errors (sketch, future)
+# FR5e — Database-source loader errors
 
-**Status:** Forward-looking — gated on FR-003 (OMDB persistence engine)
-**Date:** 2026-05-25
-**Scope:** Java first (FR-003 is the Java OMDB persistence engine), then any port that
-gains database-sourced metadata loading.
-**Depends on:** [ADR-0009](../../../spec/decisions/ADR-0009-loader-error-envelope-and-source-on-node.md) (the `format: "database"` slot is already reserved), FR-003 (database-loader implementation).
+**Status:** Schema reserved + cross-port envelope shape locked (2026-05-27). Awaiting a real database-source loader.
+**Date:** 2026-05-25 (sketched), 2026-05-27 (design questions resolved + per-port shape tests added)
+**Scope:** Java first (FR-003 ships the OMDB persistence engine), then any port that gains database-sourced metadata loading.
+**Depends on:** [ADR-0009](../../../spec/decisions/ADR-0009-loader-error-envelope-and-source-on-node.md) (the `format: "database"` slot was always reserved).
+
+## What's shipped
+
+The envelope schema is fully locked across all four ports:
+
+| Port    | Source type                | Location |
+|---------|----------------------------|----------|
+| TS      | `format: "database"; dbLocation: { table, id }; jsonPath?` | `server/typescript/packages/metadata/src/source.ts` |
+| Java    | `record DatabaseSource(DbLocation dbLocation, String jsonPath)` + `record DbLocation(String table, String id)` | `server/java/metadata/src/main/java/com/metaobjects/source/` |
+| C#      | `sealed record DatabaseSource(DbLocation DbLocation, string? JsonPath)` + `sealed record DbLocation(string Table, string Id)` | `server/csharp/MetaObjects/Source/ErrorSource.cs` |
+| Python  | `@dataclass(frozen=True) DatabaseSource(db_location, json_path)` + `@dataclass(frozen=True) DbLocation(table, id)` | `server/python/src/metaobjects/source/error_source.py` |
+
+Per-port shape tests pin the envelope:
+
+- TS: `server/typescript/packages/metadata/test/source.test.ts` — `describe("ErrorSource — database variant (FR5e schema lock)")`
+- Java: `server/java/metadata/src/test/java/com/metaobjects/source/DatabaseSourceShapeTest.java`
+- C#: `server/csharp/MetaObjects.Conformance.Tests/Fr5eDatabaseSourceShapeTests.cs`
+- Python: `server/python/tests/source/test_fr5e_database_source_shape.py`
+
+A future database-source loader (FR-003 deliberately did NOT ship one — OMDB persists *user data*, not *metamodel*) has a guaranteed-correct envelope target.
 
 ## Goal
 
-When metadata is loaded from a database (instead of files), error envelopes report
-the database location of the offending record:
+When metadata is loaded from a database (instead of files), error envelopes report the database location of the offending record:
 
 ```jsonc
 {
@@ -26,33 +44,48 @@ the database location of the offending record:
 }
 ```
 
-## Why this exists now as a sketch
+## Design lock (resolved 2026-05-27)
 
-ADR-0009 reserves `format: "database"` in the envelope so the schema is forward-stable.
-FR5e is the placeholder for the implementation FR that lands alongside FR-003. No work
-happens until FR-003 ships.
+The three open questions from the original sketch are settled:
 
-## Open design questions (deferred)
+1. **`dbLocation` shape: `{ table: string, id: string }` — locked.**
+   Composite primary keys are encoded into `id` as a single delimited string
+   (e.g. `"fld_abc123/currency"` — row id + attr name joined with `/`). Reasons:
+   - Keeps the envelope dialect-neutral. Composite-key schemas vary by RDBMS;
+     a richer `dbLocation` shape would bake one schema into the envelope.
+   - Matches the pattern that JSON pointers / JSONPath already use for
+     nested addressing.
+   - Future loaders can pick the delimiter convention; the envelope just
+     records the string.
 
-All of these wait for FR-003 to settle:
+2. **Relationship between `dbLocation` and `jsonPath`: `jsonPath` is OPTIONAL.**
+   When metadata is database-sourced, the JSONPath is a derived projection
+   over the row's payload. `dbLocation` is the primary locator; `jsonPath`
+   adds precision when the offending node is nested within a row's
+   payload (e.g. inside a JSONB blob of attributes).
 
-1. **`dbLocation` shape** — is `{ table, id }` enough, or do we need composite keys?
-   FR-003's schema dictates this.
-2. **Relationship between `dbLocation` and `jsonPath`** — when metadata is database-
-   sourced, the JSONPath is a derived projection. Is `jsonPath` optional or required?
-3. **Multi-source metadata** — what if a single load aggregates from files + database?
-   The envelope discriminator allows it; do consumers need a `format: "mixed"` variant?
+3. **Mixed-source metadata: NOT NEEDED in the envelope.**
+   Each error carries the source format of the offending node, not the
+   overall load. A mixed file+database load just produces errors with the
+   appropriate per-error `format`. No `format: "mixed"` variant required.
 
-## What FR5e WON'T do
+## Out of scope
 
-- Define the OMDB schema or persistence semantics — that's FR-003.
-- Define how database-sourced metadata is overlaid with file-sourced metadata — FR-003
-  again.
+- The OMDB metadata-table schema itself (which tables hold which metadata
+  rows) — that's a separate FR when a real database-source loader is built.
+- The relationship between database-sourced and file-sourced metadata
+  overlay/merge — same.
+- A reference database-source loader implementation — also separate.
 
-## Open questions
+## Implementing the loader (future, separate FR)
 
-To be settled during brainstorm when FR-003 nears delivery:
+When a database-source loader is built, the implementer must:
 
-1. Database location shape (table + id vs richer composite).
-2. Mixed-source error attribution.
-3. Whether to ship FR5e as part of FR-003 or as a follow-on.
+1. Emit `DatabaseSource` envelopes from every error / warning path.
+2. Use the existing parser/super-resolution/validation pipeline (same as the
+   file-source loaders).
+3. Add a conformance fixture under `fixtures/conformance/` that exercises a
+   `format: "database"` error end-to-end (will require a synthetic test loader
+   or an in-memory schema for the metaobjects tables).
+4. Decide overlay semantics with file-source metadata (will require a
+   separate ADR — this is a real cross-language decision).

server/csharp/MetaObjects.Conformance.Tests/Fr5eDatabaseSourceShapeTests.cs

@@ -0,0 +1,51 @@
+// FR5e — cross-port envelope shape lock for `format: "database"`.
+//
+// The schema is reserved by ADR-0009 and instantiable in every port today;
+// a real database-source loader is future work. This test pins the shape so
+// a future loader (FR5e implementation) has a guaranteed-correct target.
+
+using MetaObjects.Source;
+using Xunit;
+
+namespace MetaObjects.Conformance.Tests;
+
+public class Fr5eDatabaseSourceShapeTests
+{
+    [Fact]
+    public void FormatIsDatabase()
+    {
+        var src = new DatabaseSource(new DbLocation("metaobjects_field_attr", "fld_abc123/currency"));
+        Assert.Equal("database", src.Format);
+    }
+
+    [Fact]
+    public void DbLocationCarriesTableAndId()
+    {
+        var loc = new DbLocation("metaobjects_object", "obj_42");
+        Assert.Equal("metaobjects_object", loc.Table);
+        Assert.Equal("obj_42", loc.Id);
+    }
+
+    [Fact]
+    public void JsonPathIsOptional()
+    {
+        var noPath = new DatabaseSource(new DbLocation("t", "id"));
+        Assert.Null(noPath.JsonPath);
+
+        var withPath = new DatabaseSource(
+            new DbLocation("metaobjects_field_attr", "fld_abc123/currency"),
+            "$.metadata.root.children[0].field.currency.@currency");
+        Assert.NotNull(withPath.JsonPath);
+        Assert.Equal("$.metadata.root.children[0].field.currency.@currency", withPath.JsonPath);
+    }
+
+    [Fact]
+    public void CompositeKeyEncodedInIdString()
+    {
+        // FR5e design lock: composite primary keys are encoded as a single
+        // delimited string in the id, e.g. "row_id/attr_name". This keeps the
+        // envelope schema dialect-neutral and avoids a deeper DbLocation shape.
+        var loc = new DbLocation("metaobjects_field_attr", "fld_abc123/currency");
+        Assert.Equal("fld_abc123/currency", loc.Id);
+    }
+}

server/java/metadata/src/test/java/com/metaobjects/source/DatabaseSourceShapeTest.java

@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2026 Doug Mealing LLC. All Rights Reserved.
+ *
+ * FR5e — cross-port envelope shape lock for `format: "database"`. The schema
+ * is reserved by ADR-0009 and instantiable in every port today; a real
+ * database-source loader is future work. This test pins the shape so a
+ * future loader (FR5e implementation) has a guaranteed-correct target.
+ */
+package com.metaobjects.source;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertThrows;
+
+public class DatabaseSourceShapeTest {
+
+    @Test
+    public void formatIsDatabase() {
+        DatabaseSource src = new DatabaseSource(new DbLocation("metaobjects_field_attr", "fld_abc123/currency"));
+        assertEquals("database", src.format());
+    }
+
+    @Test
+    public void dbLocationCarriesTableAndId() {
+        DbLocation loc = new DbLocation("metaobjects_object", "obj_42");
+        assertEquals("metaobjects_object", loc.table());
+        assertEquals("obj_42", loc.id());
+    }
+
+    @Test
+    public void jsonPathIsOptional() {
+        DatabaseSource noPath = new DatabaseSource(new DbLocation("t", "id"));
+        assertNull(noPath.jsonPath());
+
+        DatabaseSource withPath = new DatabaseSource(
+            new DbLocation("metaobjects_field_attr", "fld_abc123/currency"),
+            "$.metadata.root.children[0].field.currency.@currency");
+        assertNotNull(withPath.jsonPath());
+        assertEquals("$.metadata.root.children[0].field.currency.@currency", withPath.jsonPath());
+    }
+
+    @Test
+    public void compositeKeyEncodedInIdString() {
+        // FR5e design lock: composite primary keys are encoded as a single
+        // delimited string in the id, e.g. "row_id/attr_name". This keeps the
+        // envelope schema dialect-neutral and avoids a deeper DbLocation shape.
+        DbLocation loc = new DbLocation("metaobjects_field_attr", "fld_abc123/currency");
+        assertEquals("fld_abc123/currency", loc.id());
+    }
+
+    @Test
+    public void rejectsNullDbLocation() {
+        assertThrows(NullPointerException.class,
+            () -> new DatabaseSource(null, null));
+    }
+}

server/python/tests/source/test_fr5e_database_source_shape.py

@@ -0,0 +1,49 @@
+"""FR5e — cross-port envelope shape lock for ``format: "database"``.
+
+The schema is reserved by ADR-0009 and instantiable in every port today;
+a real database-source loader is future work. This test pins the shape so
+a future loader (FR5e implementation) has a guaranteed-correct target.
+"""
+from __future__ import annotations
+
+import pytest
+
+from metaobjects.source import DatabaseSource, DbLocation
+
+
+def test_format_is_database():
+    src = DatabaseSource(db_location=DbLocation(table="metaobjects_field_attr",
+                                                id="fld_abc123/currency"))
+    assert src.format == "database"
+
+
+def test_db_location_carries_table_and_id():
+    loc = DbLocation(table="metaobjects_object", id="obj_42")
+    assert loc.table == "metaobjects_object"
+    assert loc.id == "obj_42"
+
+
+def test_json_path_is_optional():
+    no_path = DatabaseSource(db_location=DbLocation(table="t", id="id"))
+    assert no_path.json_path is None
+
+    with_path = DatabaseSource(
+        db_location=DbLocation(table="metaobjects_field_attr", id="fld_abc123/currency"),
+        json_path="$.metadata.root.children[0].field.currency.@currency",
+    )
+    assert with_path.json_path == "$.metadata.root.children[0].field.currency.@currency"
+
+
+def test_composite_key_encoded_in_id_string():
+    """FR5e design lock: composite primary keys are encoded as a single
+    delimited string in the id, e.g. ``"row_id/attr_name"``. This keeps the
+    envelope schema dialect-neutral and avoids a deeper DbLocation shape.
+    """
+    loc = DbLocation(table="metaobjects_field_attr", id="fld_abc123/currency")
+    assert loc.id == "fld_abc123/currency"
+
+
+def test_frozen_dataclass_is_immutable():
+    src = DatabaseSource(db_location=DbLocation(table="t", id="id"))
+    with pytest.raises(Exception):  # FrozenInstanceError
+        src.json_path = "/nope"  # type: ignore[misc]

server/typescript/packages/metadata/test/source.test.ts

@@ -54,3 +54,55 @@ describe("LoaderWarning envelope shape", () => {
     expect(w.code.startsWith("WARN_")).toBe(true);
   });
 });
+
+// FR5e — cross-port envelope shape lock for `format: "database"`. The schema
+// is reserved by ADR-0009 and instantiable in every port today; a real
+// database-source loader is future work. These tests pin the shape so a
+// future loader has a guaranteed-correct target.
+describe("ErrorSource — database variant (FR5e schema lock)", () => {
+  test("format is 'database', dbLocation carries table + id", () => {
+    const s: ErrorSource = {
+      format: "database",
+      dbLocation: { table: "metaobjects_field_attr", id: "fld_abc123/currency" },
+    };
+    expect(s.format).toBe("database");
+    if (s.format === "database") {
+      expect(s.dbLocation.table).toBe("metaobjects_field_attr");
+      expect(s.dbLocation.id).toBe("fld_abc123/currency");
+    }
+  });
+
+  test("jsonPath is optional", () => {
+    const noPath: ErrorSource = {
+      format: "database",
+      dbLocation: { table: "t", id: "id" },
+    };
+    if (noPath.format === "database") {
+      expect(noPath.jsonPath).toBeUndefined();
+    }
+
+    const withPath: ErrorSource = {
+      format: "database",
+      dbLocation: { table: "metaobjects_field_attr", id: "fld_abc123/currency" },
+      jsonPath: "$.metadata.root.children[0].field.currency.@currency",
+    };
+    if (withPath.format === "database") {
+      expect(withPath.jsonPath).toBe(
+        "$.metadata.root.children[0].field.currency.@currency",
+      );
+    }
+  });
+
+  test("composite key is encoded as a delimited string in id", () => {
+    // FR5e design lock: composite primary keys go through `id` as
+    // "row_id/attr_name". Keeps the envelope dialect-neutral and avoids a
+    // deeper DbLocation shape.
+    const s: ErrorSource = {
+      format: "database",
+      dbLocation: { table: "metaobjects_field_attr", id: "fld_abc123/currency" },
+    };
+    if (s.format === "database") {
+      expect(s.dbLocation.id).toBe("fld_abc123/currency");
+    }
+  });
+});