|
| 1 | +# Sophisticated metadata validation — architecture options & recommendation |
| 2 | + |
| 3 | +_2026-06-19. Triggered by the cross-reference gap: a dangling `relationship.@objectRef` |
| 4 | +or `identity.reference.@references` loaded silently instead of failing. The narrow fix is |
| 5 | +easy; the real question is **how metadata validation should be architected** so that |
| 6 | +sophisticated checks — especially cross-references — are correct, extensible, and |
| 7 | +consistent across the five ports._ |
| 8 | + |
| 9 | +## The problem, framed correctly |
| 10 | + |
| 11 | +Loading metadata is a **compiler front-end**, not a schema check: |
| 12 | + |
| 13 | +``` |
| 14 | +parse (per file) → merge/overlay → [ build symbol table → resolve refs → semantic checks ] → freeze |
| 15 | + └──────────────── "semantic analysis" ─────────────────┘ |
| 16 | +``` |
| 17 | + |
| 18 | +Two classes of validation, with very different needs: |
| 19 | + |
| 20 | +1. **Local / structural** — decidable from a single node: required attrs, enum membership, |
| 21 | + `@kind` values, value coercion, child placement. JSON-Schema / XSD-class checks. |
| 22 | +2. **Global / relational** — decidable only against the *whole loaded tree*: `@objectRef`, |
| 23 | + `identity.reference.@references`, `extends`, `template.@payloadRef`, `origin |
| 24 | + @from/@of/@via`. These are **name-resolution** problems — a node alone cannot answer |
| 25 | + them; you need an index of every object/field first. |
| 26 | + |
| 27 | +The gap we hit (`@objectRef` to a non-existent entity loading clean) is a *global* check |
| 28 | +that was simply never written. The interesting question is the architecture that makes the |
| 29 | +whole **global** class systematic instead of hand-coded one attr at a time. |
| 30 | + |
| 31 | +## What the field does (research) |
| 32 | + |
| 33 | +- **TypeScript compiler** (most relevant — we're TS-first): a strict two-phase split. The |
| 34 | + **binder** walks the tree once and builds a **symbol table**; the **checker** then |
| 35 | + consumes that table to resolve names and report errors. Cross-file merging is handled by |
| 36 | + a **merged-symbols table** — the direct analogue of our overlay/merge across `meta.*` |
| 37 | + files. Logic lives in the binder/checker, **not** on the AST nodes. |
| 38 | + ([binder notes](https://github.com/microsoft/TypeScript-Compiler-Notes/blob/main/codebase/src/compiler/binder.md), [TS Deep Dive: Binder](https://basarat.gitbook.io/typescript/overview/binder)) |
| 39 | +- **GraphQL** validation is a **visitor** running a **set of rules** over the AST. |
| 40 | + ([Life of a GraphQL Query — Validation](https://medium.com/@cjoudrey/life-of-a-graphql-query-validation-18a8fb52f189)) |
| 41 | +- **Crystal compiler** runs the AST through **multiple visitor passes**, one per concern |
| 42 | + (classes, vars, types…). ([Visiting an AST](https://patshaughnessy.net/2022/1/22/visiting-an-abstract-syntax-tree)) |
| 43 | +- **Smithy** (AWS — a model standard we benchmark against): validation is a registry of |
| 44 | + **validators** that match shapes with **selectors** (a DSL) and emit **validation events** |
| 45 | + with configurable **severity** + source location. Custom validators are **declarative — |
| 46 | + no code**. ([Smithy model validation](https://smithy.io/2.0/spec/model-validation.html), [Selectors](https://smithy.io/2.0/spec/selectors.html)) |
| 47 | +- **XSD** has `key`/`keyref` for *limited declarative* referential integrity; **JSON |
| 48 | + Schema is explicitly context-free** and does **not** do cross-references (referential |
| 49 | + integrity is "out of scope — application logic"). Cross-element checks need |
| 50 | + **Schematron / SHACL**. ([JSON Schema scope](https://github.com/json-schema-org/json-schema-spec/wiki/Scope-of-JSON-Schema-Validation)) |
| 51 | +- **SHACL** (W3C): fully **declarative** shapes, rules separated from execution; |
| 52 | + **parameterised constraint components** are where "a validation checklist crosses into a |
| 53 | + type system." ([SHACL overview](https://medium.com/fluree/what-is-shacl-with-examples-2697f659d465), [parameterised constraints](https://ontologist.substack.com/p/shacls-hidden-superpower-parameterised)) |
| 54 | + |
| 55 | +**Three convergent lessons:** (1) build a **symbol table** before resolving references; |
| 56 | +(2) run checks as **visitor/rule passes**, not as methods on the nodes; (3) the most |
| 57 | +sophisticated systems make cross-references **declarative + data-driven**, so new |
| 58 | +reference kinds validate for free. |
| 59 | + |
| 60 | +## Where we are today |
| 61 | + |
| 62 | +The loaders already run a **multi-pass visitor over the tree** (`validation-passes.ts`, |
| 63 | +`ValidationPhase.java`, `validation_passes.py`, `ValidationPasses.cs`) and Python even has |
| 64 | +a `_build_object_index` — a primitive **symbol table**. We are *already on the |
| 65 | +best-practice path*; it's just **ad-hoc**: the symbol table is rebuilt per pass, each |
| 66 | +ref-kind (`extends`, `@payloadRef`, origin paths) is hand-resolved separately, and |
| 67 | +`@objectRef`/`@references` were simply missed. There is **no node-level `validate()`** for |
| 68 | +metadata structure in any port, and that's consistent with the research. |
| 69 | + |
| 70 | +## Options |
| 71 | + |
| 72 | +### Option A — Node-self-validation: `validate()` on each typed node |
| 73 | + |
| 74 | +`MetaRoot.validate()` recurses; `MetaRelationship` / `ReferenceIdentity` override to check |
| 75 | +their own refs. |
| 76 | + |
| 77 | +- **Pros:** ergonomic entry point (`root.validate()`); idiomatic OO; natural in Java where |
| 78 | + nodes are typed subclasses. |
| 79 | +- **Cons:** **the research's anti-pattern.** (1) Cross-references need the *global* symbol |
| 80 | + table — a node validating itself either rebuilds the index (O(n²)) or reaches back to |
| 81 | + root, breaking encapsulation anyway. (2) **Doesn't port:** TS/Python have a *single |
| 82 | + generic `MetaData`* — there's no `MetaRelationship` subclass to override, so they'd |
| 83 | + `switch(type)` inside `validate()` = the procedural pass relocated onto a method, but now |
| 84 | + scattered. (3) Adding a new cross-cutting rule means editing N node classes. (4) Couples |
| 85 | + validation lifecycle to construction. Verdict: **the appealing API, the wrong internals.** |
| 86 | + |
| 87 | +### Option B — Keep ad-hoc procedural passes (status quo + the two missing checks) |
| 88 | + |
| 89 | +Just add `@objectRef`/`@references` resolution as two more hand-written passes (what the |
| 90 | +in-flight TS/Java change does). |
| 91 | + |
| 92 | +- **Pros:** minimal; already green in TS/Java; ports cleanly. |
| 93 | +- **Cons:** doesn't address "**more sophisticated**." The symbol table stays implicit and |
| 94 | + rebuilt; every future ref-kind is another bespoke pass; no severity model beyond |
| 95 | + error/warn; drift risk between the 5 hand-written copies per pass. |
| 96 | + |
| 97 | +### Option C — Formalized semantic phase: symbol table + rule registry + **declarative reference model** (recommended) |
| 98 | + |
| 99 | +Make explicit what the field does: |
| 100 | + |
| 101 | +1. **One symbol table** built once per load (object index by name/fqn/resolutionKey — Python |
| 102 | + already has the seed), shared by every resolving pass. The merged/overlay step is our |
| 103 | + "merged-symbols table." |
| 104 | +2. A **validation-rule registry** — each rule is a small visitor (`(root, symbols, emit) |
| 105 | + → void`). The existing passes *become* registered rules; new rules register without |
| 106 | + touching the loader or the nodes. Mirrors Smithy validators / GraphQL rule sets. |
| 107 | +3. **Declarative reference descriptors.** Today `REF_BEARING_ATTR_NAMES` already lists the |
| 108 | + ref-bearing attrs. Promote that to a typed registry: each ref attr declares *what it |
| 109 | + points at* (`objectRef → object`, `references → object(+fields)`, `payloadRef → |
| 110 | + object.value`, `extends → same-type node`, origin paths → field/relationship paths). **A |
| 111 | + single generic resolver** then validates **every** cross-reference uniformly against the |
| 112 | + symbol table — `@objectRef`/`@references` stop being special-cased and any future ref |
| 113 | + attr is covered for free. This is XSD `keyref` / Smithy selectors / SHACL, adapted. |
| 114 | +4. **Severity-tagged diagnostics** (error / warn / info) with source location — we already |
| 115 | + have the error+warning envelope; formalize the levels (Smithy's model). |
| 116 | +5. **Ergonomic entry point:** expose `root.validate()` (or `loader.validate()`) as the |
| 117 | + public API **that runs this phase** — so the API the human/agent reaches for is the nice |
| 118 | + one, while the internals are the best-practice registry, **not** logic-in-each-node. This |
| 119 | + reconciles the "`validate()` on `MetaRoot`" instinct with the research. |
| 120 | + |
| 121 | +- **Pros:** matches every mature comparable; cross-references become **data-driven and |
| 122 | + exhaustive** (the actual "sophisticated validation" ask); extensible without touching |
| 123 | + nodes or the loader; **ports identically** to OO and generic-node ports (rules are |
| 124 | + functions, not methods); one symbol table, no O(n²); conformance gates the rule set. |
| 125 | +- **Cons:** more upfront design than Option B; the declarative ref registry is a new |
| 126 | + (small) cross-port contract to keep in lockstep. |
| 127 | + |
| 128 | +### Option D — Full declarative constraint language (SHACL/Schematron-style) for all rules |
| 129 | + |
| 130 | +Express *every* rule (structural + relational) as data. |
| 131 | + |
| 132 | +- **Pros:** ultimate extensibility; non-code authoring of org-specific rules (an enterprise |
| 133 | + ask). |
| 134 | +- **Cons:** large; a second mini-language to design, port, and conformance-gate; most local |
| 135 | + checks are clearer as code. **Premature** — but Option C's reference descriptors are a |
| 136 | + deliberate first step *toward* this if demand appears. |
| 137 | + |
| 138 | +## Recommendation |
| 139 | + |
| 140 | +**Adopt Option C, incrementally — and treat Option A's `root.validate()` purely as the |
| 141 | +public entry point over C's internals, never as logic-on-nodes.** |
| 142 | + |
| 143 | +Concrete phasing: |
| 144 | + |
| 145 | +- **Phase 1 (now): land the two missing checks as registered rules.** The in-flight |
| 146 | + `@objectRef` / `@references` enforcement (TS + Java done, green; Python + C# to match) |
| 147 | + ships as-is — it's already the visitor/pass shape Option C formalizes, and it closes the |
| 148 | + correctness gap the user hit. New codes `ERR_INVALID_REFERENCE` (+ `@objectRef` folded |
| 149 | + into `ERR_INVALID_RELATIONSHIP`), gated by `error-relationship-unresolved-objectref` and |
| 150 | + `error-identity-reference-unresolved` fixtures in all five ports. |
| 151 | +- **Phase 2: extract the symbol table + rule registry.** Refactor the existing passes into |
| 152 | + a registered list sharing one built-once object index. Behavior-preserving; conformance |
| 153 | + is the safety net. Add `root.validate()` / `loader.validate()` as the entry point. |
| 154 | +- **Phase 3: declarative reference descriptors.** Replace the five hand-coded resolvers |
| 155 | + (`extends`, `objectRef`, `references`, `payloadRef`, origins) with one generic |
| 156 | + reference-resolution rule driven by per-attr ref descriptors. This is the leap to |
| 157 | + "sophisticated, especially object ref" — and where a renamed/removed target is caught |
| 158 | + uniformly, by construction, for every reference kind present and future. |
| 159 | +- **Phase 4 (optional, demand-driven): formalize diagnostic severities** and consider a |
| 160 | + thin declarative-rule surface (Option D) only if enterprise "custom org rules" demand it. |
| 161 | + |
| 162 | +Cross-port note: Java/Kotlin may *additionally* expose the idiomatic `root.validate()` |
| 163 | +instance method (it's natural with typed nodes), but its body calls the shared semantic |
| 164 | +phase — it does **not** re-implement checks per node class. TS/Python/C# expose the same |
| 165 | +entry point as a function over the generic node. Same behavior, conformance-locked. |
| 166 | + |
| 167 | +## Immediate decision needed |
| 168 | + |
| 169 | +Phase 1 is independent and already mostly done. Phases 2–3 are the "sophisticated |
| 170 | +validation" investment. Recommend: **merge Phase 1 now** (close the gap), then schedule |
| 171 | +Phase 2–3 as a focused follow-up (one FR, cross-port, conformance-gated) rather than |
| 172 | +blocking the bug fix on the refactor. |
0 commit comments