Skip to content

Commit f1e44b7

Browse files
dmealingclaude
andcommitted
Merge origin/main into sp-g-registry-conformance (FR-017 + SP-G); reconcile registry coverage snapshot (template.output attrs now exercised by main's new template-doc fixtures)
Auto-merge was clean (0 conflicts). The only semantic reconciliation needed was regenerating fixtures/registry-conformance/coverage-report.json: main's new template-doc-{document,email} doc fixtures now exercise six previously-untested template.output attrs (htmlBodyRef, kind, maxChars, requiredTags, subjectRef, textBodyRef) — pure coverage progress, no newly-untested subtype. Verified across all 5 ports. No C# codegen drift (drift test green — M:N nav still not emitted per FR-017 Phase 3, so our committed .g.cs stayed valid). conformance.yml auto-merge kept both sides (our registry-conformance TS lines + main's generator-registry-conformance CI jobs in C#/Java/Kotlin/Python). No attr.stringarray reintroduction by main. New generator-registry-conformance corpus green in every port. Pre-existing-on-main failures (NOT merge-induced, reproduced on a pristine main checkout): template-doc-document / template-doc-email fail the strict metadata conformance runners (TS/C#/Java) with "fixture declares no expectation files" — main's doc fixtures live under fixtures/conformance/ but ship only expected/*.md, which the strict runners reject (Python's tolerant runner passes). Kotlin integration-tests-kotlin KotlinCodegenMatchesReferenceTest timestamp drift (main commit f237c23/f9ed09d7) is Docker-gated and untouched by this merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2 parents 3e5f074 + 1f8f1d4 commit f1e44b7

99 files changed

Lines changed: 8880 additions & 522 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/conformance.yml

Lines changed: 31 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ jobs:
5757
cd ../render && bun test test/render-conformance.test.ts test/verify-conformance.test.ts test/extract/extract-conformance.test.ts test/output-prompt-conformance.test.ts
5858
# Validator-parity corpus (generated input validation). No Docker.
5959
cd ../integration-tests && bun test test/validation-conformance.test.ts
60+
# Doc-template + CLI suites: byte-identity template gate, embedded-template
61+
# gate, neutrality / collision guards, and the docs golden corpus +
62+
# `meta docs` command. Whole-package so future tests auto-run. No Docker.
63+
cd ../codegen-ts && bun test
64+
cd ../cli && bun test
6065
6166
- name: C# conformance
6267
if: matrix.language == 'csharp'
@@ -67,6 +72,8 @@ jobs:
6772
dotnet test MetaObjects.Render.Tests/MetaObjects.Render.Tests.csproj --nologo --verbosity quiet
6873
# Validator-parity corpus (generated DataAnnotations input validation).
6974
dotnet test MetaObjects.Codegen.Tests/MetaObjects.Codegen.Tests.csproj --filter "FullyQualifiedName~ValidationConformance" --nologo --verbosity quiet
75+
# Generator stable-name registry conformance (ADR-0021 D3).
76+
dotnet test MetaObjects.Codegen.Tests/MetaObjects.Codegen.Tests.csproj --filter "FullyQualifiedName~GeneratorRegistryConformance" --nologo --verbosity quiet
7077
7178
- name: Java conformance
7279
if: matrix.language == 'java'
@@ -80,8 +87,9 @@ jobs:
8087
mvn -pl metadata test -Dtest='ConformanceTest,YamlConformanceTest,ObjectModelConformanceTest' -q
8188
# Byte-exact render / verify / extract / output-prompt corpora (render module).
8289
mvn -pl render test -Dtest='RenderCrossPortReportTest,VerifyConformanceTest,ExtractConformanceTest,OutputPromptConformanceTest' -q
83-
# Validator-parity corpus (generated DTO jakarta.validation, codegen-spring module).
84-
mvn -pl codegen-spring test -Dtest='ValidationConformanceTest' -q
90+
# Validator-parity corpus (generated DTO jakarta.validation, codegen-spring module)
91+
# + generator stable-name registry conformance (ADR-0021 D3).
92+
mvn -pl codegen-spring test -Dtest='ValidationConformanceTest,GeneratorRegistryConformanceTest' -q
8593
8694
- name: Python conformance
8795
if: matrix.language == 'python'
@@ -93,6 +101,8 @@ jobs:
93101
uv run pytest tests/render -q
94102
# Validator-parity corpus (generated Pydantic input validation).
95103
uv run pytest tests/codegen/test_validation_conformance.py -q
104+
# Generator registry CLI ergonomics (gen --list / --generators selection, ADR-0021 D3).
105+
uv run pytest tests/codegen/test_cli_registry.py -q
96106
97107
conformance-kotlin:
98108
# Kotlin (codegen-kotlin, KotlinPoet on the JVM) ships no loader / YAML /
@@ -117,8 +127,9 @@ jobs:
117127
# SNAPSHOT-resolution race when the test module references freshly-built
118128
# artifacts.
119129
mvn -pl codegen-kotlin -am install -DskipTests -q
120-
# object-model + output-prompt + validator-parity corpora (the corpora Kotlin has).
121-
mvn -pl codegen-kotlin test -Dtest='ObjectModelConformanceTest,OutputPromptConformanceTest,ValidationConformanceTest' -q
130+
# object-model + output-prompt + validator-parity corpora (the corpora Kotlin has)
131+
# + generator stable-name registry conformance (ADR-0021 D3).
132+
mvn -pl codegen-kotlin test -Dtest='ObjectModelConformanceTest,OutputPromptConformanceTest,ValidationConformanceTest,GeneratorRegistryConformanceTest' -q
122133
123134
completeness-gate:
124135
needs: [conformance, conformance-kotlin]
@@ -128,3 +139,19 @@ jobs:
128139
- uses: oven-sh/setup-bun@v2
129140
- run: bun install
130141
- run: cd server/typescript/packages/metadata && bun run conformance:mutation
142+
143+
doc-template-drift:
144+
# Fails if the bundled package copy of the canonical doc templates, or the
145+
# embedded-templates TS module (used by the standalone `meta` binary), is
146+
# stale. sync-doc-templates.sh reproduces both from the canonical root
147+
# templates/docs/ source; a non-empty diff means a dev forgot to run it.
148+
needs: fixture-lint
149+
runs-on: ubuntu-latest
150+
steps:
151+
- uses: actions/checkout@v4
152+
- uses: oven-sh/setup-bun@v2
153+
- run: bun install
154+
- name: Regenerate bundled + embedded doc templates from canonical
155+
run: bash scripts/sync-doc-templates.sh
156+
- name: Fail if synced/generated doc-template artifacts are stale
157+
run: git diff --exit-code -- server/typescript/packages/codegen-ts/templates server/typescript/packages/codegen-ts/src/render-engine/embedded-templates.generated.ts

docs/features/cli.md

Lines changed: 56 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -23,24 +23,64 @@ command surface splits in two:
2323
| Project scaffold (`init`) | Node `meta` | `meta init` | TS projects |
2424
| **Schema migrate** | **Node `meta`** | `meta migrate` | **any backend** — schema is Node-only (ADR-0015) |
2525
| **Schema drift** (`verify --db`) | **Node `meta`** | `meta verify --db` | **any backend** — live-DB drift, Node-only |
26+
| **Codegen drift** (`verify --codegen`) | **Node `meta`** | `meta verify --codegen` | TS reference (ADR-0021 D2) — regen-to-temp + diff committed output |
27+
| **Template/prompt drift** (`verify --templates`) | **Node `meta`** | `meta verify --templates` | TS reference (ADR-0021 D2) — `{{field}}`↔payload; the bare-`verify` default |
2628
| TS codegen | Node `meta` | `meta gen` | TS projects |
27-
| C# codegen | `dotnet meta` | `dotnet meta gen` / `verify` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta` |
29+
| C# codegen | `dotnet meta` | `dotnet meta gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates`) |
2830
| Java/Kotlin codegen | Maven plugin | `mvn metaobjects:generate` (`meta:gen`) | Kotlin generators run through the same goal — see below |
29-
| Java/Kotlin codegen drift | Maven plugin | `mvn metaobjects:verify` (`meta:verify`) | regenerate + fail on drift vs committed output (generator-neutral) |
30-
| Python codegen | console-script | `metaobjects gen` / `verify` | `[project.scripts] metaobjects`**not** `meta` (that's the Node schema CLI) |
31-
32-
## Two meanings of `verify` — keep them distinct
33-
34-
- **Schema `verify --db`** (Node `meta`, any backend): live-DB drift — does the
35-
database match the metadata? Node-only, ADR-0015.
36-
- **Codegen `verify`** (`dotnet meta verify`, `mvn meta:verify`, `metaobjects
37-
verify`): regenerate code from metadata into a temp location and fail if it
38-
differs from the committed generated output — "a teammate changed metadata
39-
without regenerating → caught". This is the enterprise-CI primitive added per
40-
port; it does **not** touch a database.
41-
- (Separately, the FR-004 **template/prompt** drift check — `Renderer.verify`
42-
also surfaces under `verify` in the ports that ship it; it checks `{{...}}`
43-
references against the payload VO. It is a third, orthogonal facet.)
31+
| Java/Kotlin verify | Maven plugin | `mvn metaobjects:verify -Dmeta.verify.mode=codegen\|templates` (`meta:verify`) | parameter-driven ADR-0021 D2 modes (one goal covers BOTH Java + Kotlin): `codegen` (default, back-compat — regen + fail on drift vs committed output, generator-neutral) / `templates` (`{{field}}`↔payload drift via the render `Verify` engine). `db` rejected ("schema verify is the migrate engine, ADR-0015") |
32+
| Python codegen | console-script | `metaobjects gen` / `verify --codegen` / `verify --templates` | `[project.scripts] metaobjects`**not** `meta` (that's the Node schema CLI); ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2) |
33+
34+
## `verify` is one verb with explicit subverbs (ADR-0021 D2)
35+
36+
Historically `verify` meant *different things per port* (TS/C# = template drift,
37+
Java/Python = codegen drift, `--db` = schema drift) and the modes were not
38+
parallel. [ADR-0021 D2](../../spec/decisions/ADR-0021-codegen-surface-coherence.md)
39+
fixes that: **`verify` is one verb with three explicit subverbs**, one
40+
vocabulary everywhere, each port implementing the modes it supports.
41+
42+
| Subverb | What it checks | Touches a DB? |
43+
|---|---|---|
44+
| `verify --db` | **Schema drift** — does the live database (or snapshot) match the metadata? (migrate engine, ADR-0015) | yes |
45+
| `verify --codegen` | **Codegen drift** — regenerate from metadata into a temp dir and fail if it differs from the committed generated output. Catches "metadata changed but `meta gen` wasn't re-run" and "someone hand-edited a generated file". | no |
46+
| `verify --templates` | **Template/prompt drift**`Renderer.verify` checks each `template.*` node's `{{field}}` references against its payload VO (FR-004). | no |
47+
48+
Rules of the contract:
49+
50+
- **Combinations aggregate.** Pass any mix (`verify --db --codegen --templates`);
51+
each selected mode runs and the **exit code is non-zero if *any* mode reports
52+
drift**.
53+
- **Bare `verify` = the port's documented back-compat default.** TS/C# default to
54+
`--templates`; Java/Python default to `--codegen`. In every case bare `verify`
55+
also prints a one-line note advertising the explicit subverbs.
56+
- **`--codegen` needs to know where the committed output lives.** It diffs
57+
against the configured `outDir` (and any per-target `outDir`) from
58+
`metaobjects.config.ts`. With no config it errors clearly (exit 2) rather than
59+
silently passing — there is nothing to diff against.
60+
- **Unknown/invalid flag → exit 2** with usage.
61+
62+
**Port status (staged per ADR-0021):** the **TypeScript Node `meta` is the
63+
reference** and implements all three subverbs today. **Python `metaobjects`
64+
ships the subverbs**: `verify --codegen` (regen-to-temp + diff vs `--out`, the
65+
historical default), `verify --templates` (each `template.*` node's `{{field}}`
66+
↔ payload-VO field tree via the render `verify()` gate, resolving refs through a
67+
filesystem provider rooted at `--templates-root`), and `verify --db` which is
68+
**cleanly rejected with exit 2** ("schema verify is the migrate engine,
69+
ADR-0015"). Bare `verify` stays `--codegen` for back-compat. The **C# `dotnet
70+
meta`** port likewise ships the codegen-side subverbs: `verify --templates` (its
71+
historical template/prompt drift gate, the C# back-compat default), `verify
72+
--codegen` (regenerate the default generator suite to a temp dir and diff against
73+
the committed `--out` tree, never touching it), and a **clean `--db` rejection
74+
(exit 2)** — bare `dotnet meta verify` keeps `--templates` and prints the subverb
75+
note. The **Java/Kotlin `mvn meta:verify`** port expresses the same vocabulary as a
76+
`mode` parameter (Maven goals are parameter-driven, not flag-driven): `-Dmeta.verify.mode=codegen`
77+
(default, byte-identical to the historical goal — regen-to-temp + diff vs committed
78+
output) and `-Dmeta.verify.mode=templates` (each `template.*` node's `{{field}}`↔payload-VO
79+
field tree via the render `Verify` engine, resolving refs through a filesystem provider rooted
80+
at `-Dmeta.verify.templateRoot`). The one goal covers BOTH Java (`codegen-spring`) and Kotlin
81+
(`codegen-kotlin`) since they share it. `mode=db` is **cleanly rejected** ("schema verify is the
82+
migrate engine, ADR-0015"); an unknown mode fails listing the valid ones. (Schema `--db` remains
83+
Node-only by the ADR-0015 design — see below.)
4484

4585
## Schema is Node-only — by design
4686

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# `meta docs` binary templates + config providers — implementation plan
2+
3+
> REQUIRED SUB-SKILL: superpowers:subagent-driven-development.
4+
5+
**Goal:** Close the two documented follow-ups from the neutral-metadata-docs feature so `meta docs` is a fully standalone Tier-2 capability:
6+
1. Framework doc templates resolve inside the `bun --compile` `meta` binary (today they don't — only adopter-provided `templates/` work there).
7+
2. `meta docs` loads `metaobjects.config.ts` providers so adopters with custom field/object types can generate docs.
8+
9+
**Architecture:** Both `codegen-ts` and `cli` build with `tsc` (the binary is built separately with `bun build --compile`). So embedding cannot use a `.mustache` text-import (tsc rejects it). Instead generate a plain-string module from the canonical `templates/`, imported by the framework provider with an on-disk-first / embedded-fallback chain. Config-provider loading mirrors `gen.ts`.
10+
11+
**Tech:** TypeScript, bun 1.3.8, tsc builds, the existing `render()`/`projectProvider` chain.
12+
13+
---
14+
15+
## Task 1: Embedded framework-template fallback (fixes binary)
16+
17+
**Intent:** `FrameworkTemplatesProvider` resolves the framework doc templates from the embedded map when the on-disk `templates/` dir is unavailable (the compiled binary). On-disk stays FIRST so dev/install layouts keep honoring local edits + adopter overrides.
18+
19+
**Files:**
20+
- Create `scripts/generate-embedded-templates.ts` (bun script) — reads canonical `templates/docs/*.mustache`, writes `server/typescript/packages/codegen-ts/src/render-engine/embedded-templates.generated.ts` exporting `export const EMBEDDED_FRAMEWORK_TEMPLATES: Record<string, string> = { "docs/<name>.md": <content>, ... }` (key = the ref WITHOUT `.mustache`, matching how refs resolve; value = exact file text). Plain string literals only (tsc-safe). Header `// @generated from templates/docs/*.mustache — DO NOT EDIT; run scripts/generate-embedded-templates.ts`.
21+
- Extend `scripts/sync-doc-templates.sh` to ALSO invoke the generator (so one command syncs the package copy AND regenerates the embedded module from canonical).
22+
- Modify `server/typescript/packages/codegen-ts/src/render-engine/framework-provider.ts``FrameworkTemplatesProvider.resolve(ref)`: keep the on-disk lookup first; if it returns undefined, return `EMBEDDED_FRAMEWORK_TEMPLATES[ref]` (or undefined). Update the stale comment (lines ~33-35) that references a non-existent embedded fallback to describe the real one.
23+
- Test: `server/typescript/packages/codegen-ts/test/embedded-templates.test.ts` — (a) GATE: every canonical `templates/docs/*.mustache` has a matching embedded entry whose content is byte-identical (prevents drift); (b) the embedded map covers exactly the canonical set (no missing/extra); (c) simulate the binary case: a `FrameworkTemplatesProvider` whose on-disk dir is unresolved still resolves `docs/entity-page.md` + `docs/template-page.md` from the embedded map.
24+
25+
- [ ] Step 1: Write `embedded-templates.test.ts` (drift gate + binary-fallback). Run → FAIL (generated module doesn't exist).
26+
- [ ] Step 2: Write `scripts/generate-embedded-templates.ts`; run it to produce `embedded-templates.generated.ts`. Wire it into `sync-doc-templates.sh`.
27+
- [ ] Step 3: Update `FrameworkTemplatesProvider.resolve` (on-disk → embedded fallback) + fix the stale comment.
28+
- [ ] Step 4: Run → PASS the new test + `bun test packages/codegen-ts` (existing docs/golden/byte-identity gates unaffected — on-disk still wins in dev). Report counts.
29+
- [ ] Step 5: Binary proof — `cd packages/cli && bun build ./bin/meta.ts --compile --outfile /tmp/meta-test --external @biomejs/wasm-bundler --external @biomejs/wasm-web`, then run `/tmp/meta-test docs <fixture-metadata> --out /tmp/docsout` from a dir with NO `templates/` and assert entity + template `.md` files are produced (framework templates now resolve from the embed). Report the result. (If building the full binary is too heavy/slow, at minimum assert the embedded provider resolves with on-disk forced-undefined — but attempt the real binary.)
30+
- [ ] Step 6: Commit. `feat(codegen-ts): embed framework doc templates so they resolve in the compiled binary`
31+
32+
## Task 2: `meta docs` loads config providers (fixes custom types)
33+
34+
**Intent:** `meta docs` loads `metaobjects.config.ts` (best-effort) and passes `providers` to `loadMemory`, so adopters with custom field/object types can generate docs — mirroring `gen.ts`.
35+
36+
**Files:**
37+
- Modify `server/typescript/packages/cli/src/commands/docs.ts` — before `loadMemory`, best-effort `loadMetaobjectsConfig(projectRoot)` (import from `../lib/load-metaobjects-config.js`, as gen.ts does); if it succeeds, pass `{ providers: forgeConfig.providers }` to `loadMemory` (guard undefined, exactly like `gen.ts:42-43`). If config loading fails/absent, proceed config-less (docs must still work with no config — do NOT hard-fail; gen treats config as required, docs must NOT).
38+
- Test: `server/typescript/packages/cli/test/docs-command.test.ts` — add a case: a fixture project WITH a `metaobjects.config.ts` (or the config-loading mechanism) declaring a custom field/object type used in its metadata; `meta docs` loads + emits docs without a "type does not resolve" failure. Also keep/confirm the existing no-config case still works (config-less).
39+
40+
- [ ] Step 1: Write the custom-type-with-config docs test (mirror how gen's config tests set up `loadMetaobjectsConfig`/a fixture config). Run → FAIL (docs currently ignores providers → custom type fails to load).
41+
- [ ] Step 2: Add best-effort config+providers loading to `docs.ts` (mirror gen.ts; non-fatal when absent).
42+
- [ ] Step 3: Run → PASS the new test + the existing no-config docs tests + `bun test packages/cli`. Report counts.
43+
- [ ] Step 4: Commit. `feat(cli): meta docs loads metaobjects.config.ts providers (custom types)`
44+
45+
## Task 3: Closeout
46+
- [ ] Full suites: `bun test packages/codegen-ts packages/cli packages/render` green; counts.
47+
- [ ] Hygiene (merge-base diff): no private names/home paths/node_modules/bunfig/lockfile/dist.
48+
- [ ] Whole-branch code-review + simplifier; fix findings.
49+
- [ ] Forward-merge to main, push, remove worktree, delete branch, update memory.
50+
51+
## Notes / guards
52+
- The embedded module is a GENERATED artifact gated against canonical `templates/` — edit root templates, run `sync-doc-templates.sh` (now also regenerates the embed). Three representations (root canonical, package bundled copy, embedded module) all gated byte-identical.
53+
- On-disk MUST stay first in `FrameworkTemplatesProvider` so adopter overrides + dev edits win; embedded is fallback only.
54+
- Don't touch native code generators, the render engine, other ports, or doc neutrality. Public-repo hygiene.

0 commit comments

Comments
 (0)