Merge remote-tracking branch 'origin/main' into sp-e-cli-parity

Author: dmealing

docs/superpowers/plans/2026-05-31-typed-enums-payload.md

@@ -0,0 +1,67 @@
+# Typed Enums in Payload VOs — Cross-Port Design
+
+_Date: 2026-05-31. Status: approved (design). All 5 ports. Builds on the shipped extract tier + the FR-010/011 enum coercion engine._
+
+## Problem
+
+A `field.enum` (closed value set via `@values`) is typed **inconsistently and loosely** in each port's strict payload VO (the `extract<Name>` target): TS `unknown`, Python `str`, C# `object`, Java/Kotlin `String`. None is constrained to the enum's values — even though TS/C#/Kotlin already generate value-constrained enum types for **entities**, the payload path ignores them. The extract tier's whole value is a strongly-typed payload; an enum field typed `object`/`unknown`/`string` undermines that.
+
+## Goal
+
+Every port's strict payload types a `field.enum` with a **value-constrained, idiomatic** type; enum **arrays** likewise. The lenient/recovered mirror stays raw `string`. No engine/behavior change — typed enums are a codegen-output change only.
+
+## Foundation (already in place)
+
+- **Enum metamodel:** value set is inline per field via `@values` (a required, non-empty string array; members constrained to `[A-Za-z_][A-Za-z0-9_]*`, so **symbol == stored string** in every language). No named enum metatype. **Sharing** via abstract-field `extends`: an abstract `field.enum` declares `@values`; concrete fields `extends` it and inherit the effective values.
+- **Existing entity enum-type emitters (reuse):** TS `renderEnumTypeAliases` (union aliases, `inferred-types.ts`), C# `CollectEnumDecls` (nested `enum`, `EntityGenerator.cs`), Kotlin `emitEnumFile` (`enum class`, `KotlinEntityGenerator.kt`). Java + Python have none.
+- **Shared naming rule (reuse):** `<Super>` when the field extends an abstract enum (one shared type), else `<Owner><Field>` (PascalCase). Members verbatim.
+- **Engine enum coercion (unchanged):** normalize → `@enumAlias` → `@coerceDefault` → MALFORMED, validating against the closed set. By the time `extract` builds the strict payload, an enum value is always a valid member or the field is lost/MALFORMED.
+
+## Design
+
+### Per-port typed enum
+
+| Port | Strict-payload enum type | Where the type is emitted | Emitter | Extract coercion |
+|---|---|---|---|---|
+| **TS** | `export type X = "A" \| "B"` (string-literal union) | the payload module | reuse `renderEnumTypeAliases` | **identity** — a validated member string already satisfies the union |
+| **Python** | `Literal["A","B"]` (a named alias `X = Literal[...]` when shared via `extends`, else inline) | the payload module | new (small — annotation + optional alias) | **identity** — string satisfies `Literal` |
+| **C#** | nested `public enum X { A, B }` | inside the payload `record` | reuse the `CollectEnumDecls` pattern | `System.Enum.Parse<X>(s)` |
+| **Kotlin** | `enum class X { A, B }` (separate `.kt` file) | alongside the payload | reuse `emitEnumFile` | `X.valueOf(s)` |
+| **Java** | `public enum X { A, B }` (**new** emitter; separate file or nested, matching the payload-gen file model) | alongside the payload | new (mirror C#/Kotlin shape + naming) | `X.valueOf(s)` |
+
+Naming is the established rule in all five (`<Super>` / `<Owner><Field>`), so two fields extending one abstract enum yield **one** generated type per output unit (deduped as each entity emitter already dedupes).
+
+### The lenient mirror stays string
+
+The all-nullable `<Name>Extracted` mirror keeps the enum leaf as raw `string` (`string | null`, `str | None`, etc.) — it represents what recover salvaged (canonical member or, when classified, absent). Only the strict `extract` payload carries the typed enum. This is the clean split: **lenient = raw text + report; extract = validated typed value.**
+
+### Extract mapper coercion
+
+The extract mapper (mirror → strict) gains an enum branch:
+- **TS / Python:** identity — the mirror string is assigned directly to the union/`Literal` field (it is, by construction, a valid member). No runtime call.
+- **C# / Java / Kotlin:** `X.valueOf(s)` / `Enum.Parse<X>(s)` on the validated member string. Safe because the generated enum constants == `@values` and the engine produced a member.
+- **Enum arrays:** the scalar-array branch (already hardened to drop nulls + convert per element) converts each element via the same identity/`valueOf` rule, producing `X[]` / `List<X>` / `list[Literal]`.
+
+### Coercion safety / edge cases
+
+- Required enum, present → valid member → `valueOf`/identity safe.
+- Required enum, MALFORMED/lost → `extract` throws (existing lost-required path); never reaches `valueOf`.
+- Optional enum, absent → null where representable (TS/Python). C#/Kotlin/Java payloads type every field non-null (the existing documented all-`required` divergence), so optional-absent enums aren't representable there — unchanged by this work.
+- `valueOf`/`Parse` is defensively safe: `@values` are valid identifiers (metamodel-enforced) and equal the generated constants.
+
+## Testing
+
+- **Engine + conformance unchanged:** the `extract-conformance` corpus is engine-level (string verdicts); it does not change, and the 4 runners stay green. Enum coercion behavior is already covered there.
+- **Per-port compile-and-run proof (the gate):** extend each extractor test fixture with (a) a single enum field and (b) an enum array; regenerate; assert the strict payload field is the **typed enum** (TS union value / Python `Literal` member / C#-Java-Kotlin real enum constant e.g. `Status.PUBLISHED`), populated correctly from dirty input, AND that the lenient mirror still returns the raw string. Because the generated code COMPILES, the type is genuinely value-constrained; because it RUNS, the coercion works.
+- **Shared-enum naming proof:** one fixture where two payload fields `extends` one abstract enum → exactly one generated type, name `<Super>`, used by both fields — proving dedup + the shared-naming rule per port.
+
+## Out of scope
+
+- **Entity codegen** enum typing (already done in TS/C#/Kotlin; not the extract-target gap). Java/Python entity enum typing is a possible later parity item, not this work.
+- **Engine/coercion behavior** — unchanged (pure codegen-typing change).
+- The lenient mirror staying string is intentional, not a gap.
+- **Publish** — deferred to explicit user confirm.
+
+## Sequencing
+
+Single branch `worktree-typed-enums`, single merge. Per port: payload generator emits the enum type + types the field → extract mapper coerces → enum array → compile-and-run proof. Order likely TS/Python first (identity, simplest), then C#/Kotlin (reuse emitters + valueOf), then Java (new emitter). A final shared-naming fixture + closeout.

docs/superpowers/specs/2026-05-31-typed-enums-payload-design.md

@@ -40,7 +40,8 @@ public sealed class ExtractorCodegenTests
     //   • quantities: REQUIRED scalar-array long[]   (StrictArg long .Value unwrap)
     //   • weights : REQUIRED scalar-array double[]   (StrictArg double .Value unwrap)
     //   • flagsArr: REQUIRED scalar-array bool[]   (StrictArg bool .Value unwrap)
-    //   • priority: REQUIRED enum (strict-map `m.priority!` run-proof)
+    //   • priority: REQUIRED enum -> nested enum OrderPriority (Enum.Parse strict-map run-proof)
+    //   • labels  : REQUIRED enum ARRAY -> IReadOnlyList<OrderLabels> (per-element Enum.Parse)
     //   • note    : scalar (not @required)
     //   • shipTo  : single nested Customer (not @required)
     private const string Model = """
@@ -62,6 +63,7 @@ public sealed class ExtractorCodegenTests
         { "field.double": { "name": "weights", "isArray": true, "@required": true } },
         { "field.boolean":{ "name": "flagsArr", "isArray": true, "@required": true } },
         { "field.enum":   { "name": "priority", "@required": true, "@values": ["LOW", "HIGH"] } },
+        { "field.enum":   { "name": "labels", "isArray": true, "@required": true, "@values": ["A", "B"] } },
         { "field.string": { "name": "note" } },
         { "field.object": { "name": "shipTo", "@objectRef": "Customer" } }
       ]}},
@@ -146,6 +148,7 @@ public void Generated_extract_populates_strict_graph_from_dirty_text()
             "  \"weights\": [1.5, 2.25]," +
             "  \"flagsArr\": [true, false, true]," +
             "  \"priority\": \"HIGH\"," +
+            "  \"labels\": [\"A\", \"B\"]," +
             "  \"shipTo\": { \"name\": \"Grace\" }, }\n```";
 
         var order = extract.Invoke(null, new object?[] { orderMo, dirty })!;
@@ -197,13 +200,32 @@ public void Generated_extract_populates_strict_graph_from_dirty_text()
         Assert.Equal(new object[] { true, false, true }, flags.ToArray());
         Assert.IsType<bool>(flags[0]);
 
-        // required ENUM through the strict tier — the StrictArg enum branch (`m.priority!`,
-        // string-backed) populates the strict record from dirty input. NOTE the strict property
-        // type is `object` (not `string`): PayloadCodegen's scalar map has no enum entry, so an enum
-        // field falls through to the `object` default — but the value is the string-backed member.
+        // required ENUM through the strict tier — the StrictArg enum branch
+        // (`System.Enum.Parse<OrderPriority>(m.priority!)`) coerces the string-backed mirror member
+        // into the generated NESTED enum type. The strict property type is the nested enum
+        // (Order.OrderPriority), NOT `object` and NOT `string`.
         var priorityProp = order.GetType().GetProperty("priority")!;
-        Assert.Equal(typeof(object), priorityProp.PropertyType);
-        Assert.Equal("HIGH", priorityProp.GetValue(order));
+        Assert.True(priorityProp.PropertyType.IsEnum, "priority should be a nested enum type, not object/string");
+        Assert.Equal("OrderPriority", priorityProp.PropertyType.Name);
+        var priorityVal = priorityProp.GetValue(order)!;
+        Assert.Equal("HIGH", priorityVal.ToString());
+        Assert.Equal(System.Enum.Parse(priorityProp.PropertyType, "HIGH"), priorityVal);
+
+        // required ENUM ARRAY -> IReadOnlyList<OrderLabels> (element type IsEnum), per-element
+        // Enum.Parse from the string-backed mirror list. Proves the enum-array routes through the
+        // string-LIST reader (not the scalar enum reader) and each element coerces to the nested enum.
+        var labelsProp = order.GetType().GetProperty("labels")!;
+        Assert.True(typeof(IEnumerable).IsAssignableFrom(labelsProp.PropertyType));
+        var labelsElemType = labelsProp.PropertyType.GetGenericArguments().Single();
+        Assert.True(labelsElemType.IsEnum, "labels element should be a nested enum type");
+        Assert.Equal("OrderLabels", labelsElemType.Name);
+        Assert.Equal(new[] { "A", "B" }, labelsElemType.GetEnumNames());
+        var labels = ((IEnumerable)labelsProp.GetValue(order)!).Cast<object>().ToList();
+        Assert.Equal(new object[]
+        {
+            System.Enum.Parse(labelsElemType, "A"),
+            System.Enum.Parse(labelsElemType, "B"),
+        }, labels.ToArray());
 
         // non-@required single nested, present -> populates.
         var shipTo = order.GetType().GetProperty("shipTo")!.GetValue(order)!;
@@ -240,6 +262,7 @@ public void Generated_extract_opts_overload_populates_same_strict_graph()
             "  \"weights\": [3.5]," +
             "  \"flagsArr\": [false]," +
             "  \"priority\": \"HIGH\"," +
+            "  \"labels\": [\"B\"]," +
             "  \"shipTo\": { \"name\": \"Grace\" } }\n```";
 
         var opts = optsType.GetMethod("Defaults")!.Invoke(null, null);
@@ -253,7 +276,9 @@ public void Generated_extract_opts_overload_populates_same_strict_graph()
         var quantities = ((IEnumerable)order.GetType().GetProperty("quantities")!.GetValue(order)!)
             .Cast<object>().ToList();
         Assert.Equal(new object[] { 42L }, quantities.ToArray());
-        Assert.Equal("HIGH", order.GetType().GetProperty("priority")!.GetValue(order));
+        var priorityProp = order.GetType().GetProperty("priority")!;
+        Assert.True(priorityProp.PropertyType.IsEnum);
+        Assert.Equal("HIGH", priorityProp.GetValue(order)!.ToString());
     }
 
     [Fact]
@@ -285,13 +310,73 @@ public void Re_exposed_extract_never_throws_on_clean_input()
         const string clean =
             "{ \"orderId\": \"A-7\", \"customer\": { \"name\": \"Ada\" }," +
             "  \"lines\": [ { \"sku\": \"A\", \"qty\": 1 } ], \"tags\": [\"a\"], \"scores\": [1]," +
-            "  \"quantities\": [1], \"weights\": [1.0], \"flagsArr\": [true], \"priority\": \"LOW\" }";
+            "  \"quantities\": [1], \"weights\": [1.0], \"flagsArr\": [true], \"priority\": \"LOW\"," +
+            "  \"labels\": [\"A\"] }";
 
         var result = extractLenient.Invoke(null, new object?[] { orderMo, clean })!;
         var report = result.GetType().GetProperty("Report")!.GetValue(result)!;
         Assert.False((bool)report.GetType().GetMethod("HasLostRequired")!.Invoke(report, null)!);
     }
 
+    // ---- nested-enum-typed payload: strict typing + lenient mirror stays string ----
+
+    [Fact]
+    public void Strict_payload_types_enum_as_nested_enum_lenient_mirror_stays_string()
+    {
+        var root = Load(Model);
+
+        // STRICT payload: enum scalar -> nested enum type; enum array -> IReadOnlyList<NestedEnum>.
+        var payloadSrc = PayloadCodegen.GeneratePayloadRecords(root, "Order");
+        Assert.Contains("public enum OrderPriority { LOW, HIGH }", payloadSrc);
+        Assert.Contains("public enum OrderLabels { A, B }", payloadSrc);
+        Assert.Contains("public required OrderPriority priority { get; init; }", payloadSrc);
+        Assert.Contains("public required IReadOnlyList<OrderLabels> labels { get; init; }", payloadSrc);
+        Assert.DoesNotContain("public required object priority", payloadSrc);
+
+        // The extract mapper coerces the string mirror via System.Enum.Parse<NestedEnum>.
+        var extractorSrc = Assert.Single(new ExtractorGenerator().Generate(Ctx(root))).Content;
+        Assert.Contains("System.Enum.Parse<Order.OrderPriority>(m.priority!)", extractorSrc);
+        Assert.Contains("System.Enum.Parse<Order.OrderLabels>(", extractorSrc);
+
+        // LENIENT mirror: the enum leaf stays string-backed (scalar string?, array string list).
+        var ctx = Ctx(root);
+        var parserSrc = Assert.Single(new OutputParserGenerator().Generate(ctx)).Content;
+        Assert.Contains("string? priority { get; init; }", parserSrc);
+        Assert.Contains("IReadOnlyList<string?>? labels { get; init; }", parserSrc);
+        // No nested enum type bleeds into the lenient mirror.
+        Assert.DoesNotContain("OrderPriority priority", parserSrc);
+        Assert.DoesNotContain("OrderLabels", parserSrc);
+    }
+
+    // ---- shared-enum dedup proof: one abstract enum, two extending fields, ONE decl ----
+
+    private const string SharedEnumModel = """
+    { "metadata.root": { "package": "acme::orders", "children": [
+      { "field.enum": { "name": "Priority", "abstract": true, "@values": ["LOW", "HIGH"] } },
+      { "object.value": { "name": "Ticket", "children": [
+        { "field.string": { "name": "ticketId", "@required": true } },
+        { "field.enum":   { "name": "currentPriority",  "@required": true, "extends": "Priority" } },
+        { "field.enum":   { "name": "previousPriority", "@required": true, "extends": "Priority" } }
+      ]}}
+    ]}}
+    """;
+
+    [Fact]
+    public void Shared_abstract_enum_emits_one_nested_enum_both_fields_typed_by_super()
+    {
+        var root = Load(SharedEnumModel);
+        var payloadSrc = PayloadCodegen.GeneratePayloadRecords(root, "Ticket");
+
+        // Exactly ONE nested enum declaration, named for the super (deduped), members verbatim.
+        int enumDeclCount = System.Text.RegularExpressions.Regex.Matches(payloadSrc, @"public enum Priority \{").Count;
+        Assert.True(enumDeclCount == 1, $"expected exactly one nested enum Priority, got {enumDeclCount}");
+        Assert.Contains("public enum Priority { LOW, HIGH }", payloadSrc);
+
+        // BOTH fields typed by the shared enum.
+        Assert.Contains("public required Priority currentPriority { get; init; }", payloadSrc);
+        Assert.Contains("public required Priority previousPriority { get; init; }", payloadSrc);
+    }
+
     private static Assembly Compile(MetaRoot root)
     {
         var ctx = Ctx(root);

server/csharp/MetaObjects.Codegen.Tests/ExtractorCodegenTests.cs

@@ -281,12 +281,15 @@ private static string MapperArg(MetaData field, MetaData root)
         // Enum / scalar / scalar-array: the runtime already coerced; read + light-coerce via Dlg*.
         // A scalar ARRAY maps each element through the SAME per-kind Dlg* reader the single scalar
         // uses, so the produced element type matches the kind-typed mirror list (int?/long?/...).
-        if (field.SubType == FIELD_SUBTYPE_ENUM) return $"DlgString(ReadProp(o, {key}))";
+        // IsArray is checked BEFORE the single-scalar enum return so an ENUM array routes through the
+        // string-LIST reader (DlgList(..., DlgString)), matching its IReadOnlyList<string?>? mirror —
+        // NOT the single DlgString reader (the enum-before-isArray ordering fix).
         if (Fr010FieldMapping.IsArray(field))
         {
             string elemReader = field.SubType == FIELD_SUBTYPE_ENUM ? "DlgString" : ScalarReader(field.SubType);
             return $"DlgList(ReadProp(o, {key}), {elemReader})";
         }
+        if (field.SubType == FIELD_SUBTYPE_ENUM) return $"DlgString(ReadProp(o, {key}))";
         return $"{ScalarReader(field.SubType)}(ReadProp(o, {key}))";
     }