Merge FR-010 Python port — Phase 1: recover engine + dirty-input conformance

Brings FR-010's tolerant recover engine to the Python port (metaobjects/render/recover/), the
forgiving tier beyond FR-006's strict Pydantic parser. 8-stage pipeline, JSON+XML, never raises.
All 10 shared recover-conformance cases green — classifies dirty LLM output identically to Java/C#.
Python idioms: enum.Enum, dataclasses, Optional[int] as_int/as_long, bool excluded from numeric.
Pre-merge review+simplifier: ASCII-numeric parity gate + JSON depth cap (never-raise on deep nesting).
108 recover tests + render suite 179 green; mypy --strict + ruff clean.

Also lands the 3-phase Python port plan. Phases 2 (renderer) + 3 (attrs + serializer fix + codegen)
follow. FR-010 then ships in all 5 ports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: dmealing

docs/superpowers/plans/2026-05-29-fr-010-python-port.md

@@ -0,0 +1,55 @@
+# FR-010 Python port — native recover engine + output-format renderer + codegen
+
+> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development + cross-language-porting. Java (`server/java/render/.../recover/` + `.../prompt/`) is the reference; C# (`server/csharp/MetaObjects.Render/{Recover,Prompt}/` + `MetaObjects.Codegen/Generators/`) and TS are fresh native ports to mirror. The conformance corpus is the ORACLE (`fixtures/recover-conformance/`, `fixtures/conformance/template-output-{xml,json}-simple/`). Reimplement idiomatically (Tier-2); never change Tier-1 (classification, vocabulary, rendered output, wire format). Never edit a fixture to match the port — escalate.
+
+**Goal:** Bring FR-010 to the Python port — the LAST of the five ports. Python ships FR-006 + render + verify + codegen + conformance, but FR-010's recover engine + output-format renderer don't exist yet. Python natively reimplements both under `metaobjects/render/`, then adds the metamodel attrs + serializer fix + codegen. After this, FR-010 ships in all 5 ports.
+
+**Three phases (each a separate merge):**
+1. **Recover engine + dirty-input conformance** — `metaobjects/render/recover/` reimplemented from Java/C#; a recover-conformance pytest over `fixtures/recover-conformance/` is the gate. (≈ C# Phase 1.)
+2. **Output-format renderer** — `metaobjects/render/prompt/` (3 comment-free styles × {json,xml}); byte-identical to Java/Kotlin/C#/TS, reusing the `escapers` registry. (≈ C# Phase 2.)
+3. **Metamodel attrs + serializer fix + codegen + clean fixtures** — register `@promptStyle`/`@example`/`@instruction`/`@enumAlias`/`@enumDoc`; fix `serializer_json` to sort `properties`-attr map keys (NOT `filter`); extend the output-parser codegen with a tolerant `recover_<name>()`; add an output-prompt generator. (≈ C# Phase 3.)
+
+**Tech stack:** Python 3.11+, `metaobjects` package at `server/python/`, pytest, mypy --strict, ruff. Pydantic is the FR-006 strict tier; recover is the new forgiving tier (no Pydantic). Reuse the `metaobjects.render.escapers` registry for the renderer. Tests under `server/python/tests/`.
+
+## Tier decisions (cross-language-porting)
+- **Tier 1 (INVARIANT — must match the corpus):** per-field `FieldRecovery` classification (`RECOVERED|DEFAULTED|LOST_OPTIONAL|LOST_REQUIRED|MALFORMED`) + `empty` flag + canonical normalized value (numeric ±1e-9); pipeline behavior (strip/locate/forgiving-read/malformed-vs-absent/coerce); the rendered fragment (comment-free, valid, 3 styles); `@promptStyle`/`@enumAlias`/`@enumDoc` vocabulary; MALFORMED/TRUNCATED sentinel effect; canonical serialization (`properties` keys sorted, `filter` order preserved).
+- **Tier 2 (idiomatic Python):** module layout under `recover/`; `recover(text, schema, opts=None)` free function; enums via `enum.Enum`/`StrEnum` with SCREAMING_SNAKE values matching the corpus; MALFORMED/TRUNCATED as unique sentinel objects; the generated `recover_<name>()` returns `RecoveryResult[<Name>Recovered]` where `<Name>Recovered` is an emitted dataclass with every field `Optional[T]` (parallel to C#/Kotlin/TS nullable mirror — Pydantic payloads can't hold None for a required field). `as_int`/`as_long` both return `Optional[int]` (Python has one int type).
+- **Tier 3 (free):** internal mechanism, the conformance runner plumbing, the import-and-call proof.
+- **Bounded deferral:** nested-object recover/prompt mapping; array-of-enum (parity with the other ports — though TS implemented engine-level nested; Python may either way as long as the corpus + codegen defer).
+
+---
+
+## PHASE 1 — recover engine + dirty-input conformance (detailed)
+
+Port the engine to `metaobjects/render/recover/` (Python modules), then prove against the corpus. **The corpus is the oracle: when a case is red, ask "should it match?" before "how do I make it match"; never edit a fixture.**
+
+### P1-1: data model (`recover/types.py`)
+Port `Format`, `FieldKind`, `FieldRecovery`, `Tolerance`, `Coercion`, `FieldSpec` (+ factories `scalar`/`enum_field`/`range_`/`object_`), `RecoverSchema`, `RecoverOptions` (+ `defaults()`), `RecoverOutcome`, `RecoveryResult`, `RecoveryReport` (mutable: `set`/`add_coercion`/`mark_empty`/`states`/`coercions`/`lost_required`/`malformed`/`has_lost_required`/`is_empty`). Enums with SCREAMING_SNAKE values matching the corpus. **Test:** mirror Java `ModelTest`/`ReportTest`.
+
+### P1-2: `strip` (stage 1) — fence removal (regex, DOTALL). Mirror `StripTest`.
+### P1-3: `locate` (stages 2–3) — `locate_json` balanced-brace (string-aware) + `locate_xml`; first-closed-else-first-open; no-throw-on-`</x>`. Mirror `LocateTest`.
+### P1-4: `json_forgiving_reader` — tolerant JSON → dict; trailing commas, single/unquoted, **TRUNCATED sentinel**, no-hang on `{"xs":[}`. Port the FIXED version. Mirror `JsonForgivingReaderTest`.
+### P1-5: `xml_forgiving_reader` — tolerant XML → dict; unclosed-tag recovery, attr-quote variants, case-folding, no-throw on close-tag-start. Mirror `XmlForgivingReaderTest`.
+### P1-6: `coerce` (stage 7) — enum alias/vocab (runtime-alias-wins), numeric range clamp, **non-finite → MALFORMED**, boolean forms, on_field hook, normalizers. **Python numeric note:** use `float(...)`/`int(...)` with `math.isfinite`; reject `0x`/`0b`/`0o` radix prefixes that `int(s,0)` would accept but Java/C# reject (cross-port parity — both C# and TS added this guard); empty-string guard. `MALFORMED` = a unique sentinel object. Mirror `CoerceTest`.
+### P1-7: `recover` entry + `recover_map` + classify — assemble strip→locate→read→extract→classify→coerce; array partial-MALFORMED keeps elements; object-given-scalar → MALFORMED; TRUNCATED→MALFORMED; empty-guard. `recover_map` accessors `as_string/as_int/as_long/as_double/as_bool/as_string_list` None-safe. Mirror `RecoverTest`.
+### P1-8: dirty-input conformance runner
+`tests/conformance/test_recover_conformance.py` (parametrized over `fixtures/recover-conformance/`): parse `schema.json` (map UPPER enum strings), read `input.txt`, run `recover`, assert `expected.json`'s `empty` + `states` (EXHAUSTIVE key-set) + `data` (floats ±1e-9). All 10 cases.
+### P1-9: review + simplify + merge. `pytest` + `mypy --strict` + `ruff` green; KNOWN_GAPS note. Merge phase 1.
+
+---
+
+## PHASE 2 — output-format renderer (outline; detailed when phase 1 lands)
+Port `metaobjects/render/prompt/{prompt_style,prompt_field,output_format_spec,prompt_overrides,output_format_renderer}.py` — 3 comment-free styles × {json,xml}, reusing `escapers`. Byte-identical fragment (verbatim prose: "Fill in each field as described below:", "Respond exactly like this:", "    one of ", "      VAL = doc", "    e.g. "). Numeric-vs-quoted uses `math.isfinite` + the radix guard (NaN/Infinity/0x.. → quoted). Tests mirror the Java renderer tests (comment-free, valid JSON via `json.loads`, NaN-quoted + finite-unquoted).
+
+## PHASE 3 — metamodel attrs + serializer fix + codegen (outline)
+1. **Attrs:** `@promptStyle` (closed enum on template.output via a `_TEMPLATE_OUTPUT_ATTRS` AttrSchema with `allowed_values`, like `@format`); `@example`/`@instruction` (string, common field attrs); `@enumAlias`/`@enumDoc` (`ATTR_SUBTYPE_PROPERTIES`). Add constants.
+2. **Serializer fix:** in `serializer_json._normalize` (or the attr-value emit), sort `properties`-attr map keys ordinally while preserving `filter` order — value-shape heuristic (all-scalar values → sort; any dict/list value → preserve), same as C#/TS. Confirm filter desugar runs before serialization. Closes `template-output-{json,xml}-simple` (red in Python today).
+3. **Codegen:** extend the output-parser generator to additionally emit a `<Name>Recovered` dataclass (all `Optional`) + `recover_<name>(text, opts=None) -> RecoveryResult[<Name>Recovered]` from a baked RecoverSchema (a recover-schema emitter), for json/xml outputs; a new output-prompt generator emitting `render_<name>_format(overrides=None)` backed by the output-format renderer + an output-format-spec emitter. Reuse `type_map.py`.
+4. **Proof:** Python's advantage — `importlib` the emitted module in a test and CALL `recover_<name>()` on dirty input (@enumAlias fold) + `render_<name>_format()` (guide fragment). Plus mypy on emitted source if feasible.
+5. Clean `template-output-{json,xml}-simple` fixtures conformance (Python green).
+
+---
+
+## Self-review (phase 1)
+- All recover modules ported; corpus 10/10 green; classification + canonical-value asserted; FIXED behaviors carried (no-hang JSON, XML no-throw, non-finite MALFORMED, TRUNCATED→MALFORMED, radix-prefix guard, array partial); never-throws; nested deferred.
+- Verify-against-live-code: the Python attr-read API, the pytest parametrize idiom, the `escapers` registry (phase 2), the conformance fixture-walk + canonical-serialize compare (phase 3), the importlib proof.

server/python/src/metaobjects/render/__init__.py

@@ -1,5 +1,20 @@
-"""Render-tier engine (FR-004): the build-time template drift-check ``verify``."""
+"""Render-tier engine: the build-time template drift-check ``verify`` (FR-004) and
+the FR-010 tolerant ``recover`` parser."""
 
+from metaobjects.render.recover import (
+    FieldKind,
+    FieldRecovery,
+    FieldSpec,
+    Format,
+    RecoverOptions,
+    RecoverOutcome,
+    RecoverSchema,
+    RecoveryReport,
+    RecoveryResult,
+    Tolerance,
+    recover,
+    recover_map,
+)
 from metaobjects.render.verify import (
     ERR_OUTPUT_TAG_MISSING,
     ERR_PARTIAL_UNRESOLVED,
@@ -17,9 +32,21 @@
     "ERR_PARTIAL_UNRESOLVED",
     "ERR_REQUIRED_SLOT_UNUSED",
     "ERR_VAR_NOT_ON_PAYLOAD",
+    "FieldKind",
+    "FieldRecovery",
+    "FieldSpec",
+    "Format",
     "InMemoryProvider",
     "PayloadField",
     "Provider",
+    "RecoverOptions",
+    "RecoverOutcome",
+    "RecoverSchema",
+    "RecoveryReport",
+    "RecoveryResult",
+    "Tolerance",
     "VerifyError",
+    "recover",
+    "recover_map",
     "verify",
 ]

server/python/src/metaobjects/render/recover/KNOWN_GAPS.md

@@ -0,0 +1,53 @@
+# FR-010 Python recover engine — known gaps & intentional cross-port divergences
+
+Scope: the tolerant `recover` pipeline (`metaobjects/render/recover/`). The Java engine
+(`server/java/render/.../recover/`) is the cross-port reference; `fixtures/recover-conformance/`
+is the oracle. All 10 corpus cases pass.
+
+## Additive capability (TS + C# + Python, beyond Java/Kotlin codegen deferral)
+
+- **Nested-object recover is implemented** — a `FieldSpec` with a nested `RecoverSchema`
+  (the `object_(...)` factory) is descended into, classifying sub-fields with dotted paths
+  (`meta.n`). Mirrors the Java/C# `extract` OBJECT recursion. Dormant under the corpus (no
+  nested fixture) and under FR-010 codegen (which emits a scalar placeholder for parity), so
+  it changes no shared-corpus result.
+
+## Intentional, documented divergences (NOT bugs)
+
+The cross-port contract pins *classification + canonical value* (numbers ±1e-9), not byte-
+identical native parsing.
+
+- **Java-style numeric suffixes / hex-float literals.** Java's `Double.parseDouble` accepts
+  `"42d"`/`"42f"` and hex-float forms (→ RECOVERED); Python rejects them → **MALFORMED** (same
+  accepted divergence the C#/TS ports record). The load-bearing behavior — finite-only,
+  `NaN`/`±Infinity` → MALFORMED — is identical.
+
+- **ASCII-numeric gate (parity guard, not a divergence).** Python's `int()`/`float()` accept
+  underscore digit grouping (`"1_000"`, PEP 515), Unicode digits (`"１２３"`, `"٣"`), and radix
+  prefixes (`"0x10"`) that Java/C# reject. `coerce` gates numeric strings on an ASCII-only
+  pattern so these all classify **MALFORMED**, matching the strict cross-port behavior (C#).
+
+- **NBSP-trailing numerics.** `"1 "` → Python `str.strip()` removes the NBSP (Unicode
+  whitespace) leaving `"1"` → RECOVERED, as does C# `Trim()`; Java `trim()` keeps it →
+  MALFORMED. A pre-existing Java/C# disagreement on whitespace trimming; Python sides with C#.
+  Not corpus-exercised.
+
+## NIT (no cross-port consensus, no corpus case)
+
+- **Out-of-int64-range integers** (e.g. `"9999999999999999999"`): Python has arbitrary-precision
+  ints (RECOVERED with the exact value); Java saturates to `Long.MAX_VALUE`, C# wraps on the
+  cast. The three ports already disagree, so there is no single canonical value to match. No
+  corpus int approaches 2^63. Left as-is.
+
+## Python-specific defensive bound
+
+- **JSON nesting depth cap (`_MAX_DEPTH = 100`).** Python's recursion limit is far below the
+  JVM/.NET stack, so a pathologically deep input (hundreds of nested brackets) would raise
+  `RecursionError` — violating the never-throws contract. Past the cap the container is skipped
+  (string-aware, non-recursive) and recorded as garbled (→ MALFORMED). Far above any realistic
+  payload nesting; the other ports rely on their larger native stacks instead.
+
+## Bounded deferral (parity with all ports)
+
+- Array-of-enum is not specialized (a scalar array recovers via `as_string_list`).
+- `as_int`/`as_long` are identical (`Optional[int]`; Python has one int type), truncating toward zero.

server/python/src/metaobjects/render/recover/__init__.py

@@ -0,0 +1,56 @@
+"""FR-010 tolerant ``recover`` engine (Tier 2).
+
+A forgiving parser that takes dirty LLM output (fenced / preamble / prose-wrapped /
+truncated / trailing-comma JSON, unclosed-tag XML) and recovers it into a typed
+``dict``, classifying each field. It NEVER raises — the forgiving tier beyond
+FR-006's strict Pydantic parser.
+
+Public entry point: :func:`recover`.
+"""
+from __future__ import annotations
+
+from metaobjects.render.recover import recover_map
+from metaobjects.render.recover.coerce import MALFORMED
+from metaobjects.render.recover.json_forgiving_reader import (
+    TRUNCATED,
+    JsonForgivingReader,
+)
+from metaobjects.render.recover.recover import recover
+from metaobjects.render.recover.types import (
+    Coercion,
+    FieldKind,
+    FieldRecovery,
+    FieldSpec,
+    Format,
+    Normalizer,
+    OnField,
+    RecoverOptions,
+    RecoverOutcome,
+    RecoverSchema,
+    RecoveryReport,
+    RecoveryResult,
+    Tolerance,
+)
+from metaobjects.render.recover.xml_forgiving_reader import XmlForgivingReader
+
+__all__ = [
+    "MALFORMED",
+    "TRUNCATED",
+    "Coercion",
+    "FieldKind",
+    "FieldRecovery",
+    "FieldSpec",
+    "Format",
+    "JsonForgivingReader",
+    "Normalizer",
+    "OnField",
+    "RecoverOptions",
+    "RecoverOutcome",
+    "RecoverSchema",
+    "RecoveryReport",
+    "RecoveryResult",
+    "Tolerance",
+    "XmlForgivingReader",
+    "recover",
+    "recover_map",
+]