Skip to content

Commit 2fbe11f

Browse files
dmealingclaude
andcommitted
feat(py-render): FR-010 Phase 1 — tolerant recover engine + dirty-input conformance (Python port)
Native Python reimplementation of the FR-010 recover engine in metaobjects/render/recover/, the forgiving tier beyond FR-006's strict Pydantic parser. 8-stage pipeline (strip → locate → forgiving-read → malformed-vs-absent → coerce → classify), JSON + XML, never raises. Per-field FieldRecovery classification + canonical value pinned by the shared fixtures/recover-conformance/ corpus — all 10 cases green, matching Java/C#. Python idioms (Tier 2): enum.Enum (SCREAMING_SNAKE values for corpus parity), dataclasses, MALFORMED/TRUNCATED unique sentinels, as_int/as_long both Optional[int] (one int type) truncating toward zero, bool excluded from numeric helpers (Java instanceof Number parity). Nested-object recover implemented (like TS/C#). Carries the fixed edge cases (JSON no-hang, TRUNCATED→MALFORMED, XML no-throw, non-finite→MALFORMED, array partial, enum alias-fold). Pre-merge gates (review + simplifier): ASCII-numeric gate rejects Python-permissive syntax (underscore grouping, Unicode digits, radix prefixes) for cross-port parity (+regression); JSON depth cap (_MAX_DEPTH=100) keeps recover never-raising on pathologically deep input where Python's recursion limit would otherwise raise (+regression); simplifier modernized to match/case + X|None. KNOWN_GAPS records the divergences. 108 recover tests + render suite 179 green; mypy --strict + ruff clean on the recover package. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b90c413 commit 2fbe11f

21 files changed

Lines changed: 2078 additions & 1 deletion

server/python/src/metaobjects/render/__init__.py

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
1-
"""Render-tier engine (FR-004): the build-time template drift-check ``verify``."""
1+
"""Render-tier engine: the build-time template drift-check ``verify`` (FR-004) and
2+
the FR-010 tolerant ``recover`` parser."""
23

4+
from metaobjects.render.recover import (
5+
FieldKind,
6+
FieldRecovery,
7+
FieldSpec,
8+
Format,
9+
RecoverOptions,
10+
RecoverOutcome,
11+
RecoverSchema,
12+
RecoveryReport,
13+
RecoveryResult,
14+
Tolerance,
15+
recover,
16+
recover_map,
17+
)
318
from metaobjects.render.verify import (
419
ERR_OUTPUT_TAG_MISSING,
520
ERR_PARTIAL_UNRESOLVED,
@@ -17,9 +32,21 @@
1732
"ERR_PARTIAL_UNRESOLVED",
1833
"ERR_REQUIRED_SLOT_UNUSED",
1934
"ERR_VAR_NOT_ON_PAYLOAD",
35+
"FieldKind",
36+
"FieldRecovery",
37+
"FieldSpec",
38+
"Format",
2039
"InMemoryProvider",
2140
"PayloadField",
2241
"Provider",
42+
"RecoverOptions",
43+
"RecoverOutcome",
44+
"RecoverSchema",
45+
"RecoveryReport",
46+
"RecoveryResult",
47+
"Tolerance",
2348
"VerifyError",
49+
"recover",
50+
"recover_map",
2451
"verify",
2552
]
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# FR-010 Python recover engine — known gaps & intentional cross-port divergences
2+
3+
Scope: the tolerant `recover` pipeline (`metaobjects/render/recover/`). The Java engine
4+
(`server/java/render/.../recover/`) is the cross-port reference; `fixtures/recover-conformance/`
5+
is the oracle. All 10 corpus cases pass.
6+
7+
## Additive capability (TS + C# + Python, beyond Java/Kotlin codegen deferral)
8+
9+
- **Nested-object recover is implemented** — a `FieldSpec` with a nested `RecoverSchema`
10+
(the `object_(...)` factory) is descended into, classifying sub-fields with dotted paths
11+
(`meta.n`). Mirrors the Java/C# `extract` OBJECT recursion. Dormant under the corpus (no
12+
nested fixture) and under FR-010 codegen (which emits a scalar placeholder for parity), so
13+
it changes no shared-corpus result.
14+
15+
## Intentional, documented divergences (NOT bugs)
16+
17+
The cross-port contract pins *classification + canonical value* (numbers ±1e-9), not byte-
18+
identical native parsing.
19+
20+
- **Java-style numeric suffixes / hex-float literals.** Java's `Double.parseDouble` accepts
21+
`"42d"`/`"42f"` and hex-float forms (→ RECOVERED); Python rejects them → **MALFORMED** (same
22+
accepted divergence the C#/TS ports record). The load-bearing behavior — finite-only,
23+
`NaN`/`±Infinity` → MALFORMED — is identical.
24+
25+
- **ASCII-numeric gate (parity guard, not a divergence).** Python's `int()`/`float()` accept
26+
underscore digit grouping (`"1_000"`, PEP 515), Unicode digits (`"123"`, `"٣"`), and radix
27+
prefixes (`"0x10"`) that Java/C# reject. `coerce` gates numeric strings on an ASCII-only
28+
pattern so these all classify **MALFORMED**, matching the strict cross-port behavior (C#).
29+
30+
- **NBSP-trailing numerics.** `"1 "` → Python `str.strip()` removes the NBSP (Unicode
31+
whitespace) leaving `"1"` → RECOVERED, as does C# `Trim()`; Java `trim()` keeps it →
32+
MALFORMED. A pre-existing Java/C# disagreement on whitespace trimming; Python sides with C#.
33+
Not corpus-exercised.
34+
35+
## NIT (no cross-port consensus, no corpus case)
36+
37+
- **Out-of-int64-range integers** (e.g. `"9999999999999999999"`): Python has arbitrary-precision
38+
ints (RECOVERED with the exact value); Java saturates to `Long.MAX_VALUE`, C# wraps on the
39+
cast. The three ports already disagree, so there is no single canonical value to match. No
40+
corpus int approaches 2^63. Left as-is.
41+
42+
## Python-specific defensive bound
43+
44+
- **JSON nesting depth cap (`_MAX_DEPTH = 100`).** Python's recursion limit is far below the
45+
JVM/.NET stack, so a pathologically deep input (hundreds of nested brackets) would raise
46+
`RecursionError` — violating the never-throws contract. Past the cap the container is skipped
47+
(string-aware, non-recursive) and recorded as garbled (→ MALFORMED). Far above any realistic
48+
payload nesting; the other ports rely on their larger native stacks instead.
49+
50+
## Bounded deferral (parity with all ports)
51+
52+
- Array-of-enum is not specialized (a scalar array recovers via `as_string_list`).
53+
- `as_int`/`as_long` are identical (`Optional[int]`; Python has one int type), truncating toward zero.
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
"""FR-010 tolerant ``recover`` engine (Tier 2).
2+
3+
A forgiving parser that takes dirty LLM output (fenced / preamble / prose-wrapped /
4+
truncated / trailing-comma JSON, unclosed-tag XML) and recovers it into a typed
5+
``dict``, classifying each field. It NEVER raises — the forgiving tier beyond
6+
FR-006's strict Pydantic parser.
7+
8+
Public entry point: :func:`recover`.
9+
"""
10+
from __future__ import annotations
11+
12+
from metaobjects.render.recover import recover_map
13+
from metaobjects.render.recover.coerce import MALFORMED
14+
from metaobjects.render.recover.json_forgiving_reader import (
15+
TRUNCATED,
16+
JsonForgivingReader,
17+
)
18+
from metaobjects.render.recover.recover import recover
19+
from metaobjects.render.recover.types import (
20+
Coercion,
21+
FieldKind,
22+
FieldRecovery,
23+
FieldSpec,
24+
Format,
25+
Normalizer,
26+
OnField,
27+
RecoverOptions,
28+
RecoverOutcome,
29+
RecoverSchema,
30+
RecoveryReport,
31+
RecoveryResult,
32+
Tolerance,
33+
)
34+
from metaobjects.render.recover.xml_forgiving_reader import XmlForgivingReader
35+
36+
__all__ = [
37+
"MALFORMED",
38+
"TRUNCATED",
39+
"Coercion",
40+
"FieldKind",
41+
"FieldRecovery",
42+
"FieldSpec",
43+
"Format",
44+
"JsonForgivingReader",
45+
"Normalizer",
46+
"OnField",
47+
"RecoverOptions",
48+
"RecoverOutcome",
49+
"RecoverSchema",
50+
"RecoveryReport",
51+
"RecoveryResult",
52+
"Tolerance",
53+
"XmlForgivingReader",
54+
"recover",
55+
"recover_map",
56+
]
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
"""Stage 7: canonicalize a raw scalar string per its FieldSpec.
2+
3+
Returns the ``MALFORMED`` sentinel when the value is present but uncoercible.
4+
5+
Cross-port number rules (parity with C#/TS, an accepted divergence from Java):
6+
7+
- Non-finite (NaN / ±Infinity) → MALFORMED.
8+
- Radix-prefixed strings (``0x..`` / ``0b..`` / ``0o..``) are REJECTED. Python's
9+
``int(s, 0)`` would accept them but Java's ``Long.parseLong`` / C#'s
10+
``long.TryParse`` reject them — so the C# and TS ports added this guard; we match.
11+
- We do NOT replicate Java's ``Double.parseDouble`` suffix tolerance
12+
(``"42d"`` / hex-float) — documented accepted divergence (same as C#/TS).
13+
"""
14+
from __future__ import annotations
15+
16+
import math
17+
import re
18+
from typing import Final
19+
20+
from metaobjects.render.recover.types import (
21+
Coercion,
22+
FieldKind,
23+
FieldSpec,
24+
RecoverOptions,
25+
RecoveryReport,
26+
Tolerance,
27+
)
28+
29+
# Sentinel: the value was present but could not be coerced to the declared kind/vocabulary.
30+
MALFORMED: Final = object()
31+
32+
# A canonical ASCII numeric literal (int / decimal / scientific). Python's int()/float()
33+
# are far more permissive than Java/C# numeric parsing — they accept underscore digit
34+
# grouping ("1_000", PEP 515), Unicode digits ("123"), and radix prefixes ("0x10"). Gating
35+
# on this ASCII-only pattern rejects all of those → MALFORMED, matching the strict cross-port
36+
# behavior (C#'s TryParse). `[0-9]` (not `\d`) keeps it ASCII-only.
37+
_ASCII_NUMERIC = re.compile(r"^[+-]?(?:[0-9]+\.?[0-9]*|\.[0-9]+)(?:[eE][+-]?[0-9]+)?$")
38+
39+
40+
def value(
41+
raw: str | None,
42+
spec: FieldSpec,
43+
opts: RecoverOptions,
44+
field_path: str,
45+
report: RecoveryReport,
46+
) -> object:
47+
"""Canonicalize ``raw`` to the native type described by ``spec``, or MALFORMED."""
48+
if raw is None:
49+
return MALFORMED
50+
51+
# OnField hook takes priority.
52+
if opts.on_field is not None:
53+
hooked = opts.on_field(field_path, raw, spec)
54+
if hooked is not None:
55+
report.add_coercion(Coercion(field_path, raw, str(hooked), "onField"))
56+
return hooked
57+
58+
# Per-field runtime normalizer (bounded 20% surface). Keyed by path, then simple name.
59+
norm = opts.normalizers.get(field_path)
60+
if norm is None:
61+
norm = opts.normalizers.get(spec.name)
62+
if norm is not None:
63+
normalized = norm(raw)
64+
if normalized is not None:
65+
report.add_coercion(Coercion(field_path, raw, str(normalized), "normalizer"))
66+
return normalized
67+
68+
ci = opts.tolerance != Tolerance.STRICT
69+
match spec.kind:
70+
case FieldKind.ENUM:
71+
return _coerce_enum(raw, spec, opts, field_path, report, ci)
72+
case FieldKind.INT | FieldKind.LONG:
73+
return _coerce_int(raw, spec, field_path, report)
74+
case FieldKind.DOUBLE:
75+
return _coerce_double(raw, spec, field_path, report)
76+
case FieldKind.BOOLEAN:
77+
return _coerce_bool(raw, ci)
78+
case _:
79+
return raw
80+
81+
82+
def _coerce_enum(
83+
raw: str,
84+
spec: FieldSpec,
85+
opts: RecoverOptions,
86+
path: str,
87+
report: RecoveryReport,
88+
ci: bool,
89+
) -> object:
90+
if spec.enum_values is not None:
91+
for v in spec.enum_values:
92+
if v == raw:
93+
return v
94+
if ci and v.lower() == raw.lower():
95+
report.add_coercion(Coercion(path, raw, v, "case"))
96+
return v
97+
schema_target = None if spec.enum_alias is None else spec.enum_alias.get(raw)
98+
runtime_target = opts.aliases.get(raw)
99+
if runtime_target is not None:
100+
kind = (
101+
"runtime-alias-override"
102+
if schema_target is not None and schema_target != runtime_target
103+
else "alias"
104+
)
105+
report.add_coercion(Coercion(path, raw, runtime_target, kind))
106+
return runtime_target
107+
if schema_target is not None:
108+
report.add_coercion(Coercion(path, raw, schema_target, "alias"))
109+
return schema_target
110+
return MALFORMED
111+
112+
113+
def _coerce_int(raw: str, spec: FieldSpec, path: str, report: RecoveryReport) -> object:
114+
trimmed = raw.strip()
115+
if not _ASCII_NUMERIC.match(trimmed):
116+
return MALFORMED
117+
# Integer parse first (matches Java Long.parseLong / C# long.TryParse), then a
118+
# float fallback (matches Java's Double.parseDouble fallback).
119+
try:
120+
return _clamp(float(int(trimmed)), spec, path, report, as_long=True)
121+
except ValueError:
122+
pass
123+
try:
124+
d = float(trimmed)
125+
except ValueError:
126+
return MALFORMED
127+
return _clamp(d, spec, path, report, as_long=True)
128+
129+
130+
def _coerce_double(raw: str, spec: FieldSpec, path: str, report: RecoveryReport) -> object:
131+
trimmed = raw.strip()
132+
if not _ASCII_NUMERIC.match(trimmed):
133+
return MALFORMED
134+
try:
135+
d = float(trimmed)
136+
except ValueError:
137+
return MALFORMED
138+
return _clamp(d, spec, path, report, as_long=False)
139+
140+
141+
def _clamp(
142+
n: float, spec: FieldSpec, path: str, report: RecoveryReport, as_long: bool
143+
) -> object:
144+
# Non-finite (NaN, ±Infinity) → MALFORMED (cross-port classification parity).
145+
if not math.isfinite(n):
146+
return MALFORMED
147+
c = n
148+
if spec.min is not None and c < spec.min:
149+
c = spec.min
150+
if spec.max is not None and c > spec.max:
151+
c = spec.max
152+
if c != n:
153+
report.add_coercion(Coercion(path, _num_str(n), _num_str(c), "clamp"))
154+
# Truncate toward zero for integer kinds (math.trunc / int()).
155+
return math.trunc(c) if as_long else c
156+
157+
158+
def _num_str(n: float) -> str:
159+
# Render integral floats without a trailing ".0" to read like Java/C# longs in notes.
160+
if n == math.trunc(n) and math.isfinite(n):
161+
return str(int(n))
162+
return str(n)
163+
164+
165+
def _coerce_bool(raw: str, ci: bool) -> object:
166+
t = raw.strip().lower() if ci else raw.strip()
167+
if t in ("true", "yes", "1"):
168+
return True
169+
if t in ("false", "no", "0"):
170+
return False
171+
return MALFORMED

0 commit comments

Comments
 (0)