Skip to content

Commit 09ef727

Browse files
dmealingclaude
andcommitted
feat(python): payload_vo_generator + switch output_parser to import-style
Close the cross-port FR-006 gap for Python: a payload-VO generator now emits typed Pydantic v2 models per template (prompt + output + toolcall), mirroring what Kotlin / C# / TS already ship. The cross-port doc (`docs/features/templates-and-payloads.md`) was correctly flagging "Payload-VO codegen is not yet emitted — consumers pass a dict"; that's no longer true. Two changes: 1. **New `payload_vo_generator.py`** — class `PayloadVoGenerator` following the existing generator interface. Iterates ALL `MetaTemplate` (prompt + output + toolcall) — no subtype filter, matching the Kotlin reference (KotlinPayloadGenerator). Resolves `@payloadRef` to its `object.value` (skips entities and missing refs defensively). Emits one `<template_name_snake>_payload.py` per template containing the `<TemplateName>Payload` Pydantic v2 model. Origin.* resolution mirrors Kotlin: - `origin.passthrough @from "Entity.field"` → source field's Python type - `origin.aggregate @agg count` → `int`; `@agg avg` → `float`; `sum/min/max` → source `@of` field's type - `origin.collection @via "Parent.rel"` → walks the relationship to the target → recursive nested-model emission (deduped by FQN per-run) → `list[<TargetShortName>Payload>]`. Nested payloads land in the same file as the owning payload so the import is implicit. 2. **`output_parser_generator.py` switched from embedded to import-style** per its own docstring plan. The emitted module now does `from .<template_name>_payload import <TemplateName>Payload` and the parser becomes a thin `parse_<template_name>(text: str) -> <TemplateName>Payload` wrapping `<TemplateName>Payload.model_validate_json(text)`. Embedded inline model definitions removed. Module docstring updated to reflect the new contract (kept the throw-only design note — Python-ecosystem norm per Pydantic). Tests: codegen suite 72/72 green (added comprehensive coverage: prompt + output + toolcall emission, scalar field type mapping, all three origin subtypes, nested-collection dedupe across templates). Cross-port conformance suite 109/109 green — no regressions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 99d5e39 commit 09ef727

4 files changed

Lines changed: 1068 additions & 185 deletions

File tree

server/python/src/metaobjects/codegen/generators/output_parser_generator.py

Lines changed: 41 additions & 120 deletions
Original file line numberDiff line numberDiff line change
@@ -10,45 +10,31 @@
1010
Python's ecosystem (Pydantic, Instructor, FastAPI, LangChain structured-output)
1111
is throw-only and a dual surface would feel un-Pythonic.
1212
13-
Self-contained emit: the generated module declares an inline Pydantic v2 model
14-
derived from the payload-VO's field declarations (no separate ``payload_vo``
15-
generator required — TS embeds its Zod schema the same way). This deviates
16-
from the spec sketch's import-style example (``from .npc_response import
17-
NpcResponse``) because Python has no payload-VO generator yet; embedding the
18-
class makes the parser module usable today without a hand-written companion.
19-
If a future ``payload_vo_generator`` ships, this generator can be migrated to
20-
import from it (one-line file-name swap on the import).
21-
22-
Field subtype → Pydantic type mapping mirrors the TS Zod mapping and matches
23-
ADR-0010 §4 (Cross-language type mapping):
24-
25-
* ``field.string`` → ``str``
26-
* ``field.int / .long / .short / .byte`` → ``int``
27-
* ``field.double / .float`` → ``float``
28-
* ``field.boolean`` → ``bool``
29-
* ``field.currency`` → ``int`` (minor units; wire contract)
30-
* ``field.date / .time / .timestamp`` → ``datetime.{date,time,datetime}``
31-
* ``field.enum`` (with ``@values``) → ``Literal[...]`` of those members
32-
* Anything else → ``str`` (defensive fallback — surfaces as a warning)
33-
34-
Future per-field constraints (``@maxLength``, ``@required``-vs-optional
35-
nullability, ``@objectRef`` nesting) follow when their TS/C# counterparts
36-
ship them — kept minimal for FR6 v1 parity.
37-
"""
13+
Import-style emit: the parser module is a thin ``parse_<name>(text) -> Payload``
14+
wrapper that imports the Pydantic ``<TemplateName>Payload`` model from the
15+
sibling ``<template_name_snake>_payload.py`` (emitted by ``payload_vo_generator``).
16+
This matches the cross-port story where a single payload-VO class is reused by
17+
both prompt rendering and output parsing — TS / C# / Kotlin all do the same.
18+
19+
The generator emits an empty file list when ``payload_vo_generator`` would have
20+
emitted nothing for the same template (defensive parity with the loader's
21+
``@payloadRef`` validation pass)."""
3822
from __future__ import annotations
3923

4024
from collections.abc import Callable
41-
from typing import Any
4225

4326
from metaobjects.codegen.constants import generated_header
4427
from metaobjects.codegen.format import ruff_format
4528
from metaobjects.codegen.generator import EmittedFile, GenContext, Generator
46-
from metaobjects.meta.core.field import field_constants as fc
47-
from metaobjects.meta.core.field.meta_field import MetaField
29+
from metaobjects.codegen.generators.payload_vo_generator import (
30+
payload_class_name,
31+
payload_module_name,
32+
resolve_payload_vo,
33+
)
4834
from metaobjects.meta.core.object.meta_object import MetaObject
4935
from metaobjects.meta.meta_data import MetaData
5036
from metaobjects.meta.template import template_constants as tc
51-
from metaobjects.shared.base_types import TYPE_OBJECT, TYPE_TEMPLATE
37+
from metaobjects.shared.base_types import TYPE_TEMPLATE
5238

5339

5440
_GENERATOR_NAME = "output-parser-generator"
@@ -66,116 +52,51 @@ def _snake_case(name: str) -> str:
6652
return "".join(out)
6753

6854

69-
# Field-subtype → (Pydantic-annotation, required-import) tuple.
70-
# Empty string in the second slot means "no extra import beyond Pydantic".
71-
_FIELD_TYPE_MAP: dict[str, tuple[str, str]] = {
72-
fc.FIELD_SUBTYPE_STRING: ("str", ""),
73-
fc.FIELD_SUBTYPE_INT: ("int", ""),
74-
fc.FIELD_SUBTYPE_LONG: ("int", ""),
75-
fc.FIELD_SUBTYPE_DOUBLE: ("float", ""),
76-
fc.FIELD_SUBTYPE_FLOAT: ("float", ""),
77-
fc.FIELD_SUBTYPE_BOOLEAN: ("bool", ""),
78-
fc.FIELD_SUBTYPE_CURRENCY: ("int", ""),
79-
fc.FIELD_SUBTYPE_DATE: ("date", "from datetime import date"),
80-
fc.FIELD_SUBTYPE_TIME: ("time", "from datetime import time"),
81-
fc.FIELD_SUBTYPE_TIMESTAMP: ("datetime", "from datetime import datetime"),
82-
}
83-
84-
85-
def _python_type_for(field: MetaField) -> tuple[str, set[str]]:
86-
"""Return (annotation, set-of-import-lines) for a payload field."""
87-
sub = field.sub_type
88-
if sub in _FIELD_TYPE_MAP:
89-
annotation, import_line = _FIELD_TYPE_MAP[sub]
90-
return annotation, ({import_line} if import_line else set())
91-
if sub == fc.FIELD_SUBTYPE_ENUM:
92-
values: Any = field.attr(fc.FIELD_ATTR_VALUES)
93-
if isinstance(values, (list, tuple)) and values:
94-
members = ", ".join(repr(str(v)) for v in values)
95-
return f"Literal[{members}]", {"from typing import Literal"}
96-
return "str", set()
97-
return "str", set()
98-
99-
100-
def _find_payload_vo(root: MetaData, payload_ref: str) -> MetaObject | None:
101-
"""Locate the ``object.value`` matching ``payload_ref`` at root level.
102-
103-
FR-004's R2 invariant (the loader's ``_validate_templates`` pass) already
104-
asserts the resolved object is ``object.value``; we just locate it by
105-
short-name and return it. Loader errors short-circuit before codegen, so
106-
a missing ref here is defensive only."""
107-
for child in root.own_children():
108-
if child.type == TYPE_OBJECT and child.name == payload_ref:
109-
return child if isinstance(child, MetaObject) else None
110-
return None
111-
112-
11355
def render_output_parser(template: MetaData, root: MetaData) -> str | None:
11456
"""Render one parser module for a ``template.output`` node.
11557
58+
The emitted module imports ``<TemplateName>Payload`` from the sibling
59+
payload module (emitted by ``payload_vo_generator``) and exposes a
60+
throw-only ``parse_<name>(text)`` entry point.
61+
11662
Returns ``None`` when the ``@payloadRef`` can't be resolved (defensive;
11763
the loader's template-validation pass would normally catch this first)."""
11864
payload_ref = template.attr(tc.TEMPLATE_ATTR_PAYLOAD_REF)
11965
if not isinstance(payload_ref, str) or not payload_ref:
12066
return None
121-
payload = _find_payload_vo(root, payload_ref)
67+
payload = resolve_payload_vo(root, payload_ref)
12268
if payload is None:
12369
return None
12470

12571
template_name = template.name
126-
data_class = f"{template_name}Data"
72+
payload_class = payload_class_name(template_name) # <Name>Payload
73+
payload_module = payload_module_name(template_name) # <name>_payload
12774
parse_fn = f"parse_{_snake_case(template_name)}"
12875

129-
field_lines: list[str] = []
130-
extra_imports: set[str] = set()
131-
for field in payload.fields():
132-
if not isinstance(field, MetaField):
133-
continue
134-
annotation, imports = _python_type_for(field)
135-
extra_imports.update(imports)
136-
field_lines.append(f" {field.name}: {annotation}")
137-
13876
fqn = (
13977
f"{payload.package}::{template_name}"
14078
if payload.package
14179
else template_name
14280
)
143-
lines: list[str] = []
144-
lines.append(generated_header(template_name, fqn))
145-
lines.append("from __future__ import annotations\n")
146-
for imp in sorted(extra_imports):
147-
lines.append(imp)
148-
if extra_imports:
149-
lines.append("")
150-
lines.append("from pydantic import BaseModel")
151-
lines.append("")
152-
lines.append("")
153-
lines.append(f"class {data_class}(BaseModel):")
154-
lines.append(
155-
f' """Payload schema for the ``{template_name}`` template.output.'
156-
)
157-
lines.append("")
158-
lines.append(f" Field shape derived from the ``{payload.name}`` object.value.")
159-
lines.append(' """')
160-
if field_lines:
161-
lines.extend(field_lines)
162-
else:
163-
lines.append(" pass")
164-
lines.append("")
165-
lines.append("")
166-
lines.append(f"def {parse_fn}(text: str) -> {data_class}:")
167-
lines.append(f' """Parse an LLM response into a typed ``{data_class}``.')
168-
lines.append("")
169-
lines.append(" Raises:")
170-
lines.append(
171-
" pydantic.ValidationError: when the input does not match the schema."
172-
)
173-
lines.append(' """')
174-
lines.append(f" return {data_class}.model_validate_json(text)")
175-
lines.append("")
176-
lines.append("")
177-
lines.append(f'__all__ = ["{data_class}", "{parse_fn}"]')
178-
lines.append("")
81+
82+
lines: list[str] = [
83+
generated_header(template_name, fqn),
84+
"from __future__ import annotations\n",
85+
f"from .{payload_module} import {payload_class}",
86+
"",
87+
"",
88+
f"def {parse_fn}(text: str) -> {payload_class}:",
89+
f' """Parse an LLM response into a typed ``{payload_class}``.',
90+
"",
91+
" Raises:",
92+
" pydantic.ValidationError: when the input does not match the schema.",
93+
' """',
94+
f" return {payload_class}.model_validate_json(text)",
95+
"",
96+
"",
97+
f'__all__ = ["{parse_fn}"]',
98+
"",
99+
]
179100

180101
return "\n".join(lines)
181102

0 commit comments

Comments
 (0)