feat(cli,python): --provider hook to load a consumer provider; document custom providers (#158)

The metadata loaders already accept consumer providers in every language, but the
Python CLI's _load_root → from_directory convenience used core providers only, so
'metaobjects gen/verify/docs' could not load an app's custom subtype. Add a
repeatable '--provider module:symbol' that imports the consumer Provider (or a
list, or a zero-arg factory) and composes it on top of core_providers — parity
with TS metaobjects.config.ts providers. A provider carries a factory + optional
validator (code), so it must load as a native class, not JSON.

Docs: own-your-codegen.md gains a 'Custom types (custom providers)' section with
the per-port load mechanism — TS config.providers, Python --provider, JVM
ServiceLoader on the project classpath (META-INF/services); C# hook tracked as
future work.

Closes #158

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKaSE8U5cwN4ZGFwhJwxSY

Author: dmealing

docs/features/own-your-codegen.md

@@ -45,6 +45,31 @@ ADR-0015. Schema migrations are TypeScript-owned across all ports.)*
   the declarative template surface). This split follows each ecosystem's norms
   rather than forcing a single mechanism.
 
+## Custom types (custom providers)
+
+Beyond owning the *generators*, you can extend the *metamodel itself* — register your
+own type/subtype (a project-specific `view.*`, `field.*`, `validator.*`, …) through a
+**consumer provider**. The metadata **loaders already accept consumer providers in
+every language**: a runtime/library app that loads the metamodel plugs its provider in
+and it merges on top of the core set. The only per-port question is how the **CLI /
+build tool** hands that provider to the loader it already uses.
+
+A provider carries **code**, not just declarations — a `factory` (how to construct the
+node) plus an optional imperative validator. So the mechanism must load your **native
+provider class** in each language: a JSON file could express the declarative surface
+(types / attrs / child-rules) but not the factory or validator.
+
+| Port | How the CLI / build tool loads your provider |
+|---|---|
+| **TypeScript** | `metaobjects.config.ts` → `providers: [myProvider]`. Threaded into `meta gen`, `meta verify`, `meta docs`, **and** the offline `meta migrate` paths (baseline + generate). |
+| **Python** | `metaobjects gen \| verify \| docs --provider module:symbol` (repeatable). The symbol resolves to a `Provider` (or a list, or a zero-arg factory returning one) and is composed **on top of** the core providers — parity with TS `config.providers`. E.g. `metaobjects gen ./metaobjects --out ./gen --provider myapp.providers:view_provider`. |
+| **Java / Kotlin** | Put your compiled `MetaDataTypeProvider` on the **project classpath** with a `META-INF/services/com.metaobjects.registry.MetaDataTypeProvider` entry. `metaobjects-maven-plugin` builds the loader with the project classloader, so **Java ServiceLoader auto-discovers it** — no plugin config needed. |
+| **C#** | The loader accepts providers like every port; a first-class `dotnet meta` consumer-provider hook is tracked for a future release ([#158](https://github.com/metaobjectsdev/metaobjects/issues/158)). Today, extend the metamodel from an app that constructs the loader directly. |
+
+This split follows each ecosystem's norms — interpreted ports (TS / Python) name or
+import the provider module; compiled ports (JVM) discover it on the build classpath —
+the same "idiomatic per port" principle as generator ownership (ADR-0035 §3).
+
 ## Deprecated (removed at 1.0)
 
 Importing the built-in generators from `@metaobjectsdev/codegen-ts/generators`

server/python/src/metaobjects/cli.py

@@ -103,17 +103,97 @@ def _default_generators() -> list[Generator]:
     ]
 
 
+def _resolve_providers(specs: list[str] | None) -> tuple[list[object], list[str]]:
+    """Import consumer providers named as ``module:symbol`` (repeatable ``--provider``).
+
+    The symbol must be a metaobjects ``Provider`` (or a list of them, or a zero-arg
+    factory returning either). A provider carries CODE — factories and optional
+    validators — so it is loaded as a native Python object, not a JSON file (#158).
+    This is the Python CLI's idiomatic parity with TS ``metaobjects.config.ts``
+    ``providers``; the underlying loader already accepts providers.
+    """
+    if not specs:
+        return [], []
+    import importlib
+
+    from metaobjects.provider import Provider
+
+    providers: list[object] = []
+    errors: list[str] = []
+    for spec in specs:
+        module_name, sep, symbol = spec.partition(":")
+        if not sep or not module_name or not symbol:
+            errors.append(
+                f"--provider '{spec}' must be in 'module:symbol' form "
+                "(e.g. myapp.providers:my_provider)"
+            )
+            continue
+        try:
+            module = importlib.import_module(module_name)
+        except Exception as exc:  # ImportError + anything raised at import time
+            errors.append(f"--provider '{spec}': cannot import '{module_name}': {exc}")
+            continue
+        try:
+            obj = getattr(module, symbol)
+        except AttributeError:
+            errors.append(
+                f"--provider '{spec}': '{module_name}' has no attribute '{symbol}'"
+            )
+            continue
+        # A zero-arg factory is supported: call it to obtain the provider(s).
+        if callable(obj) and not isinstance(obj, Provider):
+            try:
+                obj = obj()
+            except Exception as exc:
+                errors.append(f"--provider '{spec}': factory '{symbol}()' raised: {exc}")
+                continue
+        for item in obj if isinstance(obj, list) else [obj]:
+            if isinstance(item, Provider):
+                providers.append(item)
+            else:
+                errors.append(
+                    f"--provider '{spec}': '{symbol}' resolved to "
+                    f"{type(item).__name__}, expected a metaobjects Provider"
+                )
+    return providers, errors
+
+
+def _providers_from_args(args: argparse.Namespace) -> tuple[list[object], bool]:
+    """Resolve ``--provider`` specs; print resolution errors. Returns (providers, ok)."""
+    providers, errors = _resolve_providers(getattr(args, "provider", None))
+    if errors:
+        print("error: invalid --provider:", file=sys.stderr)
+        for msg in errors:
+            print(f"  {msg}", file=sys.stderr)
+        return providers, False
+    return providers, True
+
+
 def _load_root(
-    metadata_dir: str, strict: bool = False
+    metadata_dir: str,
+    strict: bool = False,
+    providers: list[object] | None = None,
 ) -> tuple[MetaData | None, list[str]]:
     """Load metadata; return ``(root, error_messages)``. ``root`` is None on error.
 
     ``strict`` (ADR-0023, #96) — when True, an undeclared own ``@attr`` →
     ``ERR_UNKNOWN_ATTR``. Defaults False so ``gen`` / ``docs`` keep the legacy
     open-attr load; only ``verify`` opts in (strict-by-default with a ``--lax``
     escape).
+
+    ``providers`` (#158) — consumer providers (from ``--provider module:symbol``)
+    composed ON TOP of the core set, so a config-registered custom subtype resolves
+    through the standalone CLI just as it does when an app loads the loader directly.
+    Empty/None keeps the core-providers-only default (``from_directory`` convenience).
     """
-    result = MetaDataLoader.from_directory(metadata_dir, strict=strict)
+    if providers:
+        from metaobjects.core_types import core_providers
+
+        result = MetaDataLoader.from_directory(
+            metadata_dir, providers=[*core_providers, *providers], strict=strict
+        )
+    else:
+        result = MetaDataLoader.from_directory(metadata_dir, strict=strict)
     if result.errors:
         msgs = [f"{e.code}: {e.message}" for e in result.errors]
         return None, msgs
@@ -193,6 +273,7 @@ def _generate(
     entity_filter: list[str] | None = None,
     emit_package_init: bool = True,
     strict: bool = False,
+    providers: list[object] | None = None,
 ) -> tuple[list[str], list[str]]:
     """Run the generator suite into ``out_dir``.
 
@@ -208,7 +289,7 @@ def _generate(
     files are written. ``strict`` (ADR-0023, #96) is threaded to the load — the
     ``verify --codegen`` caller passes True; ``gen`` keeps the default False.
     """
-    root, errors = _load_root(metadata_dir, strict=strict)
+    root, errors = _load_root(metadata_dir, strict=strict, providers=providers)
     if root is None:
         return [], errors
     return _run_suite(root, out_dir, generators, entity_filter, emit_package_init), []
@@ -237,7 +318,10 @@ def _cmd_docs(args: argparse.Namespace) -> int:
         model_cross_href,
     )
 
-    root, errors = _load_root(args.metadata_dir)
+    providers, providers_ok = _providers_from_args(args)
+    if not providers_ok:
+        return 1
+    root, errors = _load_root(args.metadata_dir, providers=providers)
     if root is None:
         print("error: failed to load metadata:", file=sys.stderr)
         for msg in errors:
@@ -364,9 +448,12 @@ def _cmd_gen(args: argparse.Namespace) -> int:
         spec_gens = template_spec_to_generators(spec, provider)
 
     entities = _parse_entities(getattr(args, "entities", None))
+    providers, providers_ok = _providers_from_args(args)
+    if not providers_ok:
+        return 1
     # Load the metadata ONCE; both the default suite and the (optional)
     # --template-spec pass run against this single loaded root.
-    root, load_errors = _load_root(args.metadata_dir)
+    root, load_errors = _load_root(args.metadata_dir, providers=providers)
     if root is None:
         print("error: failed to load metadata:", file=sys.stderr)
         for msg in load_errors:
@@ -431,10 +518,13 @@ def _verify_codegen(args: argparse.Namespace) -> int:
     # ADR-0023 strict-by-default (#96): verify loads strict unless --lax is given,
     # so an undeclared/typo'd own @attr fails verify (matching Java's Maven goal).
     strict = not getattr(args, "lax", False)
+    providers, providers_ok = _providers_from_args(args)
+    if not providers_ok:
+        return 1
     with tempfile.TemporaryDirectory() as tmp:
         entities = _parse_entities(getattr(args, "entities", None))
         written, errors = _generate(
-            args.metadata_dir, tmp, None, entities, strict=strict
+            args.metadata_dir, tmp, None, entities, strict=strict, providers=providers
         )
         if errors:
             print("error: failed to load metadata:", file=sys.stderr)
@@ -508,7 +598,10 @@ def _verify_templates(args: argparse.Namespace) -> int:
     # ADR-0023 strict-by-default (#96) — same as --codegen: an undeclared @attr
     # fails verify unless --lax is passed.
     strict = not getattr(args, "lax", False)
-    root, errors = _load_root(args.metadata_dir, strict=strict)
+    providers, providers_ok = _providers_from_args(args)
+    if not providers_ok:
+        return 1
+    root, errors = _load_root(args.metadata_dir, strict=strict, providers=providers)
     if root is None:
         print("error: failed to load metadata:", file=sys.stderr)
         for msg in errors:
@@ -717,6 +810,16 @@ def _build_parser() -> argparse.ArgumentParser:
             "named entities are written. Omit to emit every entity."
         ),
     )
+    gen.add_argument(
+        "--provider",
+        action="append",
+        metavar="module:symbol",
+        help=(
+            "load a consumer metadata Provider as 'module:symbol' (repeatable) — the "
+            "Python parity of TS metaobjects.config.ts providers, so a custom subtype "
+            "resolves through the standalone CLI (#158)"
+        ),
+    )
     gen.set_defaults(func=_cmd_gen)
 
     docs = sub.add_parser(
@@ -749,6 +852,12 @@ def _build_parser() -> argparse.ArgumentParser:
         default=None,
         help="federate the model back-links to an absolute base URL (default: relative)",
     )
+    docs.add_argument(
+        "--provider",
+        action="append",
+        metavar="module:symbol",
+        help="load a consumer metadata Provider as 'module:symbol' (repeatable; #158)",
+    )
     docs.set_defaults(func=_cmd_docs)
 
     verify = sub.add_parser(
@@ -803,6 +912,12 @@ def _build_parser() -> argparse.ArgumentParser:
             "open-attr load."
         ),
     )
+    verify.add_argument(
+        "--provider",
+        action="append",
+        metavar="module:symbol",
+        help="load a consumer metadata Provider as 'module:symbol' (repeatable; #158)",
+    )
     verify.set_defaults(func=_cmd_verify)
 
     agent_docs = sub.add_parser(

server/python/tests/codegen/test_cli_providers.py

@@ -0,0 +1,146 @@
+"""#158 — the Python CLI ``--provider module:symbol`` hook.
+
+The metadata loaders already accept consumer providers in every language; the gap
+was purely at the CLI layer (``_load_root`` → ``from_directory`` used core
+providers only). These tests prove the standalone ``metaobjects gen``/``verify``
+can now load an app's custom subtype via ``--provider``, parity with the TS
+``metaobjects.config.ts`` ``providers`` path.
+"""
+from __future__ import annotations
+
+import json
+import sys
+from pathlib import Path
+
+from metaobjects.cli import main
+
+# A provider module the test writes to disk + imports. Registers the custom
+# ``validator.geocheck`` subtype (a validator, so it exercises provider-threading
+# without depending on how codegen maps a novel field's physical type).
+PROVIDER_MODULE = '''
+from metaobjects.provider import Provider
+from metaobjects.registry import TypeDefinition
+from metaobjects.meta.meta_data import MetaData
+
+geo_provider = Provider("test-geocheck", ("metaobjects-core-types",))
+geo_provider.add(TypeDefinition(
+    type="validator",
+    sub_type="geocheck",
+    factory=lambda t, s, n: MetaData(t, s, n),
+    description="A custom validator",
+))
+'''
+
+# Metadata whose ``name`` field carries the custom validator — resolves ONLY when
+# the provider is loaded.
+CUSTOM_META = {
+    "metadata.root": {
+        "package": "acme::geo",
+        "children": [
+            {
+                "object.entity": {
+                    "name": "Place",
+                    "children": [
+                        {"field.long": {"name": "id"}},
+                        {
+                            "field.string": {
+                                "name": "name",
+                                "children": [{"validator.geocheck": {"name": "chk"}}],
+                            }
+                        },
+                        {"source.rdb": {"name": "src", "@table": "places"}},
+                        {
+                            "identity.primary": {
+                                "name": "pk",
+                                "@fields": ["id"],
+                                "@generation": "increment",
+                            }
+                        },
+                    ],
+                }
+            }
+        ],
+    }
+}
+
+
+def _project(tmp_path: Path) -> str:
+    d = tmp_path / "meta"
+    d.mkdir()
+    (d / "meta.json").write_text(json.dumps(CUSTOM_META))
+    return str(d)
+
+
+def _install_provider_module(tmp_path: Path, name: str) -> None:
+    (tmp_path / f"{name}.py").write_text(PROVIDER_MODULE)
+    sys.path.insert(0, str(tmp_path))
+
+
+def test_gen_without_provider_fails_on_custom_subtype(tmp_path: Path) -> None:
+    """Baseline: the custom subtype is unknown without the provider."""
+    rc = main(["gen", _project(tmp_path), "--out", str(tmp_path / "out")])
+    assert rc != 0
+
+
+def test_gen_with_provider_loads_custom_subtype(tmp_path: Path) -> None:
+    """--provider composes the consumer provider on top of core → subtype resolves."""
+    meta_dir = _project(tmp_path)
+    _install_provider_module(tmp_path, "geo_prov_gen")
+    try:
+        rc = main(
+            [
+                "gen",
+                meta_dir,
+                "--out",
+                str(tmp_path / "out"),
+                "--provider",
+                "geo_prov_gen:geo_provider",
+            ]
+        )
+    finally:
+        sys.path.remove(str(tmp_path))
+        sys.modules.pop("geo_prov_gen", None)
+    assert rc == 0
+
+
+def test_verify_with_provider_loads_custom_subtype(tmp_path: Path) -> None:
+    """verify --codegen also threads --provider (reuses the gen code path)."""
+    meta_dir = _project(tmp_path)
+    _install_provider_module(tmp_path, "geo_prov_ver")
+    out = tmp_path / "out"
+    try:
+        # generate committed output first (with the provider), then verify no drift.
+        gen_rc = main(
+            ["gen", meta_dir, "--out", str(out), "--provider", "geo_prov_ver:geo_provider"]
+        )
+        assert gen_rc == 0
+        rc = main(
+            [
+                "verify",
+                meta_dir,
+                "--codegen",
+                "--out",
+                str(out),
+                "--provider",
+                "geo_prov_ver:geo_provider",
+            ]
+        )
+    finally:
+        sys.path.remove(str(tmp_path))
+        sys.modules.pop("geo_prov_ver", None)
+    assert rc == 0
+
+
+def test_bad_provider_spec_reports_error(tmp_path: Path) -> None:
+    """A malformed --provider spec fails cleanly (not 'module:symbol')."""
+    rc = main(
+        [
+            "gen",
+            _project(tmp_path),
+            "--out",
+            str(tmp_path / "out"),
+            "--provider",
+            "not-a-valid-spec",
+        ]
+    )
+    assert rc == 1