-
Notifications
You must be signed in to change notification settings - Fork 10
feat(config): add core component config engine #202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
maxxgx
wants to merge
2
commits into
main
Choose a base branch
from
max/comp-config-1-core
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,96 +1,71 @@ | ||
| # Config Schema Reference | ||
|
|
||
| > **Preview:** The config system is a planned API. The schemas below document the target config design. | ||
| ## ComponentConfig | ||
|
|
||
| Config files use `class_path` and `init_args` to describe explicit component construction. | ||
|
|
||
| ## ComponentSpec | ||
|
|
||
| Direct class mode: | ||
| `physicalai.config.ComponentConfig` uses the jsonargparse-compatible | ||
| `class_path` + `init_args` shape. | ||
|
|
||
| ```yaml | ||
| class_path: package.module.ClassName | ||
| init_args: | ||
| key: value | ||
| ``` | ||
|
|
||
| Registry mode: | ||
|
|
||
| ```yaml | ||
| type: registered_name | ||
| key: value | ||
| ``` | ||
| | Field | Required | Type | Description | | ||
| | ------------ | -------- | ------ | ------------------------------------------ | | ||
| | `class_path` | yes | string | Non-empty import path resolving to a class | | ||
| | `init_args` | no | object | String-keyed constructor arguments | | ||
|
|
||
| The `ComponentSpec` fields are listed below. | ||
| Omitted `init_args` means an empty mapping. No other top-level keys are | ||
| accepted. | ||
|
|
||
| | Field | Type | Description | | ||
| | ------------ | ------ | --------------------------------------- | | ||
| | `class_path` | string | Fully qualified import path | | ||
| | `init_args` | object | Constructor keyword arguments | | ||
| | `type` | string | Registered short name | | ||
| | extra fields | any | Flat constructor args for registry mode | | ||
| Values may be `null`, booleans, integers, finite floats, strings, lists, or | ||
| string-keyed mappings. Nested components use another `ComponentConfig`. | ||
| Every mapping containing `class_path` is reserved as a nested component and | ||
| must contain only `class_path` and optional `init_args`. Nesting through | ||
| components, lists, and mappings is limited to 10 levels. | ||
|
|
||
| The core rules are straightforward. | ||
| Export converts `Path` to its unchanged string form, `Enum` to its JSON-safe | ||
| value, and tuples to lists. `instantiate()` accepts the JSON representation, | ||
| not those original Python objects. | ||
|
|
||
| - A component spec must include either `class_path` or `type`. | ||
| - If both fields are present, `class_path` takes precedence. | ||
| - Nested component specs are instantiated recursively. | ||
| ## Runtime Documents | ||
|
|
||
| ## RuntimeConfig | ||
| A CLI runtime document places constructor arguments under `runtime:` and may | ||
| include method arguments under `run:`. | ||
|
|
||
| ```yaml | ||
| runtime: | ||
| robot: | ||
| class_path: physicalai.robot.so101.SO101 | ||
| class_path: physicalai.robot.SO101 | ||
| init_args: | ||
| port: /dev/ttyACM0 | ||
| calibration: ./calibration.json | ||
| action_source: | ||
| class_path: physicalai.runtime.PolicySource | ||
| class_path: physicalai.runtime.TeleopSource | ||
| init_args: | ||
| model: | ||
| class_path: physicalai.inference.InferenceModel | ||
| leader: | ||
| class_path: physicalai.robot.SharedRobot | ||
| init_args: | ||
| export_dir: ./exports/act_policy | ||
| execution: | ||
| class_path: physicalai.runtime.SyncExecution | ||
| name: leader-arm | ||
| fps: 30 | ||
| run: | ||
| duration_s: 60 | ||
| ``` | ||
|
|
||
| The most common runtime fields are listed below. | ||
|
|
||
| | Field | Type | Description | | ||
| | ----------------------- | --------------- | -------------------------------------------------------------------- | | ||
| | `runtime.robot` | `ComponentSpec` | Robot implementation | | ||
| | `runtime.action_source` | `ComponentSpec` | Action source — always explicit, e.g. `PolicySource`, `TeleopSource` | | ||
| | `runtime.fps` | number | Control loop frequency | | ||
| | `runtime.cameras` | mapping | Optional camera components | | ||
| | `runtime.callbacks` | list | Optional runtime callbacks | | ||
|
|
||
| `action_source.init_args` depends on the chosen class — `PolicySource` takes `model` and `execution` (and optionally `task`, `action_queue`); `TeleopSource` takes `leader`. | ||
|
|
||
| ## InferenceConfig | ||
|
|
||
| ```yaml | ||
| model: | ||
| class_path: physicalai.inference.InferenceModel | ||
| init_args: | ||
| export_dir: ./exports/act_policy | ||
| backend: openvino | ||
| device: CPU | ||
| ``` | ||
| `physicalai run --config` and `RobotRuntime.from_config()` also accept a bare | ||
| exported `RobotRuntime` ComponentConfig whose top-level `class_path` is | ||
| `physicalai.runtime.RobotRuntime`. | ||
|
|
||
| The most common inference fields are listed below. | ||
| ## Manifest ComponentSpec | ||
|
|
||
| | Field | Type | Description | | ||
| | ---------------------------- | --------------- | -------------------------- | | ||
| | `model` | `ComponentSpec` | Inference model component | | ||
| | `model.init_args.export_dir` | string | Exported package directory | | ||
| | `model.init_args.backend` | string | Backend name or `auto` | | ||
| | `model.init_args.device` | string | Backend device or `auto` | | ||
| Inference manifests retain their separate `ComponentSpec` compatibility | ||
| model. It supports registry `type` aliases, artifact resolution, and existing | ||
| extra-field behavior. It is not the strict captured `ComponentConfig` schema; | ||
| manifest unification is a separate design decision. | ||
|
|
||
| ## Config vs Manifest | ||
| ## Security | ||
|
|
||
| | Schema | Use | | ||
| | --------------- | -------------------------------------------------------- | | ||
| | Workflow config | A workflow config describes a workflow before execution. | | ||
| | Manifest | A manifest describes an exported package after export. | | ||
| `class_path` is executable local configuration. `instantiate()` is for | ||
| trusted application and user-authored configs only, never metadata or control | ||
| messages received from peers. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # Copyright (C) 2026 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Captured component configuration: export live components and instantiate configs. | ||
|
|
||
| Trusted local application and parent→child startup configs only. Never pass | ||
| network metadata or untrusted peer payloads to :func:`instantiate`. | ||
|
|
||
| Opt-in path: | ||
|
|
||
| - ``@export_config`` / ``@export_config(class_path=...)`` — remember | ||
| caller-supplied constructor args for :func:`to_config`. | ||
| - ``@export_config(..., scalar_var_kwargs=True)`` — seal flattened | ||
| ``**kwargs`` to JSON scalars (non-scalars fail at :func:`to_config`). | ||
| - Domain ctor args may implement :meth:`~ConfigValue.to_config_value` to | ||
| return a JSON-compatible fragment (re-normalized). | ||
|
|
||
| Transport and other callers import public names from here (no private | ||
| ``physicalai.config._*`` imports). Studio needs only | ||
| :func:`is_config_exportable` and :func:`to_config`, then domain | ||
| ``from_config`` helpers (for example ``SharedRobot.from_config``). | ||
| """ | ||
|
|
||
| from ._envelope import ( | ||
| normalize_component_config, | ||
| validate_envelope, | ||
| ) | ||
| from ._errors import ComponentConfigError, ComponentImportError | ||
| from ._export import ( | ||
| export_config, | ||
| is_config_exportable, | ||
| resolve_public_class_path, | ||
| to_config, | ||
| ) | ||
| from ._instantiate import instantiate | ||
| from ._normalize import validate_component_config | ||
| from ._types import ComponentConfig, ConfigValue, JsonScalar, JsonValue | ||
| from ._yaml import load_yaml, save_yaml, to_yaml | ||
| from .importing import import_dotted_path | ||
|
|
||
| __all__ = [ | ||
| "ComponentConfig", | ||
| "ComponentConfigError", | ||
| "ComponentImportError", | ||
| "ConfigValue", | ||
| "JsonScalar", | ||
| "JsonValue", | ||
| "export_config", | ||
| "import_dotted_path", | ||
| "instantiate", | ||
| "is_config_exportable", | ||
| "load_yaml", | ||
| "normalize_component_config", | ||
| "resolve_public_class_path", | ||
| "save_yaml", | ||
| "to_config", | ||
| "to_yaml", | ||
| "validate_component_config", | ||
| "validate_envelope", | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| # Copyright (C) 2026 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Shared validation for transport construction envelopes. | ||
|
|
||
| Robot-owner and camera-publisher stdin envelopes carry one nested | ||
| :class:`ComponentConfig` plus transport-only keys. These helpers keep the | ||
| schema-positive validation in one place; transports supply their key names | ||
| and allowlists. | ||
|
|
||
| Nothing here imports a ``class_path``. Envelopes are built in the subscriber | ||
| process, which must stay free of the driver package — the import happens in | ||
| the process that calls :func:`~physicalai.config.instantiate`. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import json | ||
| from collections.abc import Mapping | ||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| from ._errors import ComponentConfigError | ||
| from ._normalize import validate_component_config | ||
|
|
||
| if TYPE_CHECKING: | ||
| from ._types import ComponentConfig | ||
|
|
||
|
|
||
| def validate_envelope( | ||
| data: Mapping[str, Any], | ||
| *, | ||
| component_key: str, | ||
| allowed_keys: frozenset[str], | ||
| envelope_name: str, | ||
| ) -> Mapping[str, object]: | ||
| """Validate a transport stdin envelope schema-positively. | ||
|
|
||
| Requires *component_key* with a valid ComponentConfig shape and allows | ||
| only *allowed_keys*. Unknown keys (including legacy flat shapes) raise a | ||
| clear schema error before any import or hardware access. | ||
|
|
||
| Args: | ||
| data: Full stdin envelope dict. | ||
| component_key: Envelope key holding the nested ComponentConfig | ||
| (for example ``"robot"`` or ``"camera"``). | ||
| allowed_keys: Complete allowlist of envelope keys. | ||
| envelope_name: Short envelope label for error messages | ||
| (for example ``"owner"`` or ``"publisher"``). | ||
|
|
||
| Returns: | ||
| The validated ComponentConfig mapping (see | ||
| :func:`normalize_component_config` for the JSON-serializability check). | ||
|
|
||
| Raises: | ||
| TypeError: If *data* or the component value is not a mapping. | ||
| ValueError: If the component key is missing or unknown keys are present. | ||
| """ | ||
| if not isinstance(data, Mapping): | ||
| msg = f"{envelope_name} config must be a mapping, got {type(data).__name__}" | ||
| raise TypeError(msg) | ||
|
|
||
| unknown = sorted(set(data) - allowed_keys) | ||
| if unknown: | ||
| msg = ( | ||
| f"unknown {envelope_name} config keys {unknown}; " | ||
| f"require {component_key!r} with class_path + init_args " | ||
| f"(allowed envelope keys: {sorted(allowed_keys)})" | ||
| ) | ||
| raise ValueError(msg) | ||
|
|
||
| if component_key not in data: | ||
| msg = f"{envelope_name} config missing required {component_key!r} ComponentConfig" | ||
| raise ValueError(msg) | ||
|
|
||
| component = data[component_key] | ||
| if not isinstance(component, Mapping): | ||
| msg = f"{envelope_name} {component_key!r} must be a mapping, got {type(component).__name__}" | ||
| raise TypeError(msg) | ||
|
|
||
| return validate_component_config(dict(component), path=component_key) | ||
|
|
||
|
|
||
| def normalize_component_config( | ||
| config: Mapping[str, object], | ||
| *, | ||
| component_key: str, | ||
| class_label: str, | ||
| json_hint: str = "", | ||
| ) -> ComponentConfig: | ||
| """Validate a ComponentConfig without importing its ``class_path``. | ||
|
|
||
| The ``class_path`` is trusted and kept exactly as written: envelopes are | ||
| built in the subscriber process, which must not load the driver package | ||
| (and often cannot — the vendor SDK is only installed where the hardware | ||
| lives). Import errors surface later, in the process that calls | ||
| :func:`~physicalai.config.instantiate`. | ||
|
|
||
| Args: | ||
| config: Candidate ``class_path`` + ``init_args`` mapping. | ||
| component_key: Path prefix for validation errors (``"robot"`` / ``"camera"``). | ||
| class_label: Argument label for ``class_path`` errors. | ||
| json_hint: Optional suffix appended to the JSON-serializability error. | ||
|
|
||
| Returns: | ||
| A validated config whose ``class_path`` is a dotted import path. | ||
|
|
||
| Raises: | ||
| ComponentConfigError: If *config* is not a mapping. | ||
| ValueError: If ``class_path`` is not a dotted path or ``init_args`` is | ||
| not JSON-serializable. | ||
| """ | ||
| if not isinstance(config, Mapping): | ||
| msg = f"{component_key} must be a ComponentConfig mapping, got {type(config).__name__}" | ||
| raise ComponentConfigError(msg) | ||
| validated = validate_component_config(dict(config), path=component_key) | ||
| class_path = validated["class_path"] | ||
| if not class_path.strip() or "." not in class_path: | ||
| msg = f"{class_label} must be a nonempty dotted path, got {class_path!r}" | ||
| raise ValueError(msg) | ||
| init_args = validated["init_args"] | ||
| try: | ||
| json.dumps({"class_path": class_path, "init_args": init_args}, allow_nan=False) | ||
| except (TypeError, ValueError) as exc: | ||
| msg = f"{component_key}.init_args must be JSON-serializable{json_hint}: {exc}" | ||
| raise ValueError(msg) from exc | ||
| return {"class_path": class_path, "init_args": dict(init_args)} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Copyright (C) 2026 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Errors for component configuration export and instantiation.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
|
|
||
| class ComponentConfigError(Exception): | ||
| """Raised when a component config cannot be exported, validated, or built. | ||
|
|
||
| Callers typically recover by correcting configuration. Subclasses mark | ||
| distinct failure phases when callers need to distinguish them. | ||
| """ | ||
|
|
||
|
|
||
| class ComponentImportError(ComponentConfigError): | ||
| """Raised when a ``class_path`` cannot be resolved to an importable class.""" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.