Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions docs/development/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,23 @@ These rules apply when writing, editing, or reviewing code under `src/physicalai
raise ValueError(f"Path escapes base directory: {user_path!r}")
```

4. No arbitrary `class_path` import from untrusted manifests or YAML. `instantiate_component` in `inference/component_factory.py` resolves `class_path` via `ComponentRegistry` and `importlib`. Only register trusted short names; treat manifest `class_path` values as untrusted unless the export directory is trusted. Prefer registered `type` names for built-ins.

5. Enforce component nesting limits. `_MAX_COMPONENT_DEPTH` in `component_factory.py` caps recursive manifest/YAML instantiation — do not raise or bypass without a security review.
4. No arbitrary `class_path` import from untrusted manifests, YAML, or peer
payloads. `instantiate_component` in `inference/component_factory.py`
resolves `class_path` via `ComponentRegistry` and `importlib`. Only register
trusted short names; treat manifest `class_path` values as untrusted unless
the export directory is trusted. Prefer registered `type` names for
built-ins. `physicalai.config.instantiate` is a separate trusted-local /
parent→child-only construction boundary: never pass robot/camera network
metadata, Zenoh payloads, shared-memory control requests, or other
untrusted peer data into it. Camera reconfigure requests may carry only
explicitly allowlisted scalar settings; the publisher must merge them into
its trusted startup recipe without accepting a peer-selected `class_path`.

5. Enforce component nesting limits. `_MAX_COMPONENT_DEPTH` in
`component_factory.py` caps recursive manifest/YAML instantiation;
`_MAX_CONFIG_DEPTH` in `physicalai.config` caps recursive
`to_config` / `instantiate` trees — do not raise or bypass either without a
security review.

6. Never use `pickle`, `eval()`, `exec()`, `joblib`, `dill`, or `cloudpickle` on untrusted data. Prefer `json` for structured metadata, `safetensors` for weights, and `numpy.load(..., allow_pickle=False)` for arrays.

Expand Down
89 changes: 30 additions & 59 deletions docs/explanation/configuration.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,11 @@
# Configuration

The config system is intended to make Python, YAML, CLI, and Studio payloads use the same workflow shape.
PhysicalAI uses constructor-shaped data so Python, YAML, the CLI, and Studio
can describe the same runtime graph.

```bash
physicalai run --config runtime.yaml
```

## Layers

```text
Config
typed constructor args for one class

ComponentSpec
target + args for one instantiable component

Workflow config
user-authored workflow before execution

Manifest
exported package metadata after build/export

Orchestrator
live object that executes the workflow
```
## Construction Recipes

## ComponentSpec

Direct class mode:
A `ComponentConfig` names one class and the arguments needed to create it.

```yaml
class_path: physicalai.capture.UVCCamera
Expand All @@ -37,47 +15,40 @@ init_args:
height: 480
```

> **Tip:** Use stable device paths (`/dev/v4l/by-id/...`) in config files. Index-based paths like `/dev/video0` can change after reboot.

Registry mode:
Classes opt in to exporting this recipe with `@export_config`. `to_config()`
captures construction intent rather than mutable runtime state: supplied
arguments are preserved, omitted defaults remain omitted, and connections or
open handles are never exported.

```yaml
type: uvc
device: /dev/v4l/by-id/usb-Example_Camera-video-index0
width: 640
height: 480
```text
live opted-in component -> to_config() -> JSON/YAML -> instantiate() -> new component
```

If both `class_path` and `type` are present, `class_path` takes precedence.
`instantiate()` calls constructors only. The application remains responsible
for lifecycle methods such as `connect()` and `run()`.

## Typed Config
## Workflow Documents

Typed configs are useful when constructor validation and IDE support matter.
A workflow document adds command-level structure around components. For
example, `physicalai run` expects runtime constructor arguments under
`runtime:` and optional run-method arguments under `run:`. A bare exported
`RobotRuntime` ComponentConfig is also accepted and reshaped by the loader.

```python
@dataclass
class Pi05Config(Config):
chunk_size: int = 50
n_action_steps: int = 50
Nested components retain the same `class_path` + `init_args` form, so a robot,
action source, model, camera map, and callbacks form one recursive recipe.

def __post_init__(self) -> None:
if self.n_action_steps > self.chunk_size:
raise ValueError("n_action_steps must be <= chunk_size")
```

Typed configs do not decide which class to instantiate. They only validate and carry constructor arguments.
## Manifests

```python
cfg = Pi05Config(chunk_size=50)
policy = instantiate_obj(cfg, target_cls=Pi05)
```
An inference manifest describes an exported policy package: artifacts,
features, processors, runners, and compatibility metadata. Its
`ComponentSpec` supports registry aliases and artifact handling and remains
separate from strict captured `ComponentConfig` data.

## Execution Boundary
Use workflow configuration for deployment intent. Use manifests for package
metadata produced during export.

Configuration objects remain passive data. Orchestrators are responsible for creating live objects and executing workflows.
## Trust

```python
config = RuntimeConfig.load("runtime.yaml")
runtime = RobotRuntime.from_config(config)
runtime.run()
```
Resolving `class_path` imports and executes local Python code. Only trusted
application or user-authored configuration belongs at the instantiation
boundary; peer metadata and transport control messages do not.
3 changes: 2 additions & 1 deletion docs/getting-started/run-a-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ The minimal runtime configuration looks like this.
```yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
82 changes: 39 additions & 43 deletions docs/how-to/config/instantiate-components.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# Instantiate Components

> **Preview:** The config system (`physicalai.config`) is a planned API. The examples below document the target design. Currently, `ComponentSpec` lives in `physicalai.inference.manifest`.

A component spec describes one instantiable object.

The most explicit form uses a class path.
A `ComponentConfig` is a construction recipe with one importable class and
its supplied constructor arguments.
Comment thread
maxxgx marked this conversation as resolved.

```yaml
class_path: physicalai.capture.UVCCamera
Expand All @@ -14,52 +11,51 @@ init_args:
height: 480
```

The shorter form uses a registry name.
Use `instantiate()` to construct trusted local configuration. It calls the
constructor but does not call lifecycle methods such as `connect()`, `run()`,
or `start()`.

```yaml
type: uvc
device: /dev/video0
width: 640
height: 480
```python
from physicalai.config import instantiate

camera = instantiate({
"class_path": "physicalai.capture.UVCCamera",
"init_args": {"device": "/dev/video0", "width": 640, "height": 480},
})
camera.connect()
```

You can construct and instantiate the same spec from Python.
## Export a live component

```python
from physicalai.inference.manifest import ComponentSpec
from physicalai.inference.component_factory import instantiate_component
Classes opt in with `@export_config`. The decorator remembers only arguments
the caller supplied, so omitted constructor defaults remain omitted.

spec = ComponentSpec(
class_path="physicalai.capture.UVCCamera",
init_args={"device": "/dev/video0", "width": 640, "height": 480},
)
```python
from physicalai.capture import UVCCamera
from physicalai.config import to_config

camera = instantiate_component(spec)
camera = UVCCamera(device="/dev/video0", width=640, height=480)
config = to_config(camera)
```

Nested component specs are instantiated recursively.
Nested opted-in components use the same shape recursively. Configs contain
JSON values only; paths become strings, tuples become lists during export,
and non-finite floats are rejected.

```yaml
class_path: physicalai.runtime.RobotRuntime
init_args:
robot:
class_path: physicalai.robot.so101.SO101
init_args:
port: /dev/ttyACM0
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
model:
class_path: physicalai.inference.InferenceModel
init_args:
export_dir: ./exports/act_policy
execution:
class_path: physicalai.runtime.SyncExecution
cameras:
wrist:
class_path: physicalai.capture.UVCCamera
init_args:
device: /dev/video0
```python
from physicalai.config import save_yaml

save_yaml(camera, "camera.yaml")
```

`ComponentSpec` describes what should be built. Instantiation is the separate step that creates the live object.
## Trust boundary

`class_path` selects Python code to import and execute. Pass only trusted
application or user-authored configuration to `instantiate()`. Do not
instantiate robot metadata, camera metadata, shared-memory messages, or other
peer-controlled payloads.

Inference manifest `ComponentSpec` is a separate compatibility schema with
registry aliases and artifact handling. Use `physicalai.config` for captured
construction recipes; use the manifest APIs when loading exported policy
metadata.
7 changes: 5 additions & 2 deletions docs/how-to/config/write-inference-config.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Write Inference Config

> **Preview:** The config system (`physicalai.config`) is a planned API. The examples below document the target design.

Use an inference config when you need to author an inference pipeline outside an exported manifest.

```yaml
Expand All @@ -21,3 +19,8 @@ action = model.select_action(observation)
```

Use workflow config to express user-authored intent. Use a manifest to describe exported package metadata.

To export a model's supplied constructor arguments from Python, call
`physicalai.config.to_config(model)`. Rebuild that trusted local recipe with
`physicalai.config.instantiate(config)`; construction loads the model package
and can allocate substantial resources.
3 changes: 2 additions & 1 deletion docs/how-to/config/write-runtime-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ A runtime config describes a robot control workflow before execution starts.
# runtime.yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
3 changes: 2 additions & 1 deletion docs/how-to/runtime/run-policy-on-robot.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ Write a runtime configuration file.
# runtime.yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
12 changes: 7 additions & 5 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@ with runtime:
physicalai robot serve --config examples/so101/serve.yaml
```

The flat YAML fields are `name`, `robot_class`, optional `robot_kwargs`, optional
`allow_remote` (default `false`), and optional positive `rate_hz` (default 100 Hz).
Direct CLI arguments use the same names, including nested constructor values such as
`--robot_kwargs.port /dev/ttyACM0`. Serving stays in the foreground until SIGINT or
SIGTERM and returns nonzero for startup, loop, repeated-read, or disconnect failures.
The YAML fields are `name`, required `robot` (a ComponentConfig with
`class_path` + `init_args`), optional `allow_remote` (default `false`), and
optional positive `rate_hz` (default 100 Hz). Direct CLI arguments use the same
names, including nested robot constructor values such as
`--robot.init_args.port /dev/ttyACM0`. Flat `robot_class` / `robot_kwargs` are
unsupported. Serving stays in the foreground until SIGINT or SIGTERM and returns
nonzero for startup, loop, repeated-read, or disconnect failures.

Use `--verbose` to include driver construction, lock acquisition, initial observation,
endpoint declaration, and cleanup details.
Expand Down
5 changes: 2 additions & 3 deletions docs/reference/inference-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ InferenceModel(
)
```

The model can be constructed directly from an export directory or loaded from config.
The model can be constructed directly from an export directory or nested in a
trusted workflow ComponentConfig.

## Constructors

Expand All @@ -38,8 +39,6 @@ This downloads the policy package snapshot from the Hugging Face Hub and loads
it like a local export. Additional keyword arguments are forwarded to the
constructor (e.g. `backend`, `device`).

> **Note:** `InferenceModel.from_config()` is a planned API.
## Methods

### `select_action`
Expand Down
12 changes: 12 additions & 0 deletions examples/runtime/async_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,10 @@

import argparse
import signal
from pathlib import Path

from physicalai.capture import select_cameras_interactive
from physicalai.config import save_yaml, to_config
from physicalai.inference import InferenceModel
from physicalai.runtime import (
AsyncExecution,
Expand Down Expand Up @@ -102,6 +104,11 @@ def _handle_sigint(sig: int, frame: object) -> None:
rt_group.add_argument("--shared-camera", action="store_true", help="Use shared memory cameras (iceoryx2) — faster but incompatible with debugger")
rt_group.add_argument("--request-threshold", type=float, default=0.75, help="Request new inference when queue drops below this fraction of chunk_size (default: 0.75 = trigger when 75%% of actions remain)")
rt_group.add_argument("--lerp-frames", type=int, default=3, help="LerpSmoother blend duration in frames (default: 3)")
rt_group.add_argument(
"--export-config",
type=Path,
help="Save the constructed runtime as YAML and exit before connecting hardware.",
)

# Rerun
rr_group = parser.add_argument_group("rerun")
Expand Down Expand Up @@ -158,6 +165,11 @@ def _handle_sigint(sig: int, frame: object) -> None:
callbacks=callbacks,
)

if args.export_config:
save_yaml(to_config(runtime), args.export_config)
print(f"Saved runtime config to {args.export_config}")
return

with runtime:
for name, cam in cameras.items():
w = getattr(cam, "actual_width", None)
Expand Down
Loading
Loading