Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/development/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ Repository-wide contributor guidance.

- [Coding Standards](coding-standards.md) — coding, writing, and testing standards for all contributors and agents.
- [Security Rules](security.md) — security requirements for `src/physicalai/`.
- [Component config](component-config.md) — canonical accepted design note for captured component configuration.
1,098 changes: 1,098 additions & 0 deletions docs/development/component-config.md

Large diffs are not rendered by default.

20 changes: 17 additions & 3 deletions docs/development/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,23 @@ These rules apply when writing, editing, or reviewing code under `src/physicalai
raise ValueError(f"Path escapes base directory: {user_path!r}")
```

4. No arbitrary `class_path` import from untrusted manifests or YAML. `instantiate_component` in `inference/component_factory.py` resolves `class_path` via `ComponentRegistry` and `importlib`. Only register trusted short names; treat manifest `class_path` values as untrusted unless the export directory is trusted. Prefer registered `type` names for built-ins.

5. Enforce component nesting limits. `_MAX_COMPONENT_DEPTH` in `component_factory.py` caps recursive manifest/YAML instantiation — do not raise or bypass without a security review.
4. No arbitrary `class_path` import from untrusted manifests, YAML, or peer
payloads. `instantiate_component` in `inference/component_factory.py`
resolves `class_path` via `ComponentRegistry` and `importlib`. Only register
trusted short names; treat manifest `class_path` values as untrusted unless
the export directory is trusted. Prefer registered `type` names for
built-ins. `physicalai.config.instantiate` is a separate trusted-local /
parent→child-only construction boundary: never pass robot/camera network
metadata, Zenoh payloads, shared-memory control requests, or other
untrusted peer data into it. Camera reconfigure requests may carry only
explicitly allowlisted scalar settings; the publisher must merge them into
its trusted startup recipe without accepting a peer-selected `class_path`.

5. Enforce component nesting limits. `_MAX_COMPONENT_DEPTH` in
`component_factory.py` caps recursive manifest/YAML instantiation;
`_MAX_CONFIG_DEPTH` in `physicalai.config` caps recursive
`to_config` / `instantiate` trees — do not raise or bypass either without a
security review.

6. Never use `pickle`, `eval()`, `exec()`, `joblib`, `dill`, or `cloudpickle` on untrusted data. Prefer `json` for structured metadata, `safetensors` for weights, and `numpy.load(..., allow_pickle=False)` for arrays.

Expand Down
16 changes: 16 additions & 0 deletions docs/development/shared-construction-wire-decision.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Shared construction wire decision

**Status:** Accepted (recorded in the component-config design)

Private robot-owner and camera-publisher startup stdin is a same-package
ephemeral `Popen` handshake, not a persisted peer protocol. Change the
construction payload to `robot: ComponentConfig` / `camera: ComponentConfig`
with a **hard cutover**: rewrite writers, readers, and fixtures in the same
PR as the Shared\* spawn path. Do **not** add `config_format`, dual-read, or
shape-detection fallback.

Do not bump `ROBOT_TRANSPORT_PROTOCOL_VERSION` or camera frame
`PROTOCOL_VERSION` for this change; those version network/frame payloads, not
startup envelopes.

Canonical detail: [component-config.md — Private startup envelopes](component-config.md#private-startup-envelopes-hard-cutover).
27 changes: 24 additions & 3 deletions docs/explanation/cameras.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,37 @@ Camera instances are not thread-safe. Use one thread per camera instance or add

## SharedCamera

For multi-process or multi-thread access, use `SharedCamera`. It wraps any camera and handles IPC transport automatically.
`SharedCamera` lets one publisher subprocess own a camera's exclusive hardware
connection while any number of subscribers read frames over iceoryx2. It
satisfies the same `Camera` protocol as a direct driver.

Construction uses `camera=` or `from_config()`, matching `SharedRobot`. Prefer
`from_config()` (or YAML) when sharing. A disconnected `@export_config` camera
can be passed directly to `camera=`; this exports its recipe rather than
handing an open device into the child. The publisher always opens fresh. Never
keep a direct camera connected to the same device while sharing: another
connected holder causes open failure.

```python
from physicalai.capture import SharedCamera, UVCCamera
from physicalai.capture import SharedCamera
from physicalai.config import to_config

shared = SharedCamera(UVCCamera(device="/dev/video0"))
# Prefer config-only / disconnected export — no live direct camera held open
shared = SharedCamera.from_config(
{
"class_path": "physicalai.capture.UVCCamera",
"init_args": {"device": "/dev/video0", "backend": "v4l2"},
},
)
# Equivalent after to_config(disconnected_driver):
# shared = SharedCamera.from_config(to_config(driver))
shared.connect()

# Safe to read from multiple threads/processes
frame = shared.read_latest()
```

`create_camera(..., shared=True)` remains a convenience for shareable built-ins
(`uvc`, `realsense`, `basler`) that packs a type into `SharedCamera(camera=...)`.

`SharedCamera` is the recommended approach for production deployments where multiple consumers need camera frames. It avoids the need for manual synchronization and handles frame distribution efficiently.
89 changes: 30 additions & 59 deletions docs/explanation/configuration.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,11 @@
# Configuration

The config system is intended to make Python, YAML, CLI, and Studio payloads use the same workflow shape.
PhysicalAI uses constructor-shaped data so Python, YAML, the CLI, and Studio
can describe the same runtime graph.

```bash
physicalai run --config runtime.yaml
```

## Layers

```text
Config
typed constructor args for one class

ComponentSpec
target + args for one instantiable component

Workflow config
user-authored workflow before execution

Manifest
exported package metadata after build/export

Orchestrator
live object that executes the workflow
```
## Construction Recipes

## ComponentSpec

Direct class mode:
A `ComponentConfig` names one class and the arguments needed to create it.

```yaml
class_path: physicalai.capture.UVCCamera
Expand All @@ -37,47 +15,40 @@ init_args:
height: 480
```

> **Tip:** Use stable device paths (`/dev/v4l/by-id/...`) in config files. Index-based paths like `/dev/video0` can change after reboot.

Registry mode:
Classes opt in to exporting this recipe with `@export_config`. `to_config()`
captures construction intent rather than mutable runtime state: supplied
arguments are preserved, omitted defaults remain omitted, and connections or
open handles are never exported.

```yaml
type: uvc
device: /dev/v4l/by-id/usb-Example_Camera-video-index0
width: 640
height: 480
```text
live opted-in component -> to_config() -> JSON/YAML -> instantiate() -> new component
```

If both `class_path` and `type` are present, `class_path` takes precedence.
`instantiate()` calls constructors only. The application remains responsible
for lifecycle methods such as `connect()` and `run()`.

## Typed Config
## Workflow Documents

Typed configs are useful when constructor validation and IDE support matter.
A workflow document adds command-level structure around components. For
example, `physicalai run` expects runtime constructor arguments under
`runtime:` and optional run-method arguments under `run:`. A bare exported
`RobotRuntime` ComponentConfig is also accepted and reshaped by the loader.

```python
@dataclass
class Pi05Config(Config):
chunk_size: int = 50
n_action_steps: int = 50
Nested components retain the same `class_path` + `init_args` form, so a robot,
action source, model, camera map, and callbacks form one recursive recipe.

def __post_init__(self) -> None:
if self.n_action_steps > self.chunk_size:
raise ValueError("n_action_steps must be <= chunk_size")
```

Typed configs do not decide which class to instantiate. They only validate and carry constructor arguments.
## Manifests

```python
cfg = Pi05Config(chunk_size=50)
policy = instantiate_obj(cfg, target_cls=Pi05)
```
An inference manifest describes an exported policy package: artifacts,
features, processors, runners, and compatibility metadata. Its
`ComponentSpec` supports registry aliases and artifact handling and remains
separate from strict captured `ComponentConfig` data.

## Execution Boundary
Use workflow configuration for deployment intent. Use manifests for package
metadata produced during export.

Configuration objects remain passive data. Orchestrators are responsible for creating live objects and executing workflows.
## Trust

```python
config = RuntimeConfig.load("runtime.yaml")
runtime = RobotRuntime.from_config(config)
runtime.run()
```
Resolving `class_path` imports and executes local Python code. Only trusted
application or user-authored configuration belongs at the instantiation
boundary; peer metadata and transport control messages do not.
3 changes: 2 additions & 1 deletion docs/getting-started/run-a-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ The minimal runtime configuration looks like this.
```yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
82 changes: 39 additions & 43 deletions docs/how-to/config/instantiate-components.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# Instantiate Components

> **Preview:** The config system (`physicalai.config`) is a planned API. The examples below document the target design. Currently, `ComponentSpec` lives in `physicalai.inference.manifest`.

A component spec describes one instantiable object.

The most explicit form uses a class path.
A `ComponentConfig` is a construction recipe with one importable class and
its supplied constructor arguments.

```yaml
class_path: physicalai.capture.UVCCamera
Expand All @@ -14,52 +11,51 @@ init_args:
height: 480
```

The shorter form uses a registry name.
Use `instantiate()` to construct trusted local configuration. It calls the
constructor but does not call lifecycle methods such as `connect()`, `run()`,
or `start()`.

```yaml
type: uvc
device: /dev/video0
width: 640
height: 480
```python
from physicalai.config import instantiate

camera = instantiate({
"class_path": "physicalai.capture.UVCCamera",
"init_args": {"device": "/dev/video0", "width": 640, "height": 480},
})
camera.connect()
```

You can construct and instantiate the same spec from Python.
## Export a live component

```python
from physicalai.inference.manifest import ComponentSpec
from physicalai.inference.component_factory import instantiate_component
Classes opt in with `@export_config`. The decorator remembers only arguments
the caller supplied, so omitted constructor defaults remain omitted.

spec = ComponentSpec(
class_path="physicalai.capture.UVCCamera",
init_args={"device": "/dev/video0", "width": 640, "height": 480},
)
```python
from physicalai.capture import UVCCamera
from physicalai.config import to_config

camera = instantiate_component(spec)
camera = UVCCamera(device="/dev/video0", width=640, height=480)
config = to_config(camera)
```

Nested component specs are instantiated recursively.
Nested opted-in components use the same shape recursively. Configs contain
JSON values only; paths become strings, tuples become lists during export,
and non-finite floats are rejected.

```yaml
class_path: physicalai.runtime.RobotRuntime
init_args:
robot:
class_path: physicalai.robot.so101.SO101
init_args:
port: /dev/ttyACM0
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
model:
class_path: physicalai.inference.InferenceModel
init_args:
export_dir: ./exports/act_policy
execution:
class_path: physicalai.runtime.SyncExecution
cameras:
wrist:
class_path: physicalai.capture.UVCCamera
init_args:
device: /dev/video0
```python
from physicalai.config import save_yaml

save_yaml(camera, "camera.yaml")
```

`ComponentSpec` describes what should be built. Instantiation is the separate step that creates the live object.
## Trust boundary

`class_path` selects Python code to import and execute. Pass only trusted
application or user-authored configuration to `instantiate()`. Do not
instantiate robot metadata, camera metadata, shared-memory messages, or other
peer-controlled payloads.

Inference manifest `ComponentSpec` is a separate compatibility schema with
registry aliases and artifact handling. Use `physicalai.config` for captured
construction recipes; use the manifest APIs when loading exported policy
metadata.
7 changes: 5 additions & 2 deletions docs/how-to/config/write-inference-config.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Write Inference Config

> **Preview:** The config system (`physicalai.config`) is a planned API. The examples below document the target design.

Use an inference config when you need to author an inference pipeline outside an exported manifest.

```yaml
Expand All @@ -21,3 +19,8 @@ action = model.select_action(observation)
```

Use workflow config to express user-authored intent. Use a manifest to describe exported package metadata.

To export a model's supplied constructor arguments from Python, call
`physicalai.config.to_config(model)`. Rebuild that trusted local recipe with
`physicalai.config.instantiate(config)`; construction loads the model package
and can allocate substantial resources.
3 changes: 2 additions & 1 deletion docs/how-to/config/write-runtime-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ A runtime config describes a robot control workflow before execution starts.
# runtime.yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
3 changes: 2 additions & 1 deletion docs/how-to/runtime/run-policy-on-robot.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ Write a runtime configuration file.
# runtime.yaml
runtime:
robot:
class_path: physicalai.robot.so101.SO101
class_path: physicalai.robot.SO101
init_args:
port: /dev/ttyACM0
calibration: ./calibration.json
action_source:
class_path: physicalai.runtime.PolicySource
init_args:
Expand Down
Loading
Loading