fix: Improve 004 robotless deployment tutorial with dataset and model from official repo and MuJoCo visualization#197
Open
zhuo-yoyowz wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the tutorials content to make the “robotless deployment” flow smoother from a fresh environment by adding a pinned tutorial requirements file, a new Pi0.5 + LIBERO replay helper module, and MuJoCo visualization, alongside refreshed tutorial notebooks and docs.
Changes:
- Refresh tutorial landing docs and notebook set (002–005), including a new SO-101 end-to-end notebook.
- Add
examples/tutorials/requirements.txtand a newphysicalai_pi05_helper.pyto support Pi0.5 package + LIBERO replay validation and visualization. - Update top-level README header image to use a local asset.
Reviewed changes
Copilot reviewed 8 out of 26 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Switches README header image to a local asset. |
| examples/tutorials/requirements.txt | Adds pinned Python dependencies for running tutorial notebooks. |
| examples/tutorials/README.md | Rewrites tutorials landing page with new “Getting Started” and notebook index. |
| examples/tutorials/physicalai_pi05_helper.py | Adds helper utilities for dataset/model download, replay validation, and MuJoCo rendering. |
| examples/tutorials/media/README.me | Adds a minimal readme for media assets directory. |
| examples/tutorials/002_Using_Physical_AI_Studio.ipynb | Updates/introduces the Physical AI Studio walkthrough notebook content. |
| examples/tutorials/003_OpenVINO_Optimization.ipynb | Updates/introduces the OpenVINO optimization/deployment notebook content. |
| examples/tutorials/004_Test_Deployment_Without_Robot.ipynb | Updates/introduces the robotless deployment validation notebook with replay + MuJoCo visualization. |
| examples/tutorials/005_collect_train_deploy.ipynb | Updates/introduces the collect→train→deploy SO-101 workflow notebook. |
| ```bash | ||
| pip install "physicalai[so101,capture]" physicalai-train | ||
| pip install jupyter # if jupyter notebook is not already installed | ||
| phython -m venv venv |
| phython -m venv venv | ||
| source venv/bin/activate | ||
| git clone https://github.com/openvinotoolkit/physicalai.git | ||
| cd physicalai/examples/notebooks |
Comment on lines
+703
to
+704
| frame_count = min(max_rendered_frames, len(actions)) if actions is not None and len(actions) else max_rendered_frames | ||
| source = "scripted MuJoCo pick-and-place trajectory" |
Comment on lines
+193
to
+196
| "CAMERAS = [\n", | ||
| " (<camera-name>, \"uvc\", <camera description>),\n", | ||
| " (<camera-name>, \"uvc\", <camera description>),\n", | ||
| "]\n", |
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from physicalai.runtime import ActionQueue, PolicyRuntime, SyncExecution\n", |
| " execution=SyncExecution(fps=FPS, request_threshold=0.5),\n", | ||
| " fps=FPS,\n", | ||
| " cameras=cameras,\n", | ||
| " action_queue=ActionQueue(),\n", |
| @@ -0,0 +1 @@ | |||
| # Media assets | |||
Comment on lines
+121
to
+122
| "INSTALL_HARDWARE_DEPS = \"True\"\n", | ||
| "USE_SHARED_CAMERA = \"True\"\n", |
| "import sys\n", | ||
| "\n", | ||
| "WORKSPACE = Path.cwd().resolve()\n", | ||
| "TUTORIALS_REF = \"docs/tutorials\"\n", |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-This PR improves 004_Test_Deployment_Without_Robot.ipynb so users can run the full robotless deployment flow more smoothly from a fresh environment, with dataset and Pi0.5 model in IR format from official repo, and observe final result in MuJoCo env.
Changes
-Add first-run setup logic so the notebook can clone the Physical AI tutorial sources when required files are missing.
Install physicalai==0.1.1 and required runtime dependencies for a more stable tutorial experience.
-Add LIBERO dataset replay preparation using HuggingFaceVLA/libero, matching the released OpenVINO/pi05-libero-fp16-ov model.
-Add replay validation for downloaded dataset frames, task text, expert actions, and model-predicted actions.
-Keep OpenVINO device selection for available devices, preferring GPU when visible.
-Add a MuJoCo-based scripted pick-and-place visualization as a final robotless smoke test.
-Add helper utilities in physicalai_pi05_helper.py for dataset download, replay visualization, model execution, and MuJoCo rendering.