Skip to content

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Robot Synthetic Data Generation Pipeline

Original Git repository: https://github.com/PhysicalAI-AIM/Robot_synthetic_data_generation_workshop.git

End-to-end pipeline for robot manipulation on AMD GPUs (ROCm): Synthetic Data Generation → VLA Training → Simulation Evaluation.

Verified on CDNA3 (MI300 series), RDNA4 (Radeon AI PRO R9700), and RDNA3.5 (Radeon PRO W7900).

┌──────────────────────────────┐  ┌─────────────────────┐   ┌──────────────────────────┐
│ 01_gen_data_custom_scene.py  │  │  02_train_vla.py    │   │  03_eval_custom_scene.py │
│   kitchen GLB + floor_origin │  │                     │   │                          │
│   up + wrist cameras         │─▶│  SmolVLA fine-tune  │─▶│  Closed-loop eval        │
│   100 episodes, GPU render   │  │  on LeRobot dataset │   │  in Genesis kitchen sim  │
│                              │  │  HF checkpoint out  │   │  success rate + video    │
└──────────────────────────────┘  └─────────────────────┘   └──────────────────────────┘
     Franka 7-DOF                  lerobot/smolvla_base       render → VLA → PD
     pick red cube                 freeze vision encoder      action chunking
     2 cameras (up/wrist)          train expert + state_proj  randomized cube pos

Pipeline Routing — Two Paths

Path A: CDNA3 (MI300 series) Path B: RDNA4/3.5 (R9700 / W7900)
Data Generation Skip — use pre-built HuggingFace dataset End-to-end 01_gen_data_custom_scene.py kitchen scene (100 episodes)
Training Local training Local training
Evaluation CPU render (llvmpipe) GPU render (radeonsi)
Total time ~20 min (train + eval) ~30 min (gen + train + eval)

Path A: CDNA3 (MI300/MI325) — Pre-built Dataset

For MI300/MI325 nodes without GPU graphics pipeline. The 100-episode dataset is pre-generated on RDNA4 and pulled from HuggingFace.


Path B: RDNA4/3.5 (R9700 / W7900) — End-to-End

For RDNA nodes with GPU graphics pipeline. All three steps run live during the pipeline — no external dataset needed.

Model Weights

SmolVLA models are downloaded from ModelScope (China-friendly mirror):

Model ModelScope URL
SmolVLA base (450M) https://modelscope.cn/models/lerobot/smolvla_base
SmolVLM2-500M backbone https://modelscope.cn/models/HuggingFaceTB/SmolVLM2-500M-Video-Instruct

The pre-built image crpi-zi7ws1z1ui8z86ek.cn-shanghai.personal.cr.aliyuncs.com/amd_image/genesis-amd:latest already caches both models. do not need to download manually.

Quick Start

docker exec -it genesis-amd bash

# Step 0: Download kitchen scene assets (~130 MB, first time only)
python scripts/00_download_kitchen.py --mesh-only

# Step 1: Data Generation — kitchen scene + up/wrist cameras (~10-20 min)
python scripts/01_gen_data_custom_scene.py \
  --scene rustic_kitchen --anchor floor_origin \
  --camera-layout up_wrist \
  --n-episodes 100 --seed 42 \
  --repo-id local/franka-kitchen-wrist-100ep

# Step 2: Training (~10-20 min)
python scripts/02_train_vla.py \
  --dataset-id local/franka-kitchen-wrist-100ep \
  --pretrained lerobot/smolvla_base \
  --n-steps 8000 --batch-size 4 --num-workers 4 \
  --run-name smolvla_kitchen_wrist

# Step 3: Evaluation — kitchen scene (~5 min)
python scripts/03_eval_custom_scene.py \
  --checkpoint output/train/smolvla_kitchen_wrist/final \
  --dataset-id local/franka-kitchen-wrist-100ep \
  --scene rustic_kitchen --anchor floor_origin \
  --camera-layout up_wrist \
  --n-episodes 20 --seed 99 --record-video

Reference Results (kitchen+wrist scene, directly comparable)

Metric R9700 (RDNA4) W7900D (RDNA3.5)
Data gen success 100% 100%
Loss (start → end) 0.671 → 0.011 0.67 → 0.011
Peak VRAM 2.33 GB 2.27 GB
Eval success rate (GPU render, kitchen) ~75% ~70% (3 seeds pooled)

Note: Both GPUs use the same kitchen+wrist scene — results are directly comparable.


Dataset

The training dataset is pre-generated on RDNA4 and published on HuggingFace. The pre-built Docker image already caches it:

Item Value
Scene Rustic kitchen GLB + Franka Panda picking a red cube
Camera layout up (overhead) + side (wrist-mounted, eye-in-hand), 640×480
Episodes / Frames 100 / 13,500
Size ~200 MB (AV1 video, LeRobot v3.0)
Action space 9-DoF joint position (7 arm + 2 finger)
Generated on RDNA4 (Radeon AI PRO R9700), Genesis 0.4.5, seed=42

⚠️ The tensor key observation.images.side stores the wrist (eye-in-hand) camera, not a world-fixed side view. Do not mix with the legacy up+world-side dataset — key names collide but semantics differ.


Docker Setup

Build Docker Image

git clone https://github.com/AMD-AIM/Physical_AI_Challenge.git
cd Physical_AI_Challenge
bash docker/build.sh    # → genesis-amd:latest

Container

docker run -it \
  --device=/dev/kfd --device=/dev/dri --group-add video --ipc=host \
  --network=host \
  -e HF_HUB_OFFLINE=1 \
  genesis-amd:latest bash

Start Jupyter

jupyter-lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root

Access the notebook via notebooks.amd.com. All outputs are written to output/ under the directory.


Notebooks

Notebook Target Hardware Description
rdna_pipeline.ipynb R9700 / W7900 (RDNA4/3.5) End-to-end pipeline, GPU render

Content Overview

Section rdna_pipeline.ipynb
0. Environment Setup GPU detection + kitchen GLB download
1. Data Generation 100 ep kitchen scene generation
2. VLA Training SmolVLA post-training
3. Evaluation GPU-render closed-loop eval
4. Summary PNG / MP4 / JSON

All outputs (checkpoints, plots, eval videos) are written to output/ in the file browser.


Dependencies

Package Version Purpose
genesis-world ≥1.2.0 Physics simulation + rendering (Taichi backend, ROCm native).
lerobot ≥0.6.0 Dataset format + SmolVLA model
torch ≥2.1 (ROCm) Training and inference
transformers ≥4.40 SmolVLA backbone (Idefics3)
accelerate latest HuggingFace model loading
num2words latest Required by transformers SmolVLM processor
numpy ==2.1.2 Required by Genesis; must match scikit-image C extension ABI
scikit-image ≥0.22 Must be recompiled against numpy==2.1.2
xvfb system Headless rendering (apt-get install)
ffmpeg system Video encoding (apt-get install)

Hardware: CDNA3 (AMD Instinct MI300/MI325 series) or RDNA3 (AMD Radeon PRO W7900) or RDNA4 (AMD Radeon AI PRO R9700); ≥4 GB VRAM on either.


Data Flow

Genesis Scene                    LeRobot Dataset                SmolVLA
┌──────────────┐                ┌───────────────┐              ┌──────────────┐
│ Franka Panda │                │ observation   │              │ Vision       │
│ Red Cube     │──IK plan──────▶│  .state [9D] │──train──────▶│ Encoder      │
│ 2 Cameras    │   joint lerp   │  .images.up   │              │ (frozen)     │
│              │   render       │  .images.side │              │              │
│ Physics sim  │                │ action [9D]   │              │ Expert       │
│ (Genesis)    │                │ task (text)   │              │ Layers       │
└──────────────┘                └───────────────┘              │ (trainable)  │
  ▲ scene source:                                              │              │
  │ (a) flat plane (01)                                        │ → action     │
  │ (b) kitchen GLB (02)         same LeRobot format           │   chunk [50] │
                                                               │              │
Eval Loop:                                                     │              │
  render ─────────────────────────────────── inference ─────── │              │
  observe state ──────────────────────────── predict ───────── │              │
  execute action[0] ──────── PD control ──── scene.step()      └──────────────┘

Appendix A: Rendering Backend — CDNA3 vs RDNA4

Architecture EGL Renderer Type
CDNA3 (MI300/MI325 series) llvmpipe CPU software rasterization
RDNA4 (R9700) radeonsi GPU hardware rasterization

CDNA3 has no graphics pipeline — Genesis falls back to CPU llvmpipe for camera rendering. RDNA4 has a full graphics pipeline (radeonsi, hardware-accelerated), which is the primary source of the 3-4× data-generation speedup and eliminates the render-gap bias at evaluation.

CPU-render evaluation bias (MI300/MI325): CPU and GPU rasterizers produce visually different frames. A policy trained on GPU-rendered data but evaluated with CPU rendering shows a systematic ~20 pt lower success rate. On the kitchen+wrist main path: This is expected behaviour, not a bug. For benchmark-quality numbers, evaluate on an RDNA4 or another GPU-render node.

Appendix B: Known Compatibility Notes

Issue Fix
numpy / scikit-image ABI mismatch (numpy.dtype size changed) pip install --force-reinstall "scikit-image>=0.22" "numpy==2.1.2"
torchcodec pip wheel links CUDA libs, fails on ROCm bash setup_torchcodec.sh (CPU-only build)

References

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages