Synthetic Data Generation Pipeline

[gemblerz]

Goal: Create a pipeline that takes a few episodes of OpenArm manipulation and generates over a few K episodes.

Main Tasks:

• Obtain Doga's OpenARM robot manipulation dataset as human-guided episode
• Try tools like DexMimicGen and design the pipeline
• Generate few K episodes for a couple of robot tasks (e.g., pick-and-place cubic and open closet door)
• Replay episodes in Isaac Sim

Optional Tasks:

• Augment other sensor data (e.g., tactile) in generated synthetic dataset
• Analyze # of episodes VS training accuracy (if we can finetune a VLA model with this dataset)

Success Criteria:

• End-to-end workflow document showing an example workflow from putting episodes for a task and getting x1000 dataset
• Getting thousands of episode for the two robot tasks