VLA based Object pick and place

[noshluk2]

Description:
Create a perception-driven pick-and-place system for a robotic arm that can detect, segment, and pick objects based on a given instruction, then place them at the correct destination.

File : ros2 launch panda panda_vla_pipeline.launch.py in m6_to_m8 branch

Tasks:

1. Object Detection

   • Integrate a fast object detection model for identifying all objects in the workspace.

2. Segmentation

   • Apply a segmentation model to extract precise object boundaries.

3. Pose Estimation

   • Use a VLA (Vision-Language-Action) model to estimate object poses in the robotic arm’s hand frame.

4. Instruction Parsing

   • Extract target destination for each object from the text instruction.

5. Pick & Place Execution

   • Plan and execute the robotic arm trajectory to pick and place the objects at their designated destinations.