Questions about the output of VLM

[w4nanch1]

Thanks for opening such good work!

I have two questions about the work:

1. Is the target position output by the VLM the same as the action output by the VLA—namely, a trajectory composed of multiple points?

2. If so, why do we need to use this trajectory to steer the VLA? Why not directly feed the trajectory output by the VLM to the robot for execution?

I would greatly appreciate it if you could help answer my questions!

[Treeeplanter]

Hi, thank you for the great questions!

1. Does the VLM output a trajectory?
   No. The VLM outputs a differentiable reward function (code), not a sequence of points. It defines a "cost landscape" based on scene keypoints rather than a specific path.
2. Why steer the VLA instead of executing VLM outputs directly?
   VLMs are great at semantic reasoning but poor at precise numerical coordinates. We let the VLA handle the precision while the VLM provides the correction. In short: The VLA knows "how to move," and VLS uses the VLM to tell it "where to go" via gradients.