[Question] Recommended prompt & coordinate protocol for VLM grounding evaluation

[Yjonben]

Context

We are evaluating the VLM / language-generation path of InternVLA-M1 (Qwen2.5-VL; evaluating VLM extracted from Bridge VLA checkpoint qwen_vl_interface.model.*) (not the action policy rollout) on embodied grounding benchmarks via an internal evaluator (RefCOCO-family bbox, Where2Place point, RoboSpatial-Home point/yes-no, etc.).

We want to align prompts and coordinate interpretation with the officially intended format, but could not find a complete grounding evaluation protocol in the README / VLMEvalKit wrapper / paper.

Questions

1. For VLM-only inference (eval_in_vqa=True or equivalent), what is the recommended user prompt template for:
   • referring expression bbox (RefCOCO-style)
   • point / free-space pointing (Where2Place, RoboSpatial-Home context)
2. What output format should we expect and parse?
   • JSON (bbox_2d / point_2d), tags (<box>, <point>), or free-form coordinates?
3. What coordinate system should predictions use?
   • normalized 0–1, normalized 0–1000, or absolute pixel (original or resized image)?
4. Is there an official grounding eval script or VLMEvalKit config we should reuse instead of custom prompts?

Our current setup (for reference)

• Backend: official repo wrapper, language generation only
• Prompt: neutral wording like "Report the bounding box / point coordinates" (no explicit range), because official docs do not specify one
• Scoring: we try norm01 / norm1000 / pixel based on smoke outputs

Smoke observations (may be wrong due to prompt mismatch)

• README demo: "Give the bounding box for the apple." with <box>[[x1,y1,x2,y2]]</box>-style outputs
• RefCOCO smoke: pixel interpretation IoU ~0.94; norm1000 IoU ~0
• Where2Place: pixel mask hits; norm1000 fails
• Unclear whether coordinates are original-image pixel, letterbox/resized pixel, or another scale

Could you confirm the correct protocol, or point us to the official eval entry / prompt file? Thank you!