Skip to content

click: window-relative coordinates with window target #330

Description

@tig

Context

From #318 §9 (multi-monitor Edge). Instructions document screen-absolute pixel math (query window origin + capture offset), but agents still must hand-compute coordinates and re-capture after layout shifts.

Problem

click { x, y } is screen-absolute. Coordinates inferred from a window-targeted capture are window-relative. On a second monitor (e.g. window at x=1923) the offset math is error-prone; DPI/move/scroll silently breaks cached pixels.

Ask

Let click (and drag endpoints) accept window-relative coordinates when a window target (handle / window / process / foreground) is set:

{ "handle": 12345, "at": { "x": 412, "y": 88 } }

MCEC adds the window client origin (and documents any DPI/scale behavior). Alternative/complement: echo captureTransform { originX, originY, scaleX, scaleY } on capture results.

Acceptance

  • Agent can click a point seen in a window capture without manual monitor-origin math.
  • Documented behavior for multi-monitor and per-monitor DPI.

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions