Context
From #318 §9 (multi-monitor Edge). Instructions document screen-absolute pixel math (query window origin + capture offset), but agents still must hand-compute coordinates and re-capture after layout shifts.
Problem
click { x, y } is screen-absolute. Coordinates inferred from a window-targeted capture are window-relative. On a second monitor (e.g. window at x=1923) the offset math is error-prone; DPI/move/scroll silently breaks cached pixels.
Ask
Let click (and drag endpoints) accept window-relative coordinates when a window target (handle / window / process / foreground) is set:
{ "handle": 12345, "at": { "x": 412, "y": 88 } }
MCEC adds the window client origin (and documents any DPI/scale behavior). Alternative/complement: echo captureTransform { originX, originY, scaleX, scaleY } on capture results.
Acceptance
- Agent can click a point seen in a window capture without manual monitor-origin math.
- Documented behavior for multi-monitor and per-monitor DPI.
Context
From #318 §9 (multi-monitor Edge). Instructions document screen-absolute pixel math (
querywindow origin + capture offset), but agents still must hand-compute coordinates and re-capture after layout shifts.Problem
click{ x, y }is screen-absolute. Coordinates inferred from a window-targetedcaptureare window-relative. On a second monitor (e.g. window at x=1923) the offset math is error-prone; DPI/move/scroll silently breaks cached pixels.Ask
Let
click(anddragendpoints) accept window-relative coordinates when a window target (handle/window/process/foreground) is set:{ "handle": 12345, "at": { "x": 412, "y": 88 } }MCEC adds the window client origin (and documents any DPI/scale behavior). Alternative/complement: echo
captureTransform{ originX, originY, scaleX, scaleY }oncaptureresults.Acceptance