feat(bridge-windows): Phase 0e2 + 0e3 — list_windows, capture, execute (real UIA driving real Windows)

[rrader26]

Summary

Brings the Windows UIA bridge from "scaffolded" to "actually drives
Windows software." Includes list_windows, capture, and execute
plus a robustness fix for the PowerShell-emits-BOM stdin quirk and a
live Notepad demo that proves the full capture → execute → re-capture
loop works against real software.

This is the milestone where AgentMark Desktop stops being a
prototype and starts being real. Validated on Windows 11 ARM64
(Parallels) — bridge captured a live Notepad tree (37 elements, depth
6), wrote "Hello from AgentMark Desktop" into the editor via
ValuePattern.SetValue, and re-captured to verify the typed text
landed. Zero vision/AI involvement; pure structured-tree manipulation
through the protocol.

What's in this PR

Uia/StaWorker.cs

Single-threaded apartment worker. UIA is COM-STA; every public
capturer call runs on a dedicated STA thread to avoid COM proxy
leaks + deadlocks.

Uia/CaptureDtos.cs

DTOs that camelCase-serialize directly into the
@thinkfleet/agentmark v0.4 DesktopCapture shape — same wire
format the Node-side WindowsUiaBackend (Phase 0f) will consume.

Uia/RoleMapper.cs

UIA ControlType → normalised DesktopRole vocabulary. Handles
Window/Pane/ToolBar/Menu/Tab/Tree/List/Table/Button/Edit/CheckBox/
Slider/Hyperlink/etc. Falls back to InferCustom() for
ControlType.Custom (WPF / WinForms / Electron) before giving up to
"other".

Uia/UiaCapturer.cs

The real worker. Three public methods:

• ListWindows() — walks top-level UIA windows, skips
  chrome-less / off-screen / no-title ghosts, marks the one
  containing the focused element with hasFocus=true.
• Capture(req) — resolves target by windowId (hwnd:0x..),
  processId, processName, or windowTitle; falls back to focused
  window. Walks the tree depth-first with depth + element-count +
  deadline caps. Extracts value from Value / RangeValue / Text
  patterns, selection from SelectionItem, expansion from
  ExpandCollapse, toggle state from Toggle (mapped onto
  aria.pressed / aria.checked). Stashes a live element handle
  per id in a per-capture session so subsequent execute can
  resolve back to the live UIA element.
• Execute(req) — dispatches an action by element_id through
  the right UIA pattern:
  • click → InvokePattern.Invoke (with Toggle / SelectionItem
    / ExpandCollapse / focus+Space fallbacks)
  • type → ValuePattern.SetValue (with focus+keystroke fallback,
    Ctrl+A/Del clear support)
  • select → SelectionItemPattern.Select on the target, or
    expand + locate-by-name on a Selection container, or
    ValuePattern for editable combos
  • check → TogglePattern.Toggle with state convergence
  • expand → ExpandCollapsePattern.Expand / Collapse
  • focus → AutomationElement.Focus
  • scroll_to → ScrollItemPattern.ScrollIntoView with focus
    fallback
  • key → Keyboard.Type / TypeSimultaneously with modifier
    parsing (ctrl|alt|shift|meta|win) and friendly key-name
    vocabulary

Every UIA call is defensively wrapped — UIA returns
ElementNotAvailable mid-walk on dynamic UIs (Excel especially) and
we don't want one stale element to abort the whole capture or execute.

Program.cs

• Dispatcher routes list_windows / capture / execute through the
  STA thread; lazy-instantiates UIA so ping-only smoke tests stay
  fast.
• Strips UTF-8 BOM from each read line — PowerShell prepends one on
  the first WriteLine sometimes and System.Text.Json rejects the
  malformed request otherwise.

Demos

• scripts/uia-smoke-test.ps1 — exercises list_windows + capture
  against whatever windows are visible. Safe to run anywhere; reports
  empty session honestly when run from SSH (session-0 limitation).
• scripts/notepad-demo.ps1 — full end-to-end loop:
  1. Asserts Notepad is already open
  2. list_windows → find Notepad
  3. capture → locate text_area
  4. execute type → write a message
  5. capture → verify the typed text is in element.value
     Validated on Win 11 ARM64. PASSED.

Footnote: SSH session limitation

SSH login on Windows runs in a different session than the interactive
Parallels console, so UIA from an SSH-spawned bridge sees an empty
desktop. The bridge logic is correct (list_windows cleanly returns
[] with no error); the limitation is that the SSH session's window
station has no UI. Iteration-heavy phases run from the VM's
interactive PowerShell for now; a scheduled-task / PsExec trick to
launch the bridge in the interactive session from SSH is queued for
later.

Live demo proof

Step 2 -- list_windows...
Raw list_windows response (first 800 chars):
{"jsonrpc":"2.0","id":1,"result":{"windows":[
  {"windowId":"hwnd:0x00030352","processName":"conhost",...,"windowTitle":"Administrator: Windows PowerShell",...},
  {"windowId":"hwnd:0x000703F8","processName":"Notepad","processId":5144,"windowTitle":"Untitled - Notepad","windowClass":"Notepad"...},
  ...
]}}
  found: Untitled - Notepad (process=Notepad, hwnd:0x000703F8)

Step 3 -- capture Notepad...
  treeDepth=6  elementCount=37
  editor element_id : el_28d1e167 (role=text_area)
  current value     : ''

Step 4 -- execute type into editor...
  ok        : True
  newValue  : Hello from AgentMark Desktop -- phase 0e3 live demo

Step 5 -- re-capture and verify...
  editor value after type : 'Hello from AgentMark Desktop -- phase 0e3 live demo'

LIVE DEMO PASSED -- AgentMark Desktop drove real Notepad end-to-end.

Next PRs

1. Phase 0e4: same loop against Excel (bigger tree, real customer
   alignment via NowCerts users)
2. Phase 0f: Node-side WindowsUiaBackend — replaces FixtureBackend
   in the MCP server, spawns this bridge, speaks JSON-RPC. After this
   lands, Claude Code / Cursor / Codex on Windows get real desktop
   access through agentmark_desktop_* MCP tools with zero
   per-tool integration work.

🤖 Generated with Claude Code