A small TCP agent that gives AI coding agents remote hands on Windows machines — mouse, keyboard, screenshots, process control, file I/O, UI Automation, window management — over a single line-oriented socket.
Built for Claude Code and other MCP-aware LLM clients to drive virtual or physical Windows machines. Multi-target by design: the modern build (Windows 10 / 11) is in v0.2.0 release-candidate today (Protocol 2.0); a planned legacy build (Windows NT through Server 2003) will share the same wire protocol so the same controller speaks to both.
When an LLM needs to install software, configure a system, or verify a deliverable on Windows, it usually has two options — neither great:
- Hosted Computer Use APIs (pixel-based clicks, no introspection, slow round trips)
- PowerShell over WinRM (no GUI awareness, blind to dialogs and modals, lands you in non-interactive Session 0)
Agent Remote Hands sits between those: a native agent on the target exposes a wire protocol that Claude — or any MCP client — can drive directly, with rich introspection (UI Automation tree, window list, process table, registry, screenshots) and low-latency control (synthetic keyboard/mouse, foreground focus, sub-100 ms screenshots) over plain TCP.
The verbs are also opinionated guard rails rather than thin wrappers around user32!SendInput and friends. The agent and a hand-rolled PowerShell client are calling the same Win32 syscalls, but input.click x y packs the cursor move and the button events into a single atomic SendInput batch with absolute virtual-desktop coordinates — the form Win32 input should take but rarely does in practice, because the seductive shorter form (SetCursorPos + uncoupled mouse_event) drifts. screen.capture composites the OS cursor onto the captured bitmap rather than handing back the bare desktop the way BitBlt and PrintWindow leave it. The wire surface bakes in the right Win32 idioms so callers can't accidentally write the drifty short forms.
If you're an LLM about to use this agent — or writing code that calls it — start with
LLM-OPERATORS.md. It's the operator's-eye view: which docs to read, what to assume, common footguns, and a worked-example session. Ships in the release zip alongside the binary too.
Easiest: install via Scoop on the target machine.
# One-time prep — add a Defender exclusion for ~/scoop (a common Scoop
# practice for any non-mainstream binary; the install script also adds
# one for the agent's eventual install dir).
Add-MpPreference -ExclusionPath "$env:USERPROFILE\scoop"
# Add the bucket and install:
scoop bucket add isted https://github.com/WilliamIsted/scoop-bucket
scoop install agent-remote-hands
# From an elevated PowerShell, complete the system-level setup
# (firewall + Task Scheduler logon-task with restart-on-failure):
agent-remote-hands-setup -DiscoverableOr build from source and install manually (Administrator PowerShell):
cmake -S agents/windows-modern -B agents/windows-modern/build -A x64
cmake --build agents/windows-modern/build --config Release
# Step 1: configure Defender exclusion BEFORE the binary lands
.\Tools\install-agent.ps1 -PrepareDefender
# Step 2: drop remote-hands.exe into C:\Program Files\AgentRemoteHands\
# (drag-drop, RDP file paste, scp, etc — any path that lands it
# directly in the now-excluded directory)
# Step 3: complete the install
.\Tools\install-agent.ps1 -DiscoverableOr one-shot from a URL (release-asset, internal share, etc.):
.\Tools\install-agent.ps1 -SourceUrl 'https://example/remote-hands.exe' -DiscoverableThe script adds a Microsoft Defender exclusion for %ProgramFiles%\AgentRemoteHands\, places the binary there, adds binary-scoped Windows Firewall rules (TCP/8765 + UDP/5353 for mDNS when -Discoverable), and registers a Task Scheduler logon-task with restart-on-failure so the agent autostarts in the user's interactive desktop session on next logon. -Uninstall reverses everything (including the Defender exclusion).
Microsoft Defender's machine-learning heuristic flags any unsigned remote-control tool of this shape (synthetic input + screen capture + arbitrary file I/O + process kill + TCP listener) as Program:Win32/Contebrew.A!ml — regardless of the binary's specific build, install behaviour, or strings present. The detection is structural. Without a code-signing certificate, the only mitigation is to deploy into a path Defender doesn't scan. Code signing is tracked as the agent 1.0 stable-release blocker.
Sanity-check from another machine using the conformance suite:
pip install pytest
python tests/conformance/run.py <vm-host>To wire it into Claude Code (once mcp-server/ lands — see the Roadmap), drop a .mcp.json into your project root:
{
"mcpServers": {
"agent-remote-hands": {
"command": "python",
"args": ["/abs/path/to/mcp-server/server.py"],
"env": {
"REMOTE_HANDS_HOST": "<vm-host>",
"REMOTE_HANDS_PORT": "8765",
"REMOTE_HANDS_GUIDANCE": "hint",
"REMOTE_HANDS_TIER_POLICY": "ask"
}
}
}
}REMOTE_HANDS_* env vars set the project default for each setting; the LLM (or you, in natural language) can override most of them mid-session via MCP tools — see #55 for the settings-registry pattern.
| Path | Purpose |
|---|---|
PROTOCOL.md |
v2 wire-protocol spec — length-prefixed framing, structured errors, three-tier permission model, 66 verbs across 11 namespaces |
docs/windows-automation-notes.md |
Operational gotchas — MSI mutex, Session 0 isolation, foreground lock policy, integrity-level pitfalls |
agents/windows-modern/ |
Windows 10 / 11 agent — C++17, IUIAutomation, WIC for PNG, hand-rolled mDNS responder. Built with CMake. |
tests/conformance/ |
Python pytest suite — capability-gated tests across all 11 namespaces; runs against any agent that speaks the protocol |
agents/windows-modern/tests/unit/ |
doctest unit tests for the pure-logic modules (framing, JSON, error codes, tier model) |
agents/windows-nt/ |
(planned) Legacy agent (NT 4 → Server 2003) — straight C, WinSock, GDI BitBlt |
mcp-server/ |
Python MCP bridge — exposes wire verbs as named tools to MCP-aware clients (Claude Code, Claude Desktop, …) with tier-aware tool filtering |
client/hostctl |
(planned) Reference Python CLI |
client/hostctl-discover |
(planned) mDNS LAN scanner |
Tools/install-agent.ps1 |
PowerShell installer — adds a Defender exclusion, copies the binary to %ProgramFiles%, adds binary-scoped firewall rules, registers a Task Scheduler logon-task with restart-on-failure. -Uninstall reverses it. |
Tools/scoop/ |
Scoop manifest reference + bucket-update notes. Published manifest lives at WilliamIsted/scoop-bucket. |
.github/workflows/release.yml |
Release CI — on v*.*.* tag push, builds windows-modern Release, packages remote-hands.exe + install script + docs into a zip, attaches the zip + SHA256SUMS to the GitHub Release. |
examples/vagrant/ |
(planned) Win11 dev fixture for VirtualBox / VMware / Hyper-V |
Three layers:
- Agent — a single binary per target OS family. Listens on TCP
8765(configurable). Runs in the user's interactive desktop session via Task Scheduler logon-task autostart so it can drive the visible UI. - Wire protocol — length-prefixed framing over plain TCP. 66 verbs across 11 namespaces (
screen,window,input,element,file,process,registry,clipboard,system,watch,connection). Three-tier permission model (observe/drive/power) with file-token elevation. Subscription-based streaming viawatch.*verbs and out-of-bandEVENTframes. Every agent advertises its capability set insystem.infoandsystem.capabilitiesso clients negotiate features without breaking on older or restricted builds. - MCP server (or any client) (planned) — bridges the wire protocol to a higher-level interface. The Python MCP server will expose named tools to Claude Code, filter them by what the agent advertises, and use description copy + MCP annotations to nudge the model toward semantic actions (
click_elementoverclick(x,y),find_elementover OCR).
Connections are per-thread, capped at the agent's advertised max_connections (default 4). watch.* subscriptions hold one connection for their duration; clients open side connections for interleaved commands.
When started with REMOTE_HANDS_DISCOVERABLE=1 (or --discoverable), the agent advertises itself on the LAN via mDNS / DNS-SD as _remote-hands._tcp.local.. Any DNS-SD browser will see it; a dedicated Python scanner is planned (see Roadmap).
The protocol has no built-in authentication today, so discovery is opt-in per deployment — advertising on an untrusted network is a footgun. The v0.4 milestone (Protocol 4.0 — see Roadmap) introduces SSPI authentication; until then, advertise only on trusted networks and consider firewalling 8765 to known clients.
tests/conformance/ runs against any agent on any platform. Each module exercises one namespace and gates by the agent's advertised capabilities — agents that don't implement a verb don't fail, they get the test skipped.
python tests/conformance/run.py <agent-host> 8765The suite is the executable contract: anything passing it speaks the wire protocol correctly.
Three architectural increments tracked as GitHub milestones:
| Milestone | Protocol | Theme | Notes |
|---|---|---|---|
| v0.2 | 2.0 | Stable protocol + per-connection tier system + agent-feedback fixes | First ratified Protocol 2.0. Three-tier model (observe / drive / power) gated by connection.tier_raise, file-token auth, and a backlog of observability + ergonomics improvements surfaced by real LLM driving sessions. Single-process. |
| v0.3 | 2.1 | CRUDX tier vocabulary + spec-driven schemas | Renames the wire tiers to a five-rung CRUDX ladder (read < create < update < delete < extra_risky); renames clipboard.read/write to clipboard.get/set; the bridge's tools/list schemas are now lifted from Repos/Protocol/spec/verbs/*.json. Wire-breaking; clean cut, no aliases. |
| v0.3 | 3.0 | Privsep dispatcher | Privileged dispatcher process + tier-restricted worker processes. OS-enforced separation: the kernel refuses out-of-tier operations regardless of agent code paths. Compromise containment. |
| v0.4 | 4.0 | SSPI auth + caller impersonation | Per-connection workers spawned under the authenticated caller's Windows identity. Filesystem and registry ACLs match the caller's permissions, not the agent's. Full WinRM-style auth model. |
Issues and PRs are welcome. Useful labels:
agent-feedback— surfaced by an LLM agent using the tool in real tasksagent-authored— issue body or PR text written by an LLM (provenance marker)high-priority/low-priority— direct cost vs. polish (no priority label = medium)- standard
bug/enhancement/documentation
When filing a wire-protocol change, please update PROTOCOL.md, add a conformance test, and run the suite against at least one target build before submitting.
The "Built with Claude Code" badge at the top is honest — a substantial portion of the source code, the v2 wire-protocol spec, and the test suites was drafted by Claude Code under direction. What the badge doesn't capture is the iteration: the architectural decisions, security-model trade-offs, naming conventions, and "wait, this is wrong" course-corrections that shaped the result. Those are human work, and there are many more hours of them than there are of "Claude wrote this and shipped it."
Treat the project as collaborative output, not autonomous agent work. Bug reports, architectural critiques, and PRs are welcome.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use the files in this repository except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0, or in the LICENSE file at the repo root.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright 2026 William Isted and contributors.