From f9857f0fb1b972552669d3c9812bfba7f1c50fcc Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 08:19:00 -0600 Subject: [PATCH 1/7] docs: adopt "computer use" terminology for the agent surface Anthropic (Claude) and OpenAI (Codex) both use "computer use" as the term for an AI agent that sees a screen and drives a keyboard/mouse. MCEC's agent surface is that same capability for Windows, so weave the phrase through the docs that introduce and describe it (README, docs site index, Agent Control, Agent Safety, AGENTS.md, Remote Control's cross-reference callout, install/configuration/security notes) rather than leaving it implicit or only in the one spot it already appeared. --- AGENTS.md | 3 ++- README.md | 10 ++++++---- SECURITY.md | 3 ++- dev/agent-server-architecture.md | 2 +- docs/agent_control.md | 9 +++++---- docs/configuration.md | 4 ++-- docs/index.md | 13 +++++++------ docs/install.md | 7 ++++--- docs/remote_control.md | 3 ++- docs/safety-emergency-stop-and-provisioning.md | 4 +++- 10 files changed, 34 insertions(+), 24 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7b6d8be..a2d1d13 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,8 @@ # AGENTS.md: driving (and testing) MCEC with an agent MCEC 3.0 (the **Model Context Environment Controller**) gives an AI agent eyes, hands, and a safe -front door on Windows. This file is the **self-reinforcing guidance loop**: the canonical guidance an +front door on Windows — Windows **computer use**, in the sense Claude, Codex, and similar agents use the +term. This file is the **self-reinforcing guidance loop**: the canonical guidance an agent needs, plus the recipe to *dogfood* it (drive MCEC through its own MCP server) so the guidance stays honest. Each time MCEC changes, re-run the dogfood and refine the guidance below and in `AgentServer.Instructions`. diff --git a/README.md b/README.md index c029a91..f253482 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,11 @@ By [Tig Kindel](https://twitter.com/tigkindel) - Copyright © [Kindel](http://ww agents on Windows, and the same battle-tested TCP/serial remote control for integration systems it has always been. -For agents, it is a small native Windows daemon a computer-use model **mounts, sees through, and drives** -over the **Model Context Protocol (MCP)**: capture windows, read UI Automation trees, find controls, launch -apps, and actuate keyboard/mouse input. For control systems, it listens on TCP/IP or a serial port and +For agents, it is a small native Windows daemon that gives a **computer use** model — the same capability +Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — something to +**mount, see through, and drive** over the **Model Context Protocol (MCP)**: capture windows, read UI +Automation trees, find controls, launch apps, and actuate keyboard/mouse input. For control systems, it +listens on TCP/IP or a serial port and translates remote commands into keystrokes, text, mouse moves, window messages, and app launches ([Control4](https://www.control4.com/), [Crestron](http://www.crestron.com/), [iRule](http://www.iruleathome.com/), and others). The 3.0 agent surface is **purely additive**; classic remote control is unchanged. @@ -39,7 +41,7 @@ Full guides live on the [docs site](https://tig.github.io/mcec/): * [Install](https://tig.github.io/mcec/install.html) — winget, what gets installed where, side-by-side copies * [Configuration](https://tig.github.io/mcec/configuration.html) — Settings, `mcec.settings`, commands, logging -* [Agent Control](https://tig.github.io/mcec/agent_control.html) — observation, targeting, actuation, MCP/HTTP +* [Agent Control](https://tig.github.io/mcec/agent_control.html) — the computer use surface: observation, targeting, actuation, MCP/HTTP * [Agent Safety](https://tig.github.io/mcec/safety-emergency-stop-and-provisioning.html) — consent, provisioning, emergency stop, overlay * [Remote Control](https://tig.github.io/mcec/remote_control.html) — TCP/serial commands and User Activity Monitor * [Examples](https://tig.github.io/mcec/examples.html) — worked agent-driving recipes diff --git a/SECURITY.md b/SECURITY.md index 6261606..eae6299 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -24,7 +24,8 @@ Security fixes target the latest 3.0.x release. Please reproduce on the current ## What is most valuable -MCEC's agent surface is **off by default** and gated behind explicit, independent opt-ins (see +MCEC's agent surface — its **computer use** capability — is **off by default** and gated behind explicit, +independent opt-ins (see [Agent Safety](docs/safety-emergency-stop-and-provisioning.md)). Reports are especially valuable when they show a way to: diff --git a/dev/agent-server-architecture.md b/dev/agent-server-architecture.md index bdef4cc..bccf92d 100644 --- a/dev/agent-server-architecture.md +++ b/dev/agent-server-architecture.md @@ -6,7 +6,7 @@ # Agent Server: Architecture Notes Developer doc ([`dev/`](README.md)); not published to GitHub Pages. A short tour of the MCEC 3.0 agent -subsystem. It lives under +subsystem — the machinery behind MCEC's **computer use** surface. It lives under `src/Agent/` and is **additive**: it reuses the existing `Command` / `CommandInvoker` pattern and adds no breaking changes to any existing command, transport, or default. diff --git a/docs/agent_control.md b/docs/agent_control.md index ec75826..824d76e 100644 --- a/docs/agent_control.md +++ b/docs/agent_control.md @@ -5,7 +5,8 @@ # Agent Control -MCEC 3.0 turns the MCE Controller daemon into a small, opt-in automation server for +MCEC 3.0 turns the MCE Controller daemon into a small, opt-in **computer use** server — the same +capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — for AI agents and scripts running on a Windows PC. It gives an agent three things: - **Eyes**: capture a screenshot of a window (or the foreground window) as a PNG. @@ -541,9 +542,9 @@ rest of the tree is returned. ## Using MCEC from a desktop agent app MCEC can run **headless** as an MCP **stdio** server (no main window, no tray icon; the -on-screen command overlay and the emergency-stop hotkey still work) so a desktop agent app -(such as a desktop AI assistant or custom MCP client) can spawn it on demand and talk to it over standard -input/output: +on-screen command overlay and the emergency-stop hotkey still work) so a desktop agent app with +computer use capability (such as a desktop AI assistant or custom MCP client) can spawn it on demand +and talk to it over standard input/output: ``` mcec.exe mcp # or the equivalent legacy spelling: mcec.exe --mcp diff --git a/docs/configuration.md b/docs/configuration.md index 1265b5a..f17ff0b 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -10,8 +10,8 @@ Everything MCEC does is configured in one of three places: the **File ▸ Settin running MCEC, every Settings tab, the agent gates in `mcec.settings`, enabling commands, and logging. For **installing** MCEC and where its files live, see [Install](install.md); for the **security model** -behind the agent gates, see [Agent Safety](safety-emergency-stop-and-provisioning.md); for the agent -tools themselves, see [Agent Control](agent_control.md). +behind the agent gates, see [Agent Safety](safety-emergency-stop-and-provisioning.md); for the computer +use tools themselves, see [Agent Control](agent_control.md). If you want to use MCEC from a desktop agent app, see [Agent Control](agent_control.md). In practice: enable diff --git a/docs/index.md b/docs/index.md index 97d28cf..66f884f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,11 +6,12 @@ agents on Windows, and the same battle-tested TCP/serial remote control for integration systems it has always been. -It is a small, self-contained native Windows daemon that a computer-use model can **mount, see through, -and drive**: capture a window as a PNG, read its UI Automation tree, find and wait for controls, launch -apps, and actuate keyboard/mouse/window input, over the **Model Context Protocol (MCP)**. The agent -surface is opt-in and off by default; the 3.0 agent features are purely additive over the classic -remote-control command surface (network and serial), which is unchanged. +It is a small, self-contained native Windows daemon that gives a **computer use** model — the same +capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — +something to **mount, see through, and drive**: capture a window as a PNG, read its UI Automation tree, +find and wait for controls, launch apps, and actuate keyboard/mouse/window input, over the **Model Context +Protocol (MCP)**. The agent surface is opt-in and off by default; the 3.0 agent features are purely +additive over the classic remote-control command surface (network and serial), which is unchanged. **Install with winget:** `winget install Kindel.mcec` — see [Install](install.html). @@ -20,7 +21,7 @@ remote-control command surface (network and serial), which is unchanged. running side-by-side copies, and disposable provisioned instances for agents. * **[Configuration](configuration.html)** — everything you can configure: the Settings dialog (every tab), the `mcec.settings` file, the command table, and logging. -* **[Agent Control](agent_control.html)** — the agent surface: the observation, +* **[Agent Control](agent_control.html)** — the computer use surface: the observation, targeting, and actuation tools, the structured result envelope, and the MCP / localhost-HTTP transports. * **[Agent Safety](safety-emergency-stop-and-provisioning.html)** — command-access consent, disposable isolated session provisioning, the emergency-stop hotkey, and the on-screen command overlay. diff --git a/docs/install.md b/docs/install.md index 3a6ee7f..1bd2f74 100644 --- a/docs/install.md +++ b/docs/install.md @@ -42,9 +42,10 @@ This directory-per-instance isolation is exactly what provisioning (below) autom ## Provisioning: disposable instances for agents The installed copy under Program Files deliberately **refuses to serve agents**: running `mcec.exe --mcp` -or starting the MCP/HTTP endpoint from Program Files is refused, because enabling the agent gates in your -installed config would leave them enabled if a session crashed. Instead, an authorized agent asks MCEC for -a fresh, disposable copy to drive (there is no `--provision` flag; provisioning is an in-product feature): +or starting the MCP/HTTP endpoint from Program Files is refused, because enabling the computer use gates +in your installed config would leave them enabled if a session crashed. Instead, an authorized agent asks +MCEC for a fresh, disposable copy to drive (there is no `--provision` flag; provisioning is an in-product +feature): 1. Turn on **Allow agents to provision disposable instances** on the Settings dialog's **Agent** tab (`AllowSessionProvisioning`). This is the one opt-in an operator performs. diff --git a/docs/remote_control.md b/docs/remote_control.md index 6d543a2..1323603 100644 --- a/docs/remote_control.md +++ b/docs/remote_control.md @@ -22,7 +22,8 @@ MCEC was originally built to integrate a Windows Media Center home-theater PC (H whole-house A/V system, but it is general enough for any control system that can send text to a TCP/IP port or serial port. -> **Looking for the agent/AI side?** MCEC 3.0 is also an opt-in automation server for AI agents over the +> **Looking for the agent/AI side?** MCEC 3.0 is also an opt-in **computer use** server — the same +> capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — over the > **Model Context Protocol (MCP)**: see the [Agent Control](agent_control.md) and > [Configuration](configuration.md) chapters. This page covers the classic remote-control features, which are > unchanged and remain the default behavior. diff --git a/docs/safety-emergency-stop-and-provisioning.md b/docs/safety-emergency-stop-and-provisioning.md index ce67d5e..90a50d2 100644 --- a/docs/safety-emergency-stop-and-provisioning.md +++ b/docs/safety-emergency-stop-and-provisioning.md @@ -5,7 +5,9 @@ # Agent Safety -Four related safety features for MCEC's agent surface. When MCEC is agent-driving the desktop, the **target +Four related safety features for MCEC's agent surface — MCEC's **computer use** capability, in the same +sense Claude, Codex, and similar agents use the term for seeing a screen and driving a keyboard and mouse. +When MCEC is agent-driving the desktop, the **target app has focus, not MCEC**; so the operator needs (a) a controlled way to grant more capability mid-session, (b) assurance that a session cannot leave the installed copy in an unsafe state, (c) a way to instantly intervene when a run goes wrong, and (d) visible narration that MCEC is driving. Command-access consent, From a1c7841d197bc4aa3cb4cdf554a1e79201ab1d38 Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 10:55:06 -0600 Subject: [PATCH 2/7] docs: drop em dashes from the computer-use terminology pass Replace the em dashes introduced in the prior commit with parentheses (prose asides) or plain hyphens/colons (bullet-point headings), and codify the house rule against em dashes in AGENTS.md so it isn't reintroduced. --- AGENTS.md | 7 +++++-- README.md | 16 ++++++++-------- SECURITY.md | 2 +- dev/agent-server-architecture.md | 2 +- docs/agent_control.md | 4 ++-- docs/index.md | 16 ++++++++-------- docs/remote_control.md | 4 ++-- docs/safety-emergency-stop-and-provisioning.md | 4 ++-- 8 files changed, 29 insertions(+), 26 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a2d1d13..9d86fcc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,8 +6,8 @@ # AGENTS.md: driving (and testing) MCEC with an agent MCEC 3.0 (the **Model Context Environment Controller**) gives an AI agent eyes, hands, and a safe -front door on Windows — Windows **computer use**, in the sense Claude, Codex, and similar agents use the -term. This file is the **self-reinforcing guidance loop**: the canonical guidance an +front door on Windows (Windows **computer use**, in the sense Claude, Codex, and similar agents use the +term). This file is the **self-reinforcing guidance loop**: the canonical guidance an agent needs, plus the recipe to *dogfood* it (drive MCEC through its own MCP server) so the guidance stays honest. Each time MCEC changes, re-run the dogfood and refine the guidance below and in `AgentServer.Instructions`. @@ -183,6 +183,9 @@ controller-bootstrap follow-ups. holds contributor and maintainer material only (CI, releasing, code signing, agent-server architecture notes, evidence bundles, doc-image regeneration recipes); it is not part of the site. **Release instructions:** [`dev/RELEASING.md`](dev/RELEASING.md) (cut from `main`, never `develop`). +- **No em dashes (—) in prose, anywhere in this repo.** Use parentheses or semicolons for an aside; for a + bullet-point heading (`* [Link](url) — description`) use a plain hyphen (`-`) or a colon (`:`) instead. + This applies to Markdown docs and code comments alike. - **Agent-facing guidance is part of "Done": not optional, not "later."** Any change to how an agent observes/targets/acts (a new tool, arg, failure mode, warning/error category, or driving technique) MUST update the connect-time playbook in diff --git a/README.md b/README.md index f253482..05749c0 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,8 @@ By [Tig Kindel](https://twitter.com/tigkindel) - Copyright © [Kindel](http://ww agents on Windows, and the same battle-tested TCP/serial remote control for integration systems it has always been. -For agents, it is a small native Windows daemon that gives a **computer use** model — the same capability -Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — something to +For agents, it is a small native Windows daemon that gives a **computer use** model (the same capability +Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) something to **mount, see through, and drive** over the **Model Context Protocol (MCP)**: capture windows, read UI Automation trees, find controls, launch apps, and actuate keyboard/mouse input. For control systems, it listens on TCP/IP or a serial port and @@ -39,12 +39,12 @@ MCP client setup, and teardown — is in [Agent Control → Quick start](https:/ Full guides live on the [docs site](https://tig.github.io/mcec/): -* [Install](https://tig.github.io/mcec/install.html) — winget, what gets installed where, side-by-side copies -* [Configuration](https://tig.github.io/mcec/configuration.html) — Settings, `mcec.settings`, commands, logging -* [Agent Control](https://tig.github.io/mcec/agent_control.html) — the computer use surface: observation, targeting, actuation, MCP/HTTP -* [Agent Safety](https://tig.github.io/mcec/safety-emergency-stop-and-provisioning.html) — consent, provisioning, emergency stop, overlay -* [Remote Control](https://tig.github.io/mcec/remote_control.html) — TCP/serial commands and User Activity Monitor -* [Examples](https://tig.github.io/mcec/examples.html) — worked agent-driving recipes +* [Install](https://tig.github.io/mcec/install.html): winget, what gets installed where, side-by-side copies +* [Configuration](https://tig.github.io/mcec/configuration.html): Settings, `mcec.settings`, commands, logging +* [Agent Control](https://tig.github.io/mcec/agent_control.html) - the computer use surface: observation, targeting, actuation, MCP/HTTP +* [Agent Safety](https://tig.github.io/mcec/safety-emergency-stop-and-provisioning.html): consent, provisioning, emergency stop, overlay +* [Remote Control](https://tig.github.io/mcec/remote_control.html): TCP/serial commands and User Activity Monitor +* [Examples](https://tig.github.io/mcec/examples.html): worked agent-driving recipes Developers and agents: [AGENTS.md](AGENTS.md) (connect-time guidance and the MCEC-drives-MCEC dogfood test). Contributor docs (CI, signing, architecture, image regeneration) live in [`dev/`](dev/). diff --git a/SECURITY.md b/SECURITY.md index eae6299..1779b0b 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -24,7 +24,7 @@ Security fixes target the latest 3.0.x release. Please reproduce on the current ## What is most valuable -MCEC's agent surface — its **computer use** capability — is **off by default** and gated behind explicit, +MCEC's agent surface (its **computer use** capability) is **off by default** and gated behind explicit, independent opt-ins (see [Agent Safety](docs/safety-emergency-stop-and-provisioning.md)). Reports are especially valuable when they show a way to: diff --git a/dev/agent-server-architecture.md b/dev/agent-server-architecture.md index bccf92d..27cc82a 100644 --- a/dev/agent-server-architecture.md +++ b/dev/agent-server-architecture.md @@ -6,7 +6,7 @@ # Agent Server: Architecture Notes Developer doc ([`dev/`](README.md)); not published to GitHub Pages. A short tour of the MCEC 3.0 agent -subsystem — the machinery behind MCEC's **computer use** surface. It lives under +subsystem (the machinery behind MCEC's **computer use** surface). It lives under `src/Agent/` and is **additive**: it reuses the existing `Command` / `CommandInvoker` pattern and adds no breaking changes to any existing command, transport, or default. diff --git a/docs/agent_control.md b/docs/agent_control.md index 824d76e..2100533 100644 --- a/docs/agent_control.md +++ b/docs/agent_control.md @@ -5,8 +5,8 @@ # Agent Control -MCEC 3.0 turns the MCE Controller daemon into a small, opt-in **computer use** server — the same -capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — for +MCEC 3.0 turns the MCE Controller daemon into a small, opt-in **computer use** server (the same +capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) for AI agents and scripts running on a Windows PC. It gives an agent three things: - **Eyes**: capture a screenshot of a window (or the foreground window) as a PNG. diff --git a/docs/index.md b/docs/index.md index 66f884f..93cb2c2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,8 +6,8 @@ agents on Windows, and the same battle-tested TCP/serial remote control for integration systems it has always been. -It is a small, self-contained native Windows daemon that gives a **computer use** model — the same -capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — +It is a small, self-contained native Windows daemon that gives a **computer use** model (the same +capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) something to **mount, see through, and drive**: capture a window as a PNG, read its UI Automation tree, find and wait for controls, launch apps, and actuate keyboard/mouse/window input, over the **Model Context Protocol (MCP)**. The agent surface is opt-in and off by default; the 3.0 agent features are purely @@ -17,21 +17,21 @@ additive over the classic remote-control command surface (network and serial), w ## The chapters -* **[Install](install.html)** — install with winget or the signed installer; what gets installed where, +* **[Install](install.html)**: install with winget or the signed installer; what gets installed where, running side-by-side copies, and disposable provisioned instances for agents. -* **[Configuration](configuration.html)** — everything you can configure: the Settings dialog (every +* **[Configuration](configuration.html)** - everything you can configure: the Settings dialog (every tab), the `mcec.settings` file, the command table, and logging. -* **[Agent Control](agent_control.html)** — the computer use surface: the observation, +* **[Agent Control](agent_control.html)** - the computer use surface: the observation, targeting, and actuation tools, the structured result envelope, and the MCP / localhost-HTTP transports. -* **[Agent Safety](safety-emergency-stop-and-provisioning.html)** — command-access consent, disposable +* **[Agent Safety](safety-emergency-stop-and-provisioning.html)**: command-access consent, disposable isolated session provisioning, the emergency-stop hotkey, and the on-screen command overlay. -* **[Remote Control](remote_control.html)** — the classic role: listen on TCP/IP or a +* **[Remote Control](remote_control.html)** - the classic role: listen on TCP/IP or a serial port and translate remote commands into keystrokes, text, mouse, window messages, and app launches ([Control4](https://www.control4.com/), [iRule](http://www.iruleathome.com/), [Crestron](http://www.crestron.com/), and others). The same chapter covers the **User Activity Monitor**, which runs the flow in reverse: it reports when someone is actively using the PC, turning the machine into an occupancy sensor that can drive lighting and scenes. -* **[Examples](examples.html)** — worked agent-driving recipes (hero GIFs, prompt demos) and how to add more. +* **[Examples](examples.html)**: worked agent-driving recipes (hero GIFs, prompt demos) and how to add more. See also [AGENTS.md](https://github.com/tig/mcec/blob/main/AGENTS.md) for connect-time agent guidance and the dogfood recipe (MCEC driving MCEC). diff --git a/docs/remote_control.md b/docs/remote_control.md index 1323603..8f78e7f 100644 --- a/docs/remote_control.md +++ b/docs/remote_control.md @@ -22,8 +22,8 @@ MCEC was originally built to integrate a Windows Media Center home-theater PC (H whole-house A/V system, but it is general enough for any control system that can send text to a TCP/IP port or serial port. -> **Looking for the agent/AI side?** MCEC 3.0 is also an opt-in **computer use** server — the same -> capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse — over the +> **Looking for the agent/AI side?** MCEC 3.0 is also an opt-in **computer use** server (the same +> capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) over the > **Model Context Protocol (MCP)**: see the [Agent Control](agent_control.md) and > [Configuration](configuration.md) chapters. This page covers the classic remote-control features, which are > unchanged and remain the default behavior. diff --git a/docs/safety-emergency-stop-and-provisioning.md b/docs/safety-emergency-stop-and-provisioning.md index 90a50d2..bf9c860 100644 --- a/docs/safety-emergency-stop-and-provisioning.md +++ b/docs/safety-emergency-stop-and-provisioning.md @@ -5,8 +5,8 @@ # Agent Safety -Four related safety features for MCEC's agent surface — MCEC's **computer use** capability, in the same -sense Claude, Codex, and similar agents use the term for seeing a screen and driving a keyboard and mouse. +Four related safety features for MCEC's agent surface (MCEC's **computer use** capability, in the same +sense Claude, Codex, and similar agents use the term for seeing a screen and driving a keyboard and mouse). When MCEC is agent-driving the desktop, the **target app has focus, not MCEC**; so the operator needs (a) a controlled way to grant more capability mid-session, (b) assurance that a session cannot leave the installed copy in an unsafe state, (c) a way to instantly From a3895e4afd75ae9d755f12b96d4bd356b9fb428c Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 11:00:55 -0600 Subject: [PATCH 3/7] docs: add FAQ section to the overview page Answers the "why not just use Claude/Copilot computer use", safety, and history questions that came up discussing this PR; the history answer reuses the existing Version history bullets. --- docs/index.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/docs/index.md b/docs/index.md index 93cb2c2..75f99a3 100644 --- a/docs/index.md +++ b/docs/index.md @@ -36,6 +36,42 @@ additive over the classic remote-control command surface (network and serial), w See also [AGENTS.md](https://github.com/tig/mcec/blob/main/AGENTS.md) for connect-time agent guidance and the dogfood recipe (MCEC driving MCEC). +## Frequently Asked Questions (FAQ) + +**Q: Claude, Copilot, and other agent platforms already support computer use. Why would I use MCEC?** + +A: Those platforms aim for cross-platform reach, so their computer use is built primarily on computer +vision: a screenshot, interpreted by a vision model, on every step. That is slow (a network round trip per +observation) and token-expensive (a full-window PNG on every turn). MCEC is Windows-only by design, so it +can go deeper than pixels: `query` reads the real Windows UI Automation tree (control type, name, +automation ID, bounds, enabled state, value), letting an agent target and verify a specific control instead +of re-parsing an image every step. `capture` (a real screenshot) is still there for when vision is the +right tool. And because MCEC speaks the **Model Context Protocol (MCP)**, it works with any MCP-capable +agent, Claude, Codex, or a custom client, so a workflow or automated system built on it isn't locked to one +AI provider. + +**Q: Is it safe?** + +A: It depends on how you use it, but the defaults are safe. The agent surface is off until you opt in; +every command ships individually disabled; every agent action is audit-logged on screen (the overlay) and +in the log; and a global emergency-stop hotkey lets the operator halt a session instantly. See +[Agent Safety](safety-emergency-stop-and-provisioning.html) for the full model. + +**Q: What's the history of this thing?** + +A: + +* **3.0 (2026)**: Rebranded to the **Model Context Environment Controller**. Agent automation over MCP: + observation (`capture`/`query`/`record`), targeting (`find`/`wait-for`), actuation + (`invoke`/`launch`/`drag`/`click`), emergency stop, and isolated session provisioning; all opt-in and + off by default. +* **2.x (2019)**: Major rework: robust client/server, User Activity Monitor (occupancy sensing), Commands + Window with built-in test mode, per-monitor DPI support, config in `%APPDATA%`. +* **1.x (2004–2017)**: Born as Media Center Edition Controller (MS later dropped the "Edition" from + "Windows Media Center") for Windows Media Center HTPCs. Grew keyboard/mouse/window-message + simulation, `chars:` with Unicode escapes, serial support, multi-client TCP, and the `.commands` + extension file. Moved from SourceForge to CodePlex to GitHub. + ## Version history * **3.0 (2026)**: Rebranded to the **Model Context Environment Controller**. Agent automation over MCP: From 99a552d5b32fba30915f7a8611d4645d8aeb4883 Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 11:05:12 -0600 Subject: [PATCH 4/7] docs: expand the Eyes bullet to cover MCEC's full observation surface --- docs/agent_control.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/agent_control.md b/docs/agent_control.md index 2100533..6211925 100644 --- a/docs/agent_control.md +++ b/docs/agent_control.md @@ -9,10 +9,15 @@ MCEC 3.0 turns the MCE Controller daemon into a small, opt-in **computer use** s capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) for AI agents and scripts running on a Windows PC. It gives an agent three things: -- **Eyes**: capture a screenshot of a window (or the foreground window) as a PNG. +- **Eyes**: far more than a screenshot. `capture` still takes a PNG of a window (or a screen + region), but `query` reads a window's live **UI Automation tree** (control type, name, + automation ID, bounds, enabled/offscreen state, value), `displays` reports per-monitor + geometry (bounds, DPI, scale), and `windows`/`find`/`wait-for` discover and wait on windows + and UI elements by name, class, or automation ID. An agent can see structured state, not + just pixels. - **Hands**: invoke any existing MCEC command (the actuation layer you already use). -- **A front door**: query/find windows and UI elements, wait for conditions, and - drive all of the above over **MCP** (Model Context Protocol) or a tiny **HTTP** floor. +- **A front door**: drive all of the above over **MCP** (Model Context Protocol) or a tiny + **HTTP** floor, gated off by default and audited when on. The agent surface is a set of new commands (`capture`, `query`, `displays`, `windows`, `window`, `find`, `wait-for`, `invoke`, `record`, `launch`, `drag`, `click`, and `focus`) exposed as **tools over MCP/HTTP** From 3dadea901d42ae7dd15b0c616a4239dc532fd012 Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 11:08:22 -0600 Subject: [PATCH 5/7] docs: rename Agent Control heading to Agent Control (Computer Use) --- docs/agent_control.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/agent_control.md b/docs/agent_control.md index 6211925..f5bfd11 100644 --- a/docs/agent_control.md +++ b/docs/agent_control.md @@ -3,7 +3,7 @@ // Published under the MIT License - Source on GitHub: https://github.com/tig/mcec --> -# Agent Control +# Agent Control (Computer Use) MCEC 3.0 turns the MCE Controller daemon into a small, opt-in **computer use** server (the same capability Claude, Codex, and similar agents use to see a screen and drive a keyboard and mouse) for From ace0ae869d45eaeb1bab4b9559a1db2e33b3134e Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 11:10:09 -0600 Subject: [PATCH 6/7] docs: drop A: prefixes in FAQ and remove duplicate Version history section --- docs/index.md | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/docs/index.md b/docs/index.md index 75f99a3..99b3d93 100644 --- a/docs/index.md +++ b/docs/index.md @@ -40,7 +40,7 @@ the dogfood recipe (MCEC driving MCEC). **Q: Claude, Copilot, and other agent platforms already support computer use. Why would I use MCEC?** -A: Those platforms aim for cross-platform reach, so their computer use is built primarily on computer +Those platforms aim for cross-platform reach, so their computer use is built primarily on computer vision: a screenshot, interpreted by a vision model, on every step. That is slow (a network round trip per observation) and token-expensive (a full-window PNG on every turn). MCEC is Windows-only by design, so it can go deeper than pixels: `query` reads the real Windows UI Automation tree (control type, name, @@ -52,28 +52,13 @@ AI provider. **Q: Is it safe?** -A: It depends on how you use it, but the defaults are safe. The agent surface is off until you opt in; +It depends on how you use it, but the defaults are safe. The agent surface is off until you opt in; every command ships individually disabled; every agent action is audit-logged on screen (the overlay) and in the log; and a global emergency-stop hotkey lets the operator halt a session instantly. See [Agent Safety](safety-emergency-stop-and-provisioning.html) for the full model. **Q: What's the history of this thing?** -A: - -* **3.0 (2026)**: Rebranded to the **Model Context Environment Controller**. Agent automation over MCP: - observation (`capture`/`query`/`record`), targeting (`find`/`wait-for`), actuation - (`invoke`/`launch`/`drag`/`click`), emergency stop, and isolated session provisioning; all opt-in and - off by default. -* **2.x (2019)**: Major rework: robust client/server, User Activity Monitor (occupancy sensing), Commands - Window with built-in test mode, per-monitor DPI support, config in `%APPDATA%`. -* **1.x (2004–2017)**: Born as Media Center Edition Controller (MS later dropped the "Edition" from - "Windows Media Center") for Windows Media Center HTPCs. Grew keyboard/mouse/window-message - simulation, `chars:` with Unicode escapes, serial support, multi-client TCP, and the `.commands` - extension file. Moved from SourceForge to CodePlex to GitHub. - -## Version history - * **3.0 (2026)**: Rebranded to the **Model Context Environment Controller**. Agent automation over MCP: observation (`capture`/`query`/`record`), targeting (`find`/`wait-for`), actuation (`invoke`/`launch`/`drag`/`click`), emergency stop, and isolated session provisioning; all opt-in and From 8440831985878000361b1c9d651a643c64864e0a Mon Sep 17 00:00:00 2001 From: Tig Date: Fri, 10 Jul 2026 11:10:45 -0600 Subject: [PATCH 7/7] docs: add FAQ section to README --- README.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/README.md b/README.md index 05749c0..78a0e84 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,40 @@ Full guides live on the [docs site](https://tig.github.io/mcec/): Developers and agents: [AGENTS.md](AGENTS.md) (connect-time guidance and the MCEC-drives-MCEC dogfood test). Contributor docs (CI, signing, architecture, image regeneration) live in [`dev/`](dev/). +## Frequently Asked Questions (FAQ) + +**Q: Claude, Copilot, and other agent platforms already support computer use. Why would I use MCEC?** + +Those platforms aim for cross-platform reach, so their computer use is built primarily on computer +vision: a screenshot, interpreted by a vision model, on every step. That is slow (a network round trip per +observation) and token-expensive (a full-window PNG on every turn). MCEC is Windows-only by design, so it +can go deeper than pixels: `query` reads the real Windows UI Automation tree (control type, name, +automation ID, bounds, enabled state, value), letting an agent target and verify a specific control instead +of re-parsing an image every step. `capture` (a real screenshot) is still there for when vision is the +right tool. And because MCEC speaks the **Model Context Protocol (MCP)**, it works with any MCP-capable +agent, Claude, Codex, or a custom client, so a workflow or automated system built on it isn't locked to one +AI provider. + +**Q: Is it safe?** + +It depends on how you use it, but the defaults are safe. The agent surface is off until you opt in; +every command ships individually disabled; every agent action is audit-logged on screen (the overlay) and +in the log; and a global emergency-stop hotkey lets the operator halt a session instantly. See +[Agent Safety](https://tig.github.io/mcec/safety-emergency-stop-and-provisioning.html) for the full model. + +**Q: What's the history of this thing?** + +* **3.0 (2026)**: Rebranded to the **Model Context Environment Controller**. Agent automation over MCP: + observation (`capture`/`query`/`record`), targeting (`find`/`wait-for`), actuation + (`invoke`/`launch`/`drag`/`click`), emergency stop, and isolated session provisioning; all opt-in and + off by default. +* **2.x (2019)**: Major rework: robust client/server, User Activity Monitor (occupancy sensing), Commands + Window with built-in test mode, per-monitor DPI support, config in `%APPDATA%`. +* **1.x (2004–2017)**: Born as Media Center Edition Controller (MS later dropped the "Edition" from + "Windows Media Center") for Windows Media Center HTPCs. Grew keyboard/mouse/window-message + simulation, `chars:` with Unicode escapes, serial support, multi-client TCP, and the `.commands` + extension file. Moved from SourceForge to CodePlex to GitHub. + ## Integrations * [Control4 User Activity Driver](https://github.com/tig/User_Activity)