Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ obj/
*.user
*.userprefs

# Swift Package Manager output (used by apps/agent-runner/bridges/macos)
.build/
.swiftpm/
Packages/
*.xcodeproj
DerivedData/

# JetBrains
.idea/
*.iml
Expand Down
29 changes: 29 additions & 0 deletions apps/agent-runner/bridges/macos/Package.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// swift-tools-version:5.9
//
// AgentMark macOS bridge. Mirrors the Windows bridge architecture —
// stdio JSON-RPC 2.0 server, Node-side `MacosAxapiBackend` spawns this
// and proxies capture/execute calls. Phase 0e'1 here ships ping +
// capabilities; AXAPI capture/execute land in 0e'2/0e'3.

import PackageDescription

let package = Package(
name: "AgentMarkBridgeMacos",
platforms: [
// AXAPI requires macOS; setting a recent baseline keeps the
// Accessibility APIs we'll need (kAXMainAttribute etc.) safe.
.macOS(.v13),
],
products: [
.executable(
name: "agentmark-bridge-macos",
targets: ["AgentMarkBridgeMacos"]
),
],
targets: [
.executableTarget(
name: "AgentMarkBridgeMacos",
path: "Sources/AgentMarkBridgeMacos"
),
]
)
96 changes: 96 additions & 0 deletions apps/agent-runner/bridges/macos/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# AgentMark macOS AXAPI Bridge

The macOS-side sidecar process for AgentMark Desktop. Mirrors the
Windows UIA bridge — same stdio JSON-RPC 2.0 protocol, same
`DesktopCaptureBackend` contract on the Node side. Uses the system
Accessibility framework (AXAPI) to walk and drive native macOS app UIs.

## Build

Requires the Swift toolchain (ships with Xcode Command Line Tools, or
`xcode-select --install`). From this directory:

```bash
swift build
```

Output binary: `.build/debug/agentmark-bridge-macos`.

For a release build with optimisations:

```bash
swift build -c release
# Binary at .build/release/agentmark-bridge-macos
```

## Run + smoke test

The bridge speaks stdio JSON-RPC 2.0. One JSON message per line.

```bash
./scripts/smoke-test.sh
```

That pipes two requests (`ping` + `capabilities`) into the binary,
validates the responses, prints the bridge's stderr diagnostics.

For interactive testing:

```bash
.build/debug/agentmark-bridge-macos
```

It blocks on stdin. Paste:

```
{"jsonrpc":"2.0","id":1,"method":"ping"}
```

Expected (single line):

```json
{"jsonrpc":"2.0","id":1,"result":{"pong":true,"version":"0.4.0","arch":"arm64","processId":12345}}
```

Ctrl-D closes stdin and the bridge exits cleanly.

## Protocol

All responses are JSON-RPC 2.0. Diagnostic output goes to stderr only —
stdout is reserved for framed JSON messages.

| Method | Status | Description |
|---|---|---|
| `ping` | ✅ this phase | Liveness check, returns pong + bridge version |
| `capabilities` | ✅ this phase | Lists supported methods + AXAPI provider info |
| `list_windows` | planned | Enumerate top-level AXAPI windows via NSWorkspace + AXUIElement |
| `capture` | planned | Walk a window's AXAPI tree → `DesktopCapture` JSON |
| `execute` | planned | Drive an action by element_id (AXPress / AXSetValue / etc.) |

The protocol — request/response envelopes, error codes, BOM-stripping —
is byte-identical to the Windows bridge. The Node-side
`MacosAxapiBackend` will be a near-exact mirror of `WindowsUiaBackend`.

## Accessibility permission

macOS guards AXAPI behind **System Settings → Privacy & Security →
Accessibility**. When the bridge starts walking trees (Phase 0e'2+),
the parent process (or this binary, if launched directly) must be
granted. The bridge will probe `AXIsProcessTrusted()` and surface a
clear `accessibilityNotGranted` JSON-RPC error if not.

For developer machines: grant the terminal you launch from (Terminal /
iTerm / Warp). For production / Claude Desktop integration: Claude
Desktop itself must be granted (it's the parent process that spawns
this bridge).

## Architecture

- **Single-threaded.** AXAPI calls are not strictly main-thread-only but
CFRunLoop integration matters; for capture/execute we'll route through
a dedicated serial DispatchQueue. For ping/capabilities the entry-point
thread is fine.
- **UTF-8 only.** stdin / stdout treated as UTF-8 throughout; BOM stripped
defensively from the first stdin write.
- **stderr for diagnostics, stdout for framed JSON.** Same discipline as
the Windows bridge so log aggregation works in the AgentMark MCP server.
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// Method routing. New methods land here; the same name/shape as the
// Windows bridge so the Node-side `MacosAxapiBackend` (Phase 0f') can
// mirror `WindowsUiaBackend` exactly.

import Foundation

final class Dispatcher {
private let bridgeVersion: String

init(bridgeVersion: String) {
self.bridgeVersion = bridgeVersion
}

func dispatch(request: JsonRpcRequest) throws -> Any {
switch request.method {
case "ping":
return handlePing()
case "capabilities":
return handleCapabilities()
default:
throw RpcError(
code: .methodNotFound,
message: "Unknown method: \(request.method). Supported: ping, capabilities.",
requestId: request.id
)
}
}

private func handlePing() -> Any {
return [
"pong": true,
"version": bridgeVersion,
"arch": currentArchString(),
"processId": ProcessInfo.processInfo.processIdentifier,
] as [String: Any]
}

private func handleCapabilities() -> Any {
return [
"bridge": "agentmark-bridge-macos",
"version": bridgeVersion,
"methods": ["ping", "capabilities"],
"axapiProvider": "Accessibility (AXAPI)",
"platform": "macos",
] as [String: Any]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
// JSON-RPC 2.0 wire types. Identical envelope to the Windows bridge.

import Foundation

// MARK: - Request

/// JSON-RPC 2.0 request. We accept any JSON value for `id` (number,
/// string, or null); store it raw and echo it back unmodified on the
/// response so different clients (number ids, uuid string ids) all work.
struct JsonRpcRequest {
let id: JsonRpcId
let method: String
let params: JsonValue?

static func decode(jsonString: String) throws -> JsonRpcRequest {
guard let data = jsonString.data(using: .utf8) else {
throw RpcError(code: RpcErrorCode.parseError, message: "Input is not valid UTF-8")
}
guard let any = try? JSONSerialization.jsonObject(with: data, options: [.allowFragments]) else {
throw RpcError(code: RpcErrorCode.parseError, message: "Input is not valid JSON")
}
guard let obj = any as? [String: Any] else {
throw RpcError(code: RpcErrorCode.invalidRequest, message: "JSON-RPC request must be an object")
}
guard let method = obj["method"] as? String, !method.isEmpty else {
throw RpcError(code: RpcErrorCode.invalidRequest, message: "Missing `method`")
}
let id = JsonRpcId.fromAny(obj["id"])
let params: JsonValue? = obj["params"].flatMap(JsonValue.fromAny)
return JsonRpcRequest(id: id, method: method, params: params)
}
}

// MARK: - Response

struct JsonRpcResponse {
let id: JsonRpcId
let kind: Kind

enum Kind {
case success(result: Any)
case error(code: Int, message: String)
}

static func success(id: JsonRpcId, result: Any) -> JsonRpcResponse {
JsonRpcResponse(id: id, kind: .success(result: result))
}

static func error(id: JsonRpcId, code: Int, message: String) -> JsonRpcResponse {
JsonRpcResponse(id: id, kind: .error(code: code, message: message))
}

func encode() throws -> String {
var dict: [String: Any] = [
"jsonrpc": "2.0",
"id": id.toAny(),
]
switch kind {
case .success(let result):
dict["result"] = result
case .error(let code, let message):
dict["error"] = [
"code": code,
"message": message,
]
}
let data = try JSONSerialization.data(
withJSONObject: dict,
options: [.withoutEscapingSlashes]
)
guard let s = String(data: data, encoding: .utf8) else {
throw RpcError(code: RpcErrorCode.internalError, message: "Failed to serialise response as UTF-8")
}
return s
}
}

// MARK: - JsonRpcId

/// JSON-RPC ids can be number, string, or null. Hold the original
/// representation so we round-trip exactly.
enum JsonRpcId {
case number(Double)
case string(String)
case null

static func fromAny(_ value: Any?) -> JsonRpcId {
if value == nil { return .null }
if let v = value as? NSNumber {
// NSNumber covers Int/Double/Bool from JSONSerialization
if CFGetTypeID(v) == CFBooleanGetTypeID() { return .null }
return .number(v.doubleValue)
}
if let s = value as? String { return .string(s) }
return .null
}

func toAny() -> Any {
switch self {
case .number(let d):
// Emit ints as ints when possible so clients see id=1, not id=1.0.
if d.truncatingRemainder(dividingBy: 1) == 0
&& d >= Double(Int.min) && d <= Double(Int.max) {
return Int(d)
}
return d
case .string(let s):
return s
case .null:
return NSNull()
}
}
}

// MARK: - JsonValue

/// Lightweight wrapper for arbitrary JSON values plucked from the
/// request's `params` field. Most handlers project the few fields they
/// need out via the `dict`/`string(at:)` etc. helpers.
enum JsonValue {
case object([String: Any])
case array([Any])
case string(String)
case number(Double)
case bool(Bool)
case null

static func fromAny(_ value: Any) -> JsonValue {
if let d = value as? [String: Any] { return .object(d) }
if let a = value as? [Any] { return .array(a) }
if let s = value as? String { return .string(s) }
if let n = value as? NSNumber {
if CFGetTypeID(n) == CFBooleanGetTypeID() { return .bool(n.boolValue) }
return .number(n.doubleValue)
}
return .null
}

var dict: [String: Any]? {
if case .object(let d) = self { return d }
return nil
}

func string(_ key: String) -> String? {
return (dict?[key] as? String)
}

func int(_ key: String) -> Int? {
if let n = dict?[key] as? NSNumber { return n.intValue }
return nil
}

func bool(_ key: String) -> Bool? {
if let n = dict?[key] as? NSNumber,
CFGetTypeID(n) == CFBooleanGetTypeID() {
return n.boolValue
}
return nil
}
}

// MARK: - Errors

enum RpcErrorCode: Int {
case parseError = -32700
case invalidRequest = -32600
case methodNotFound = -32601
case invalidParams = -32602
case internalError = -32603

// Bridge-specific (above -32000)
case windowNotFound = -32010
case elementNotFound = -32011
case unsupportedPattern = -32012
case actionFailed = -32013
case accessibilityNotGranted = -32020
}

struct RpcError: Error {
let code: Int
let message: String
var requestId: JsonRpcId = .null

init(code: RpcErrorCode, message: String, requestId: JsonRpcId = .null) {
self.code = code.rawValue
self.message = message
self.requestId = requestId
}
}
Loading
Loading