G4 Recorders — UI Automation & Chromium Recording Tools

What is this?
The Two Tools at a Glance
Quick Start
- Download and Run
- Verify the Services
Recording Event Contract (shared)
UIA Recorder — Desktop Recorder
Chromium Recorder — Browser Recorder
SignalR Client Examples
G4.Recorders.ClientApp — Monitor Both Hubs
G4 Recorders Path Finder — Quick Start
License

What is this?

G4 Recorders is a growing suite of recording tools that share one event contract. It currently ships two recorders, with Android and iOS recorders planned to extend the same contract to mobile platforms:

UIA Recorder (G4.Recorders.Uia) — a Windows UI Automation inspection and recording tool. It peeks at desktop UI elements (returning their ancestor chain, like an XPath for Windows UI) and records global keyboard and mouse events enriched with UI context.
Chromium Recorder (G4.Recorders.Chromium) — a browser recording tool. A bundled Chromium recorder extension captures in-page interactions (with a DOM locator chain) and streams them to a hub, which relays them to consumers.

Every recorder in the suite broadcasts the same ReceiveRecordingEvent message over SignalR, so a consumer can watch interactions across desktop, browser, and (in the future) mobile platforms through a single, uniform contract.

Use cases:

Debug and inspect desktop applications and web pages to troubleshoot UI behavior.
Build automated UI tests or robotic process automation (RPA) workflows.
Record user input with UI context for replay or analysis.
Monitor UI and user actions in real time.

The Two Tools at a Glance

Tool	Records	Hub port	How recording starts	REST surface
UIA Recorder	Windows desktop (global input)	`9955`	Automatically (always-on capture)	`peek` + `ping`
Chromium Recorder	Chromium browser (via extension)	`9956`	When a recorder browser is running	`ping` only

Both hubs share the path /hub/v4/g4/peek and emit the same ReceiveRecordingEvent payload (see Recording Event Contract).

Quick Start

Requirements: .NET 10 Runtime/SDK. UIA Recorder requires Windows (and Administrator privileges are recommended for global input hooks). Chromium Recorder additionally requires a Chromium/Chrome browser with the recorder extension loaded.

Download and Run

Go to Releases and download the latest artifact: 👉 Releases: https://github.com/g4-api/g4-recorders/releases
Unzip the archive to a folder with write permissions (e.g., C:\Tools\G4.Recorders).
Run the executable(s) from that folder:
- G4.Recorders.Uia.exe — desktop recorder + peek API (listens on 9955). Run as administrator for global hooks.
- G4.Recorders.Chromium.exe — browser recorder hub (listens on 9956).

Verify the Services

Each service exposes a Swagger UI and a health-check endpoint:

# UIA Recorder
curl http://localhost:9955/api/v4/g4/ping
# Output: Pong

# Chromium Recorder
curl http://localhost:9956/api/v4/g4/ping
# Output: Pong

UIA Recorder Swagger: http://localhost:9955/swagger
Chromium Recorder Swagger: http://localhost:9956/swagger

CORS tip: If you call from a webview or browser, ensure your origin is allowed via the ORIGINS env var or AllowedOrigins config. VS Code webviews (vscode-webview://) and the Chromium recorder extension (chrome-extension://) are already handled.

Recording Event Contract (shared)

Both tools broadcast recorded interactions to connected clients using the SignalR message ReceiveRecordingEvent. The payload is always wrapped in a { "value": { ... } } envelope with this shape:

chain — the element chain (locator + ordered path of nodes) where the event occurred.
event — the specific event name (e.g., Down, Click).
machineName — the machine that recorded the event.
offset — the pointer distance from the target rectangle's top-left corner, defaulting to { "x": 0, "y": 0 }.
timestamp — Unix epoch milliseconds.
type — the event category (e.g., Keyboard, Mouse).
value — the event payload (e.g., the pressed key, or coordinates).

Sample ReceiveRecordingEvent payload (UIA Recorder keyboard event):

{
    "value": {
        "chain": {
            "locator": "/Desktop/Window[@Name='...']/Document[@Name='Text Area']",
            "path": [
                {
                    "automationId": "Console Window",
                    "className": "ConsoleWindowClass",
                    "controlTypeId": 50032,
                    "controlType": "Window",
                    "isTopWindow": true,
                    "isTriggerElement": false,
                    "name": "...exe",
                    "processId": 31036
                },
                {
                    "automationId": "Text Area",
                    "bounds": { "height": 519, "X": 104, "Y": 104, "width": 993 },
                    "controlTypeId": 50030,
                    "controlType": "Document",
                    "isTopWindow": false,
                    "isTriggerElement": true,
                    "name": "Text Area",
                    "processId": 31036
                }
            ],
            "trigger": "Focus"
        },
        "event": "Down",
        "offset": { "x": 0, "y": 0 },
        "timestamp": 1757618366611,
        "type": "Keyboard",
        "value": { "scanCode": 30, "virtualKey": 65, "key": "a" }
    }
}

Note: The path array is ordered top-down. The last element is always the target element (the trigger) and carries more metadata than its ancestors to keep the payload compact. Chromium Recorder events use the same top-level shape; only the chain node contents differ (DOM elements instead of UIA elements).

For UIA mouse events, offset.x is the captured pointer X coordinate minus the trigger bounds' X, and offset.y is the pointer Y coordinate minus the trigger bounds' Y. Chromium events currently retain the shared zero default until DOM-relative offset capture is enabled.

UIA Recorder — Desktop Recorder

UIA Recorder listens on http://localhost:9955. Recording events flow automatically as soon as the service is running — there is no session to start; move the mouse or type and events are broadcast to every connected client.

Command-Line Usage

# Peek at focused element
G4.Recorders.Uia peek -f

# Peek at specific coordinates
G4.Recorders.Uia peek -x 100 -y 200

cURL API Usage

The RecorderController provides REST endpoints to peek at UI elements (UIA Recorder only).

# Peek at specific coordinates (x=250, y=300)
curl "http://localhost:9955/api/v4/g4/peek?x=250&y=300"

# Peek at the currently focused element
curl "http://localhost:9955/api/v4/g4/peek?focused=true"

The response is the ancestor chain object (same shape as the chain field in a recording event).

SignalR Hub Usage

Hub URL: http://localhost:9955/hub/v4/g4/peek

Client → Server Methods

SendHeartbeat()
SendPeekAt({ xPos, yPos })
SendPeekFocused()

Server → Client Events

ReceiveHeartbeat — heartbeat confirmation
ReceivePeek — ancestor chain result
ReceiveRecordingEvent — keyboard/mouse events with UI context (broadcast automatically)

See SignalR Client Examples for Python, JavaScript, and C# clients.

Chromium Recorder — Browser Recorder

Chromium Recorder listens on http://localhost:9956. Unlike UIA Recorder, it does not capture anything by itself — a Chromium recorder extension running in a browser produces the events.

How It Works

A client asks Chromium Recorder to launch a browser (StartRecorder), or you start a browser that already has the recorder extension loaded.
The extension connects to the hub and pushes in-page interactions via SendRecordingEvent.
The hub re-broadcasts each interaction to every connected consumer as ReceiveRecordingEvent — the same message and envelope UIA Recorder uses.
On StopRecorder, the hub asks the extension to close the browser gracefully (a CloseBrowser message), then force-stops the process if it does not exit in time.

The extension has its own setup and usage guide: src/G4.Recorders.Chromium.Extension/README.md.

SignalR Hub Usage (Chromium Recorder)

Hub URL: http://localhost:9956/hub/v4/g4/peek

Client → Server Methods

StartRecorder(driverParameters) → returns the launched browser process id
StopRecorder(processId) → returns true if a tracked browser was stopped, false if the id is unknown/already closed
SendHeartbeat()
SendRecordingEvent(event) — used by the extension (producer); consumers do not call this

Server → Client Events

ReceiveHeartbeat — heartbeat confirmation
ReceiveRecordingEvent — browser interactions with a DOM chain
CloseBrowser — sent to the extension to close the browser during a graceful stop

No REST peek: Chromium Recorder is controlled entirely over SignalR. The only REST endpoint is the ping health check.

Launching and Stopping a Browser

StartRecorder takes a W3C-style driver-parameters object; only the Chromium binary and optional arguments are used:

{
    "capabilities": {
        "alwaysMatch": {
            "goog:chromeOptions": {
                "binary": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
                "args": ["--disable-gpu"]
            }
        }
    }
}

Each launch gets its own dedicated user-data directory, so it never collides with an already-running browser. Keep the returned process id to stop it later with StopRecorder(processId).

Health Check

curl http://localhost:9956/api/v4/g4/ping
# Output: Pong

SignalR Client Examples

The listening pattern is identical for both hubs — only the URL changes (9955 for UIA Recorder, 9956 for Chromium Recorder). The Chromium-only StartRecorder/StopRecorder calls are shown alongside.

Python Example

Using the community client signalrcore.

pip install signalrcore

from signalrcore.hub_connection_builder import HubConnectionBuilder

# Use 9955 for UIA Recorder or 9956 for Chromium Recorder.
hub = (
    HubConnectionBuilder()
    .with_url("http://localhost:9955/hub/v4/g4/peek")
    .with_automatic_reconnect({"type": "raw", "keep_alive_interval": 10, "reconnect_interval": 5})
    .build()
)

def on_heartbeat(payload):
    print("Heartbeat:", payload)

def on_peek(payload):
    print("Peek:", payload)

def on_recording_event(payload):
    event = (payload[0] if isinstance(payload, list) else payload).get("value", {})
    print(f"RecordingEvent: type={event.get('type')} event={event.get('event')} value={event.get('value')}")

hub.on("ReceiveHeartbeat", on_heartbeat)
hub.on("ReceivePeek", on_peek)
hub.on("ReceiveRecordingEvent", on_recording_event)

hub.start()

# UIA Recorder: events flow automatically. Optional peek/heartbeat calls:
hub.send("SendHeartbeat", [])
hub.send("SendPeekFocused", [])

# Chromium Recorder only: launch a recorder browser (events start once it is recording).
# hub.send("StartRecorder", [{
#     "capabilities": {"alwaysMatch": {"goog:chromeOptions": {
#         "binary": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe", "args": []
#     }}}
# }])

input("Press <Enter> to stop...\n")
hub.stop()

JavaScript Example

Using @microsoft/signalr (browser or Node.js).

npm i @microsoft/signalr

import * as signalR from "@microsoft/signalr";

// Use 9955 for UIA Recorder or 9956 for Chromium Recorder.
const connection = new signalR.HubConnectionBuilder()
    .withUrl("http://localhost:9955/hub/v4/g4/peek")
    .withAutomaticReconnect()
    .build();

connection.on("ReceiveHeartbeat", (payload) => console.log("Heartbeat:", payload));
connection.on("ReceivePeek", (payload) => console.log("Peek chain:", payload?.value));
connection.on("ReceiveRecordingEvent", (payload) => {
    const event = payload?.value || {};
    console.log("RecordingEvent:", event.type, event.event, event.value);
});

async function main() {
    await connection.start();

    // UIA Recorder: events flow automatically. Optional peek/heartbeat:
    await connection.invoke("SendHeartbeat");
    await connection.invoke("SendPeekFocused");

    // Chromium Recorder only: launch and later stop a recorder browser.
    // const processId = await connection.invoke("StartRecorder", {
    //     capabilities: { alwaysMatch: { "goog:chromeOptions": {
    //         binary: "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe", args: []
    //     } } }
    // });
    // await connection.invoke("StopRecorder", processId);
}

main().catch(console.error);

C# Example

Using Microsoft.AspNetCore.SignalR.Client.

dotnet add package Microsoft.AspNetCore.SignalR.Client

using Microsoft.AspNetCore.SignalR.Client;
using System.Text.Json;

// Use 9955 for UIA Recorder or 9956 for Chromium Recorder.
var connection = new HubConnectionBuilder()
    .WithUrl("http://localhost:9955/hub/v4/g4/peek")
    .WithAutomaticReconnect()
    .Build();

connection.On<JsonElement>("ReceiveRecordingEvent", envelope =>
{
    var recordingEvent = envelope.GetProperty("value");
    var type = recordingEvent.GetProperty("type").GetString();
    var name = recordingEvent.GetProperty("event").GetString();
    Console.WriteLine($"RecordingEvent: {type} / {name}");
});

await connection.StartAsync();

// UIA Recorder: events flow automatically. Optional peek/heartbeat:
await connection.InvokeAsync("SendHeartbeat");
await connection.InvokeAsync("SendPeekFocused");

// Chromium Recorder only: launch and later stop a recorder browser.
// var driverParameters = new
// {
//     capabilities = new
//     {
//         alwaysMatch = new Dictionary<string, object?>
//         {
//             ["goog:chromeOptions"] = new { binary = @"C:\Program Files\Google\Chrome\Application\chrome.exe", args = Array.Empty<string>() }
//         }
//     }
// };
// var processId = await connection.InvokeAsync<int>("StartRecorder", driverParameters);
// var isStopped = await connection.InvokeAsync<bool>("StopRecorder", processId);

Console.WriteLine("Press <Enter> to exit...");
Console.ReadLine();

G4.Recorders.ClientApp — Monitor Both Hubs

G4.Recorders.ClientApp (src/G4.Recorders.ClientApp) is a .NET 10 console test harness that connects to both hubs at once and prints every recorder event to the console. It is the quickest way to watch desktop and browser events side by side.

cd src/G4.Recorders.ClientApp
dotnet run                # compact colored line per event
dotnet run -- --verbose   # full JSON per event

Choose sources — toggle each hub in appsettings.json (Hubs:UiaEnabled / Hubs:ChromiumEnabled) or at the command line (--no-uia, --no-chromium, --uia, --chromium).
Drive Chromium — set Chromium:BinaryPath in appsettings.json, then press s to launch a recorder browser and x to stop it.
Keys — s start Chromium · x stop Chromium · v toggle verbose · h help · q quit.

UIA events appear as soon as you move/click/type; Chromium events appear once a recorder browser is running.

G4 Recorders Path Finder — Quick Start

G4 Recorders Path Finder shows the UI-Automation path (an XPath-like locator) for whatever is currently under your mouse pointer. Hover any app control; the locator appears in the text box for copy/paste into your automation or QA tools.

Requirements

Windows desktop.
If you need to peek elevated apps, run Path Finder as Administrator.

Launch

Open G4 Recorders Path Finder v1.0 by running G4.Recorders.Uia.PathFinder.exe.

You’ll see a title, a locator text box, a Start/Stop button, and a Faster/Slower slider.

╭── G4 Recorders — Title ───────────────────────────────╮
│                                                       │
│  Locator                                              │
│  ┌─────────────────────────────────────────────────┐  │
│  │                                                 │  │
│  └─────────────────────────────────────────────────┘  │
│                                                       │
│  [ ▶ Start / ⬛ Stop]                                │
│                                                       │
│  Faster  ◄───────┈┈┈┈┈●┈┈┈┈────────►  Slower          │
│                                                       │
╰───────────────────────────────────────────────────────╯

Start peeking

Click ▶ Start (or press Alt+S).
Move your mouse over the target application.
The locator appears and updates in the text box.

Adjust update speed

Use the Faster ←→ Slower slider to change refresh rate.
Faster ≈ ~500 ms; Slower ≈ ~3000 ms.
Faster feels live; slower reduces CPU and makes copying easier.

Copy the locator

Select the locator in the text box and press Ctrl+C.
Paste into your test or automation scripts.
If the text changes while selecting, press ⬛ Stop first, then copy.

Stop peeking

Click ⬛ Stop (or Alt+S) to pause tracking.

Typical workflow

Open the target app.
Start Path Finder and hover precisely over the desired control (button, textbox, menu item).
Adjust the slider if updates are too fast or too slow.
Stop, copy the locator, and paste where needed.

Tips

Be precise: tiny mouse movements can change the target element.
Hover the actionable sub-area (icon/text) if a widget is composite.
Keep the Path Finder window out of the target area to avoid peeking it.
Run as Administrator when inspecting admin-level windows.

Troubleshooting

No updates: ensure you pressed Start; try a slower rate; run as Administrator for elevated apps.
Hard to copy: press Stop, then copy; or slide toward Slower.
Unexpected element: re-hover the exact clickable region; complex controls may have nested elements.

Keyboard & accessibility

Alt+S toggles Start/Stop.
Ctrl+C copies the locator from the text box.

Exit

Close the window (X or Alt+F4). Any active tracking stops automatically.

License

MIT License. See LICENSE for details.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.github/workflows		.github/workflows
src		src
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md

Uh oh!

Folders and files

Latest commit

History

Repository files navigation