🐾 OpenClaw Unity Plugin

TL;DR: Vibe-code your game development remotely from anywhere! 🌍

한줄요약: 이제 집밖에서도 원격으로 바이브코딩으로 게임 개발 가능합니다! 🎮

Connect Unity to OpenClaw AI assistant via HTTP. Works in Editor mode without hitting Play!

✨ Key Features

• 🎮 Works in Editor & Play Mode - No need to hit Play to use AI tools
• 🔌 Auto-Connect - Connects when Unity starts, maintains connection across mode changes
• 📋 Console Integration - Capture and query Unity logs for debugging
• 🎬 Scene Management - List, load, and inspect scenes
• 🔧 Component Editing - Add, remove, and modify component properties
• 📸 Debug Tools - Screenshots, hierarchy view, and more
• 🎯 Input Simulation - Keyboard, mouse, and UI interaction for game testing
• 🔄 Editor Control - Trigger recompilation and asset refresh remotely
• 🔒 Security Controls - Configure what operations are allowed

Requirements

Component	Version
Unity	2021.3+
OpenClaw	2026.2.3+

Installation

Option 1: Git URL (Recommended)

1. Open Unity Package Manager (Window > Package Manager)
2. Click + → Add package from git URL...
3. Enter:

   https://github.com/TomLeeLive/openclaw-unity-plugin.git
   

Option 2: Local Package

1. Clone this repository
2. In Unity: Window > Package Manager → + → Add package from disk...
3. Select the package.json file

Quick Start

1. Install OpenClaw Gateway Extension (Required)

Copy the gateway extension files to OpenClaw:

# Copy extension files
cp -r OpenClawPlugin~/* ~/.openclaw/extensions/unity/

# Restart gateway to load the extension
openclaw gateway restart

# Verify
openclaw unity status

Note: OpenClawPlugin~ contains the gateway extension that enables unity_execute and unity_sessions tools. This is required for OpenClaw to communicate with Unity.

2. Install Unity Package

See Installation above for Git URL or local package setup.

3. Configure in Unity

1. Open Window > OpenClaw Plugin
2. Set Gateway URL: http://localhost:18789 (default)
3. Connection is automatic when Unity starts
4. Status shows green when connected

4. Chat with OpenClaw

Ask OpenClaw to inspect your scene, create objects, or debug issues - all without entering Play mode!

5. Install OpenClaw Skill (Optional)

The companion skill provides workflow patterns and tool references for the AI:

# Clone skill to OpenClaw workspace
git clone https://github.com/TomLeeLive/openclaw-unity-skill.git ~/.openclaw/workspace/skills/unity-plugin

The skill provides:

• Quick reference for all 50 tools
• Common workflow patterns (scene inspection, UI testing, etc.)
• Detailed parameter documentation
• Troubleshooting guides

Note: The skill is separate from the gateway extension. The extension enables the tools; the skill teaches the AI how to use them effectively.

📚 Documentation

• Development Guide - Architecture, extending tools, and contribution guidelines
• Testing Guide - Complete testing guide with examples

Available Tools (50 total)

Console (3 tools)

Tool	Description
console.getLogs	Get Unity console logs (with type filter)
console.getErrors	Get error/exception logs (with optional warnings)
console.clear	Clear captured logs

Scene (5 tools)

Tool	Description
scene.list	List all scenes in build settings
scene.getActive	Get active scene info
scene.getData	Get scene hierarchy data
scene.load	Load a scene by name (Play mode)
scene.open	Open a scene in Editor mode

GameObject (7 tools)

Tool	Description
gameobject.find	Find by name, tag, or component type
gameobject.create	Create GameObject or primitive
gameobject.destroy	Destroy a GameObject
gameobject.delete	Delete a GameObject (alias for destroy)
gameobject.getData	Get detailed object data
gameobject.setActive	Enable/disable object
gameobject.setParent	Change parent

Transform (6 tools)

Tool	Description
transform.getPosition	Get world position (x, y, z)
transform.getRotation	Get rotation in Euler angles
transform.getScale	Get local scale
transform.setPosition	Set world position
transform.setRotation	Set rotation (Euler)
transform.setScale	Set local scale

Component

Tool	Description
component.add	Add component to object
component.remove	Remove component
component.get	Get component data
component.set	Set field/property value
component.list	List available types

Script

Tool	Description
script.execute	Execute simple commands
script.read	Read script file contents
script.list	List script files in project

Application

Tool	Description
app.getState	Get play mode, FPS, etc.
app.play	Enter play mode (Editor)
app.pause	Toggle pause (Editor)
app.stop	Exit play mode (Editor)

Debug

Tool	Description
debug.log	Write to console
debug.screenshot	Capture screenshot (with UI)
debug.hierarchy	Text hierarchy view

Editor (NEW in v1.2.0)

Tool	Description
editor.refresh	Refresh AssetDatabase (triggers recompile)
editor.recompile	Request script recompilation
editor.focusWindow	Focus Editor window (game/scene/console/hierarchy/project/inspector)
editor.listWindows	List all open Editor windows

Input Simulation (NEW in v1.2.0)

Tool	Description
input.keyPress	Press and release a key
input.keyDown	Press and hold a key
input.keyUp	Release a key
input.type	Type text into input field
input.mouseMove	Move mouse cursor
input.mouseClick	Click at position
input.mouseDrag	Drag from A to B
input.mouseScroll	Scroll wheel
input.getMousePosition	Get current cursor position
input.clickUI	Click UI element by name

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Unity Editor                             │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           OpenClawEditorBridge                          │ │
│  │           [InitializeOnLoad]                            │ │
│  │                                                          │ │
│  │  • EditorApplication.delayCall → safe init              │ │
│  │  • EditorApplication.update → connection polling        │ │
│  │  • SessionState → survives Play mode transitions        │ │
│  └──────────────────────┬─────────────────────────────────┘ │
│                         │                                    │
│                         ▼                                    │
│  ┌────────────────────────────────────────────────────────┐ │
│  │         OpenClawConnectionManager                       │ │
│  │         (Singleton - shared across modes)               │ │
│  │                                                          │ │
│  │  • HTTP polling for commands                            │ │
│  │  • Main thread execution queue                          │ │
│  │  • Automatic reconnection                               │ │
│  │  • JSON parsing with nested object support              │ │
│  └──────────────────────┬─────────────────────────────────┘ │
│                         │                                    │
│                         ▼                                    │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           OpenClawTools (50 tools)                      │ │
│  │                                                          │ │
│  │  • Scene/GameObject/Component manipulation              │ │
│  │  • Debug tools (screenshot, hierarchy)                  │ │
│  │  • Input simulation (keyboard, mouse, UI)               │ │
│  │  • Editor control (recompile, refresh)                  │ │
│  └────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
                              │
                              │ HTTP
                              ▼
┌──────────────────────────────────────────────────────────────┐
│                   OpenClaw Gateway                            │
│                   http://localhost:18789                      │
│                                                               │
│  Endpoints:                                                   │
│  • POST /unity/register  - Register Unity session             │
│  • POST /unity/heartbeat - Keep session alive                 │
│  • GET  /unity/poll      - Poll for commands                  │
│  • POST /unity/result    - Send tool execution results        │
└──────────────────────────────────────────────────────────────┘

Configuration

Create via Assets > Create > OpenClaw > Config and place in Resources folder.

Setting	Description	Default
gatewayUrl	OpenClaw gateway URL	http://localhost:18789
apiToken	Optional API token	(empty)
autoConnect	Connect on start	true
showStatusOverlay	Show status in Game view	true
captureConsoleLogs	Capture logs for AI	true
allowCodeExecution	Allow code execution	true
allowFileAccess	Allow file operations	true
allowSceneModification	Allow scene changes	true

Example Usage

Scene Inspection

You: What GameObjects are in my scene?

OpenClaw: [Executes debug.hierarchy]

Your scene has:
▶ Main Camera [Camera, AudioListener]
▶ Directional Light [Light]
▶ Player [PlayerController, Rigidbody]
  ▶ Model [MeshRenderer]
▶ UI Canvas [Canvas, GraphicRaycaster]
  ▶ PlayButton [Button]

Game Testing with Input Simulation

You: Test the login flow - enter username "TestPlayer" and click Play

OpenClaw: 
[Executes input.clickUI {name: "UsernameInput"}]
[Executes input.type {text: "TestPlayer"}]
[Executes input.clickUI {name: "PlayButton"}]
[Executes debug.screenshot]

Done! Clicked username input, typed "TestPlayer", and clicked Play button.
Screenshot attached showing the result.

Remote Recompilation

You: I updated the PlayerController script, recompile Unity

OpenClaw: [Executes editor.recompile]

Script recompilation requested. Unity will reload shortly.

Troubleshooting

Bridge won't connect

1. Check Gateway status: openclaw gateway status
2. Verify URL: default is http://localhost:18789
3. Check Window > OpenClaw Plugin for errors

Connection lost during Play mode transition

• Plugin uses SessionState to survive domain reloads
• Auto-reconnects after Play mode transition
• If stuck, use editor.refresh or click "Force Reconnect"

Screenshot shows wrong content

• In Play mode: Uses ScreenCapture (includes UI)
• In Editor mode: Uses Camera.main.Render() (no overlay UI)
• Use Play mode for accurate game screenshots

Script changes not applied after Play mode restart

Unity's "Enter Play Mode Settings" can skip domain reload for faster iteration, but this prevents script recompilation.

Symptoms:

• Code changes don't take effect when re-entering Play mode
• Old behavior persists despite saving scripts
• editor.refresh or editor.recompile has no effect during Play mode

Solution:

1. Go to Edit → Project Settings → Editor
2. Find "Enter Play Mode Settings"
3. Check ✅ "Reload Domain"

What this does:

Setting	Reload Domain ON	Reload Domain OFF
Script changes	✅ Applied on Play	❌ Ignored until manual refresh
Play mode entry	~2-5 seconds	~0.5 seconds
Static variables	Reset	Preserved
Best for	Development with active coding	Testing/playing without code changes

Tip: Keep "Reload Domain" ON during development. Only disable it when you need fast iteration without code changes.

⚠️ Important Notes

• Development Only: Disable allowCodeExecution in production builds
• TextMeshPro: Plugin works with or without TMPro (uses reflection)
• Unity 6: Deferred initialization prevents UPM EPIPE crashes

Changelog

See CHANGELOG.md for version history.

License

MIT License - See LICENSE

---

Made with 🐾 by the OpenClaw community