Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,17 @@ temp_extracted_asar
dist-electron/
dist-electron-build/
out/

.venv
.venv-local
.venv-build
.vscode
.idea
.pytest_cache
.coverage
.coverage.*
.coverage.*.*
.coverage.*.*.*
.coverage.*.*.*.*
.coverage.*.*.*.*.*
.coverage.*.*.*.*.*.*
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,17 @@ BUILD_STATUS.md
dist-electron/
dist-electron-build/
out/

.venv
.venv-local
.venv-build
.vscode
.idea
.pytest_cache
.coverage
.coverage.*
.coverage.*.*
.coverage.*.*.*
.coverage.*.*.*.*
.coverage.*.*.*.*.*
.coverage.*.*.*.*.*.*
16 changes: 13 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ and assets without wiring multiple tools together. It’s not a full IDE or asse
- **Unity UI Elements**: Generate Unity UI prefabs (health bars, buttons, dialogue boxes, HUD layouts) for both uGUI and UI Toolkit
- **Incremental Asset Generation**: Save assets directly into an active Unity project
- **Pixel-Art Sprites**: Generate and process 2D sprite sheets with automatic cropping
- **Unity MCP Integration**: Real-time interaction with the Unity Editor using [Unity-MCP-SK-Plugin](https://github.com/Ozymandros/Unity-MCP-SK-Plugin) and [Unity-MCP-Server](https://github.com/Ozymandros/Unity-MCP-Server)
- **Unity MCP integration**: Python bridge [`unity-mcp-plugin`](https://github.com/Ozymandros/Unity-MCP-SK-Plugin) (Semantic Kernel; see `backend/requirements.txt` for Git vs local editable install) plus MCP server [**Unity-MCP-Server**](https://github.com/Ozymandros/Unity-MCP-Server) — both are required; see [docs/UNITY_MCP_SERVER.md](docs/UNITY_MCP_SERVER.md) for server install/Git URLs/env vars, and [docs/UNITY_MCP_INTEGRATION.md](docs/UNITY_MCP_INTEGRATION.md) for the full SK setup.
- Save and reuse provider settings and preferences locally
- Configure global and per-request system key prompts for tailored generation
- Keep output structured so Unity can open it right away
Expand All @@ -80,6 +80,7 @@ Scaffolded and functional. See docs for development and packaging details. The p

- [Architecture overview](docs/ARCHITECTURE.md)
- [Development guide](docs/DEVELOPMENT.md)
- [Unity MCP Server (install, Git URL, env vars)](docs/UNITY_MCP_SERVER.md)
- [Packaging and distribution](docs/PACKAGING.md)
- [System Prompts guide](docs/SYSTEM_PROMPTS.md)

Expand Down Expand Up @@ -358,9 +359,18 @@ Electron app.
- If a provider request errors, check `logs/` for the failed request log.
- If a Docker build is slow, ensure `node_modules/` and venvs are ignored.

## Unity MCP Integration
## Unity MCP integration

See [docs/UNITY_MCP_INTEGRATION.md](docs/UNITY_MCP_INTEGRATION.md) for details on the Semantic Kernel MCP integration, configuration, and usage.
Dependencies and setup (both repos are required):

| Role | GitHub |
|------|--------|
| MCP server (Unity tools, stdio) | [Unity-MCP-Server](https://github.com/Ozymandros/Unity-MCP-Server) |
| Python Semantic Kernel plugin (`unity_mcp`) | [Unity-MCP-SK-Plugin](https://github.com/Ozymandros/Unity-MCP-SK-Plugin) |

- **Unity MCP Server only (not the Python plugin):** [docs/UNITY_MCP_SERVER.md](docs/UNITY_MCP_SERVER.md)
- **Semantic Kernel + paths:** [docs/UNITY_MCP_INTEGRATION.md](docs/UNITY_MCP_INTEGRATION.md)
- **Expanded vs router SK modes:** [docs/UNITY_MCP_SK_INTEGRATION.md](docs/UNITY_MCP_SK_INTEGRATION.md)

## Download

Expand Down
2 changes: 1 addition & 1 deletion backend/app/agents/SKILLS_USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Provides real-time integration with a running Unity Editor via the Unity MCP Ser
- Plugin Source: [Unity-MCP-SK-Plugin](https://github.com/Ozymandros/Unity-MCP-SK-Plugin)
- Server Source: [Unity-MCP-Server](https://github.com/Ozymandros/Unity-MCP-Server)

Requires the `unity-mcp-plugin` package installed in the Python environment. For the full tool list and parameter details, see the server’s tool reference (e.g. Unity-MCP-Server `Skills/SKILL.md` or README).
Requires the `unity-mcp-plugin` package installed in the Python environment (see `backend/requirements.txt` and [docs/UNITY_MCP_INTEGRATION.md](../../../docs/UNITY_MCP_INTEGRATION.md) for the SK bridge). **MCP server install, clone URL, and `UNITY_MCP_*` locations:** [docs/UNITY_MCP_SERVER.md](../../../docs/UNITY_MCP_SERVER.md). For the full tool list and parameter details, see the server’s tool reference (e.g. Unity-MCP-Server `Skills/SKILL.md` or README).

**Path handling:** The server requires **projectPath** (project root) and **fileName** (or **folderName** for folder/list tools) on all file, scene, and asset tools. Pass `projectPath` as the project root directory and `fileName` as the path under the project (e.g. `Assets/Scenes/MyScene.unity`). Do not duplicate path segments (e.g. do not put the project root inside `fileName`).

Expand Down
45 changes: 33 additions & 12 deletions backend/app/agents/unity_mcp_plugin.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
import asyncio
import logging
import os
from contextlib import asynccontextmanager

from semantic_kernel import Kernel
from semantic_kernel.connectors.mcp import MCPStdioPlugin
from unity_mcp import UnityMcpOptions, UnityMCPPlugin

LOGGER = logging.getLogger("unity_mcp_plugin")
_ENV_MCP_COMMAND = "UNITY_MCP_COMMAND"
_ENV_MCP_ARGS = "UNITY_MCP_ARGS"
_DEFAULT_MCP_EXECUTABLE = "unity-mcp"


async def _check_unity_mcp_live() -> bool:
Expand All @@ -20,6 +24,9 @@ async def _check_unity_mcp_live() -> bool:
plugin = kernel.get_plugin("UnityMCP")
if plugin and "ping" in plugin:
await kernel.invoke(plugin["ping"])
elif hasattr(mcp_plugin, "is_healthy"):
if not mcp_plugin.is_healthy():
return False
return True
except Exception as e:
LOGGER.debug("Unity MCP live check failed: %s", e)
Expand All @@ -41,22 +48,36 @@ def unity_mcp_plugin_available_for_writing() -> bool:
@asynccontextmanager
async def create_unity_mcp_plugin():
"""
Async context manager for the Unity MCP plugin (global unity-mcp tool).
Async context manager for the Unity MCP plugin.

Uses the local `unity-mcp-plugin` package implementation and initializes
the plugin against the configured Unity MCP executable.

Use with: async with create_unity_mcp_plugin() as mcp_plugin: ...
"""
LOGGER.info("Creating UnityMCP plugin...")
command = os.environ.get(_ENV_MCP_COMMAND, _DEFAULT_MCP_EXECUTABLE).strip() or _DEFAULT_MCP_EXECUTABLE
mcp_args = os.environ.get(_ENV_MCP_ARGS, "").strip()
if mcp_args:
LOGGER.warning(
"%s is set but ignored by unity-mcp-plugin local package implementation.",
_ENV_MCP_ARGS,
)

try:
plugin = MCPStdioPlugin(
name="UnityMCP",
description="Unity Editor automation tools",
command="unity-mcp",
args=[],
load_tools=True,
request_timeout=30,
options = UnityMcpOptions(
executable_path=command,
connection_timeout_seconds=30,
request_timeout_seconds=30,
enable_message_logging=False,
)
async with plugin as mcp_plugin:
LOGGER.info("UnityMCP plugin created successfully.")
yield mcp_plugin
plugin = UnityMCPPlugin.create(options)
await plugin.initialize()
LOGGER.info("UnityMCP plugin created successfully.")
try:
yield plugin
finally:
await plugin.cleanup()
except Exception as e:
LOGGER.error("Failed to create UnityMCP plugin: %s", e)
raise
2 changes: 1 addition & 1 deletion backend/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "unity-generator-backend",
"version": "1.7.0",
"version": "1.8.0",
"description": "Backend for Unity Generator",
"private": true,
"scripts": {
Expand Down
2 changes: 1 addition & 1 deletion backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "unity-generator-backend"
version = "0.10.0"
version = "1.8.0"
description = "Unity project generation and finalization backend with AI integration"
requires-python = ">=3.11"
license = {text = "MIT"}
Expand Down
10 changes: 8 additions & 2 deletions backend/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
fastapi
uvicorn
mcp[cli]
semantic-kernel[mcp]>=1.15.0
pydantic
requests
Pillow
jinja2
google-genai
anthropic
anthropic
semantic-kernel[mcp]>=1.15.0
# unity-mcp-plugin (Semantic Kernel bridge for Unity MCP) — use ONE of:
# (A) No sibling folder: install from Git (HTTPS or SSH). Example if you have access:
# unity-mcp-plugin @ git+ssh://git@github.com/Ozymandros/Unity-MCP-SK-Plugin.git
# (Public HTTPS fails with 404 if the repo is private — use SSH, a fork, or a wheel.)
# (B) Default dev layout: clone Unity-MCP-SK-Plugin next to this repo as "unity-mcp-server-plugin":
unity-mcp-plugin @ git+https://github.com/Ozymandros/unity-mcp-server-plugin.git
44 changes: 36 additions & 8 deletions backend/tests/test_unity_mcp_pluginwrapper.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,42 @@
from __future__ import annotations

from unittest.mock import AsyncMock, MagicMock, patch

import pytest

from app.agents.unity_mcp_plugin import create_unity_mcp_plugin, unity_mcp_plugin_available_for_writing


# When re-enabling: update tests to match the new Unity-MCP-Server contract
# (projectPath + fileName/folderName on tools, ping with no args).
@pytest.mark.skip(reason="Requires running MCP server or better mocking of async connection")
@pytest.mark.asyncio
async def test_unity_mcp_pluginwrapper_connect_and_discover(monkeypatch):
pass
async def test_unity_mcp_pluginwrapper_create_initialize_and_cleanup() -> None:
"""Ensure wrapper creates plugin via local package and performs lifecycle calls."""
mock_plugin = MagicMock()
mock_plugin.initialize = AsyncMock()
mock_plugin.cleanup = AsyncMock()

with patch("app.agents.unity_mcp_plugin.UnityMCPPlugin.create", return_value=mock_plugin) as mock_create:
async with create_unity_mcp_plugin() as plugin:
assert plugin is mock_plugin
mock_create.assert_called_once()
mock_plugin.initialize.assert_awaited_once()
mock_plugin.cleanup.assert_awaited_once()


@pytest.mark.skip(reason="Requires running MCP server or better mocking of async connection")
@pytest.mark.asyncio
async def test_unity_mcp_pluginwrapper_error(monkeypatch):
pass
async def test_unity_mcp_pluginwrapper_error_propagates() -> None:
"""Ensure initialization errors are not swallowed by wrapper context manager."""
mock_plugin = MagicMock()
mock_plugin.initialize = AsyncMock(side_effect=RuntimeError("boom"))
mock_plugin.cleanup = AsyncMock()

with patch("app.agents.unity_mcp_plugin.UnityMCPPlugin.create", return_value=mock_plugin):
with pytest.raises(RuntimeError, match="boom"):
async with create_unity_mcp_plugin():
pass
mock_plugin.cleanup.assert_not_awaited()


def test_unity_mcp_plugin_available_for_writing_false_on_failure() -> None:
"""Availability helper must return False when plugin creation fails."""
with patch("app.agents.unity_mcp_plugin.create_unity_mcp_plugin", side_effect=RuntimeError("down")):
assert unity_mcp_plugin_available_for_writing() is False
2 changes: 1 addition & 1 deletion docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ Finalize supports two backends. In **Unity Engine Settings** (or the finalize re
- **`batch`**: Always use injected Editor scripts and Unity batch mode. Use this for CI or headless environments where no Editor window is running.
- **`mcp`**: Always use the MCP-Unity (Semantic Kernel) plugin. Fails with a clear error if the plugin is not configured.

To use MCP for local development, set **`UNITY_USE_MCP=1`** and either **`UNITY_MCP_SERVER_URL`** (HTTP) or ensure **`unity-mcp`** (or the command in **`UNITY_MCP_COMMAND`**) is on PATH for stdio. The backend connects via the MCP Python SDK and calls the Unity-MCP-Server contract tools. For CI pipelines, set `unity_automation_mode` to `"batch"` so finalize does not depend on MCP. See [Unity Engine Integration](UNITY_INTEGRATION.md) for the tool contract and configuration.
To use MCP for local development, set **`UNITY_USE_MCP=1`** and either **`UNITY_MCP_SERVER_URL`** (HTTP) or ensure **`unity-mcp`** (or the command in **`UNITY_MCP_COMMAND`**) is on PATH for stdio. The backend connects via the MCP Python SDK and calls the Unity-MCP-Server contract tools. For CI pipelines, set `unity_automation_mode` to `"batch"` so finalize does not depend on MCP. **Server install, Git clone URL, and where env vars are read:** [UNITY_MCP_SERVER.md](UNITY_MCP_SERVER.md). See [Unity Engine Integration](UNITY_INTEGRATION.md) for the tool contract and configuration.

### Debugging Unity batch execution

Expand Down
33 changes: 26 additions & 7 deletions docs/UNITY_MCP_INTEGRATION.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
# Unity MCP Integration with Semantic Kernel

This document describes the integration of Unity's Model Context Protocol (MCP) with Semantic Kernel using the official MCPSsePlugin, replacing legacy TCP/JSON-RPC code with a robust, maintainable, and auto-discoverable approach.
> **Unity MCP Server (install, Git URL, env vars — not the Python plugin):** see **[UNITY_MCP_SERVER.md](./UNITY_MCP_SERVER.md)**.

This document describes the integration of Unity's Model Context Protocol (MCP) with Semantic Kernel using the Python **`unity-mcp-plugin`** package from **[Unity-MCP-SK-Plugin](https://github.com/Ozymandros/Unity-MCP-SK-Plugin)** (Git URL or local editable install—see `backend/requirements.txt`). The backend talks to the **Unity MCP Server** over stdio. Both pieces are required and are separate GitHub repositories.

## Required dependencies (two repositories)

| Component | Purpose | GitHub repository | How this project uses it |
|-----------|---------|-------------------|----------------------------|
| **Unity-MCP-Server** | Official MCP server that exposes Unity Editor automation tools (JSON-RPC over stdio when used as a global .NET tool). | **[github.com/Ozymandros/Unity-MCP-Server](https://github.com/Ozymandros/Unity-MCP-Server)** | Install and run the `unity-mcp` global tool (or your built server executable). Point `UNITY_MCP_COMMAND` at that executable if it is not on `PATH`. |
| **unity-mcp-plugin** (Python distribution) | Semantic Kernel bridge: discovers MCP tools and registers them as SK functions. | **[github.com/Ozymandros/Unity-MCP-SK-Plugin](https://github.com/Ozymandros/Unity-MCP-SK-Plugin)** | `backend/requirements.txt`: default **`-e ../unity-mcp-server-plugin`**, or switch to **`unity-mcp-plugin @ git+...`** (SSH/fork if private). Imported as `from unity_mcp import UnityMCPPlugin, UnityMcpOptions`. |

**Install order (typical dev setup):**

1. Clone **[Unity-MCP-Server](https://github.com/Ozymandros/Unity-MCP-Server)** and install the server per its README (often a global .NET tool named `unity-mcp` on your `PATH`; follow that repo’s install script or `dotnet tool` instructions).
2. Install **`unity-mcp-plugin`**: either follow the **default** in `backend/requirements.txt` (sibling clone + `-e`), **or** comment that out and use **`unity-mcp-plugin @ git+ssh://...`** (or your fork) so the app does not depend on a fixed folder next to this repo. If `git+https` fails with **404**, the GitHub repo is likely **private**—use SSH, a public fork, or a wheel.

**Optional dev workflow:** clone **Unity-MCP-SK-Plugin** anywhere and use `pip install -e /path/to/clone` while developing the bridge.

## Path terminology (don’t confuse these)

Expand All @@ -10,13 +26,13 @@ This document describes the integration of Unity's Model Context Protocol (MCP)
## Overview
- **No Unity/C# code required**: This integration is entirely Python-side.
- **Auto-discovery**: All Unity tools exposed by the MCP server are automatically available to Semantic Kernel.
- **No manual kernel_function decorators**: The MCPSsePlugin handles tool registration.
- **No manual kernel_function decorators**: The local Unity MCP plugin package handles tool registration.
- **Robust error handling**: Connection and protocol errors are surfaced clearly.

## Configuration
- **Install requirements**: Ensure `semantic-kernel[mcp]` is in `backend/requirements.txt`.
- **Install requirements**: Ensure `backend/requirements.txt` installs **`unity-mcp-plugin`** (see comments in that file: Git URL vs `-e` sibling path).
- **Set MCP server command** (this is the **Unity MCP Server executable**, not a Unity project path):
- `UNITY_MCP_COMMAND`: Path to the MCP server executable (default: `unity-mcp-server`)
- `UNITY_MCP_COMMAND`: Path to the MCP server executable (default: `unity-mcp`)
- `UNITY_MCP_ARGS`: Arguments for the executable (default: empty)

Example (path to the **server executable**, not to a Unity project):
Expand All @@ -25,7 +41,7 @@ export UNITY_MCP_COMMAND="C:\Path\To\UnityMcp.Server.exe"
```

## Usage
- The integration is implemented in `backend/app/agents/unity_mcp_plugin.py` as `UnityMCPPluginWrapper`.
- The integration is implemented in `backend/app/agents/unity_mcp_plugin.py` using `unity_mcp.UnityMCPPlugin` from the **`unity-mcp-plugin`** package.
- The agent in `backend/app/agents/unity_agent.py` uses this wrapper to connect and interact with Unity tools.
- All Unity automation tools (e.g., create_scene, create_script) are auto-discovered and callable from the agent.

Expand Down Expand Up @@ -60,8 +76,11 @@ export UNITY_MCP_COMMAND="C:\Path\To\UnityMcp.Server.exe"
- All Unity tool changes/additions are now automatically available to the agent—no Python code changes required.

## References
- [Semantic Kernel MCP Plugin Documentation](https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/adding-mcp-plugins)
- [Model Context Protocol (MCP) Spec](https://modelcontextprotocol.io/)
- **[Unity MCP Server — install & config in this repo](./UNITY_MCP_SERVER.md)** (Git URLs, `UNITY_MCP_COMMAND`, file locations)
- [Unity-MCP-Server (GitHub)](https://github.com/Ozymandros/Unity-MCP-Server)
- [Unity-MCP-SK-Plugin / unity-mcp-plugin (GitHub)](https://github.com/Ozymandros/Unity-MCP-SK-Plugin)
- [Semantic Kernel — MCP plugins](https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/adding-mcp-plugins)
- [Model Context Protocol (MCP) spec](https://modelcontextprotocol.io/)

---

Expand Down
Loading
Loading