Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
name: nemo-studio-copilot
name: nemo-studio-assistant
created_timestamp: 2026-07-28T20:09:49Z
author: Danielle Ali and Codex
---

# Agent Spec: nemo-studio-copilot
# Agent Spec: nemo-studio-assistant

> This file is the durable contract for the local NeMo Platform assistant.
> Keep it aligned with the implementation under `agents/nemo-studio-copilot/`.
> Keep it aligned with the implementation under `agents/nemo-studio-assistant/`.
> The adjacent `agent.yaml` and `skills/` directory are the clean, size-bounded
> fileset used for Platform registration; the source project retains its
> packaging config and MCP executable.
Expand Down Expand Up @@ -82,15 +82,15 @@ The mission is grounded in the current deployment proof of concept and its imple

## Evaluation Setup

Unit coverage lives in `agents/nemo-studio-copilot/tests/test_nemo_studio_copilot.py`. Run it with:
Unit coverage lives in `agents/nemo-studio-assistant/tests/test_nemo_studio_assistant.py`. Run it with:

```bash
uv run --frozen pytest agents/nemo-studio-copilot/tests/test_nemo_studio_copilot.py -v
uv run --frozen pytest agents/nemo-studio-assistant/tests/test_nemo_studio_assistant.py -v
```
Comment thread
dmariali marked this conversation as resolved.

The prior NAT evaluation YAML was removed because it depended on NAT's custom workflow and `/generate/full` endpoint. Each retained case in `nemo-studio-copilot-eval-data.json` now has deterministic unit coverage for its required `nemo_api` tool path, collected by the root CI test suite.
The prior NAT evaluation YAML was removed because it depended on NAT's custom workflow and `/generate/full` endpoint. Each retained case in `nemo-studio-assistant-eval-data.json` now has deterministic unit coverage for its required `nemo_api` tool path, collected by the root CI test suite.

Manual Studio validation is documented in `agents/nemo-studio-copilot/tests/smoke_test.md`. Current coverage gaps include fileset listing, missing-workspace clarification, fast-path failure containment, destructive-action ambiguity, iteration limits, retry limits, cancellation, and end-to-end latency thresholds.
Manual Studio validation is documented in `agents/nemo-studio-assistant/tests/smoke_test.md`. Current coverage gaps include fileset listing, missing-workspace clarification, fast-path failure containment, destructive-action ambiguity, iteration limits, retry limits, cancellation, and end-to-end latency thresholds.

## Change Scope

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
config_format: nemo-agents-spec-v1
name: nemo-studio-copilot
description: NeMo Studio copilot for discovering, configuring, operating, evaluating, and troubleshooting NeMo Platform resources through approval-aware SDK actions.
name: nemo-studio-assistant
description: NeMo Studio assistant for discovering, configuring, operating, evaluating, and troubleshooting NeMo Platform resources through approval-aware SDK actions.

instructions:
system:
Expand Down Expand Up @@ -56,7 +56,7 @@ mcp:
servers:
nemo_studio:
transport: stdio
url: env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-copilot-mcp
url: env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-assistant-mcp
exposure: harness_native

tools:
Expand All @@ -70,4 +70,4 @@ telemetry:
enabled: false
provider: relay
output_dir: ./artifacts/relay
project: nemo-studio-copilot
project: nemo-studio-assistant
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "Benchmark task execution contract: complete every numbered require
---
# Benchmark execution contract

This skill defines the execution requirements that every nemo-studio-copilot run under
This skill defines the execution requirements that every nemo-studio-assistant run under
`tests/agentic-use/` must satisfy so the canonical gate
(`tests/agentic-use/passrate_token_policy_gate.py`) can score the run on
verifier pass-rate and token totals. See
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,3 @@ dist/
build/
.venv/
node_modules/
tests/
src/nemo_studio_copilot/nemo-studio-copilot-eval-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ RUN --mount=type=cache,id=uv_cache,target=/root/.cache/uv,sharing=locked \
chmod -R a+rX /opt/uv /workspace/.venv


LABEL org.opencontainers.image.title="nemo-studio-copilot" \
LABEL org.opencontainers.image.title="nemo-studio-assistant" \
org.opencontainers.image.version="0.1.0" \
org.opencontainers.image.authors="Danielle Ali" \
org.opencontainers.image.created="2026-08-05T19:39:51Z" \
org.opencontainers.image.description="NeMo Studio Copilot — a Fabric Deep Agents assistant for NeMo Platform tasks." \
org.opencontainers.image.description="NeMo Studio Assistant — a Fabric Deep Agents assistant for NeMo Platform tasks." \
org.opencontainers.image.revision="ca85d0febce9ced44a8d7dfbb2e3391d829697f9" \
org.opencontainers.image.source="https://github.com/NVIDIA-NeMo/nemo-platform.git" \
com.nemo.agent.id="c9acc9e1c696" \
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
config_format: nemo-agents-spec-v1
name: nemo-studio-copilot
description: NeMo Studio copilot for discovering, configuring, operating, evaluating, and troubleshooting NeMo Platform resources through approval-aware SDK actions.
name: nemo-studio-assistant
description: NeMo Studio assistant for discovering, configuring, operating, evaluating, and troubleshooting NeMo Platform resources through approval-aware SDK actions.

instructions:
system:
Expand Down Expand Up @@ -42,21 +42,21 @@ models:

skills:
paths:
- src/nemo_studio_copilot/skills/auditor
- src/nemo_studio_copilot/skills/benchmark-execution
- src/nemo_studio_copilot/skills/entities
- src/nemo_studio_copilot/skills/evaluator
- src/nemo_studio_copilot/skills/files
- src/nemo_studio_copilot/skills/guardrails
- src/nemo_studio_copilot/skills/inference
- src/nemo_studio_copilot/skills/secrets
- src/nemo_studio_copilot/skills/workspace
- src/nemo_studio_assistant/skills/auditor
- src/nemo_studio_assistant/skills/benchmark-execution
- src/nemo_studio_assistant/skills/entities
- src/nemo_studio_assistant/skills/evaluator
- src/nemo_studio_assistant/skills/files
- src/nemo_studio_assistant/skills/guardrails
- src/nemo_studio_assistant/skills/inference
- src/nemo_studio_assistant/skills/secrets
- src/nemo_studio_assistant/skills/workspace

mcp:
servers:
nemo_studio:
transport: stdio
url: env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-copilot-mcp
url: env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-assistant-mcp
exposure: harness_native

tools:
Expand All @@ -70,4 +70,4 @@ telemetry:
enabled: false
provider: relay
output_dir: ./artifacts/relay
project: nemo-studio-copilot
project: nemo-studio-assistant
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "nemo-studio-copilot"
name = "nemo-studio-assistant"
version = "0.1.0"
description = "NeMo Studio Copilot — a Fabric Deep Agents assistant for NeMo Platform tasks."
description = "NeMo Studio Assistant — a Fabric Deep Agents assistant for NeMo Platform tasks."
requires-python = ">=3.12,<3.14"
dependencies = [
"nemo-platform",
Expand All @@ -17,10 +17,10 @@ dependencies = [
]

[project.scripts]
nemo-studio-copilot-mcp = "nemo_studio_copilot.mcp_server:main"
nemo-studio-assistant-mcp = "nemo_studio_assistant.mcp_server:main"

[tool.hatch.build.targets.wheel]
packages = ["src/nemo_studio_copilot"]
packages = ["src/nemo_studio_assistant"]

[tool.uv.sources]
nemo-agents-plugin = { path = "../../plugins/nemo-agents", editable = true }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "Benchmark task execution contract: complete every numbered require
---
# Benchmark execution contract

This skill defines the execution requirements that every nemo-studio-copilot run under
This skill defines the execution requirements that every nemo-studio-assistant run under
`tests/agentic-use/` must satisfy so the canonical gate
(`tests/agentic-use/passrate_token_policy_gate.py`) can score the run on
verifier pass-rate and token totals. See
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Fabric-native MCP boundary for NeMo Studio Copilot tools."""
"""Fabric-native MCP boundary for NeMo Studio Assistant tools."""

from fastmcp import FastMCP
from nemo_studio_copilot.register import (
from nemo_studio_assistant.register import (
ask_user_question,
check_status,
job_progress,
Expand All @@ -30,7 +30,7 @@


def create_server() -> FastMCP:
server = FastMCP("NeMo Studio Copilot")
server = FastMCP("NeMo Studio Assistant")
for tool in TOOLS:
server.tool(tool)
return server
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""NeMo Studio Copilot tool implementations exposed through MCP."""
"""NeMo Studio Assistant tool implementations exposed through MCP."""

import json
import os
Expand All @@ -14,7 +14,7 @@
from pydantic import BaseModel

DEFAULT_WORKSPACE = "default"
STUDIO_CALLBACK_PATH = "/studio/api/copilot/mcp/{session_id}"
STUDIO_CALLBACK_PATH = "/studio/api/assistant/mcp/{session_id}"
STUDIO_CALLBACK_TIMEOUT_SECONDS = 3600.0
_READ_ONLY_SDK_ACTIONS = frozenset({"get", "get_logs", "get_status", "list", "read", "retrieve", "search"})

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "Benchmark task execution contract: complete every numbered require
---
# Benchmark execution contract

This skill defines the execution requirements that every nemo-studio-copilot run under
This skill defines the execution requirements that every nemo-studio-assistant run under
`tests/agentic-use/` must satisfy so the canonical gate
(`tests/agentic-use/passrate_token_policy_gate.py`) can score the run on
verifier pass-rate and token totals. See
Expand Down
82 changes: 82 additions & 0 deletions agents/nemo-studio-assistant/tests/smoke_test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# NeMo Studio Assistant Fabric smoke test

The assistant is a `nemo-agents-spec-v1` Fabric agent using the preinstalled
Deep Agents adapter. Its platform and Studio tools are exposed by the packaged
`nemo-studio-assistant-mcp` stdio server.

## Static and unit validation

From the repository root:
Comment thread
dmariali marked this conversation as resolved.

```bash
PYTHONPATH=agents/nemo-studio-assistant/src \
uv run --frozen pytest agents/nemo-studio-assistant/tests/test_nemo_studio_assistant.py -v
uv run ruff check agents/nemo-studio-assistant/src agents/nemo-studio-assistant/tests
uv run ruff format --check agents/nemo-studio-assistant/src agents/nemo-studio-assistant/tests
uv run --frozen ty check
```

The tests validate Fabric translation, the Deep Agents adapter, all packaged
skills, the MCP tool surface, SDK serialization, session validation, and
approval gating for mutations.

## Register, package, and deploy

These commands change platform state and should be run independently so each
result can be verified:

```bash
export NMP_BASE_URL=http://127.0.0.1:8080
nemo agents create --name nemo-studio-assistant \
--agent-config agents/nemo-studio-assistant-spec/agent.yaml
nemo agents package \
--agent agents/nemo-studio-assistant/agent.yaml \
--pyproject agents/nemo-studio-assistant/pyproject.toml \
--tag nemo-studio-assistant:fabric-local
nemo agents deploy --agent nemo-studio-assistant \
--name nemo-studio-assistant-fabric-deployment \
--mode docker --image nemo-studio-assistant:fabric-local
nemo agents deployments wait nemo-studio-assistant-fabric-deployment
```

The generated image must use the Fabric server entrypoint, not `nat serve`.

## Invocation

```bash
nemo agents invoke --agent-deployment nemo-studio-assistant-fabric-deployment \
--input "List all workspaces on the platform"
nemo agents invoke --agent-deployment nemo-studio-assistant-fabric-deployment \
--input "List the available models and inference providers using the platform API."
```

For mutation approval, invoke through Studio and ask the assistant to create a
workspace named `nemo-studio-assistant-smoke-test`. Verify that Studio renders an
approval prompt before the SDK call, and that declining leaves platform state
unchanged.

## Studio streaming

Open `http://127.0.0.1:8080/studio`, select the assistant deployment, and send
`List all workspaces`. Verify incremental, nonduplicated text and a terminal
success or error. Then exercise a selector and an approved mutation to verify
the callback MCP tools.

## Cleanup

After verification, remove the temporary workspace, then undeploy and delete
the agent so another smoke-test run starts cleanly:

```bash
nemo workspaces delete nemo-studio-assistant-smoke-test
nemo agents undeploy nemo-studio-assistant-fabric-deployment --yes
nemo agents delete nemo-studio-assistant --yes
```

## Automated evaluation coverage

The former NAT `/generate/full` evaluation YAML remains removed. Each case in
`src/nemo_studio_assistant/nemo-studio-assistant-eval-data.json` now has
deterministic unit coverage for its required `nemo_api` tool path in
`test_nemo_studio_assistant.py`. Root pytest discovery includes these unit tests,
so `make test-unit-ci` runs them in CI without a live model endpoint.
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@
import pytest
from nemo_agents_plugin.agent_config import load_agent_config
from nemo_agents_plugin.fabric.translator import translate_agent_config
from nemo_studio_copilot import register
from nemo_studio_copilot.mcp_server import create_server
from nemo_studio_assistant import register
from nemo_studio_assistant.mcp_server import create_server

AGENT_ROOT = Path(__file__).parents[1]
SPEC_ROOT = AGENT_ROOT.parent / "nemo-studio-copilot-spec"
EVAL_DATA_PATH = AGENT_ROOT / "src/nemo_studio_copilot/nemo-studio-copilot-eval-data.json"
SPEC_ROOT = AGENT_ROOT.parent / "nemo-studio-assistant-spec"
EVAL_DATA_PATH = AGENT_ROOT / "src/nemo_studio_assistant/nemo-studio-assistant-eval-data.json"

pytestmark = pytest.mark.unit

Expand Down Expand Up @@ -65,7 +65,7 @@ def test_agent_config_translates_to_fabric_deepagents() -> None:
assert translated.models["default"].provider == "nvidia"
assert translated.mcp is not None
assert translated.mcp.servers["nemo_studio"].url == (
"env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-copilot-mcp"
"env NMP_BASE_URL=$NMP_BASE_URL NMP_WORKSPACE=$NMP_WORKSPACE nemo-studio-assistant-mcp"
)


Expand Down Expand Up @@ -398,7 +398,7 @@ def test_studio_callback_url_uses_workspace(monkeypatch: pytest.MonkeyPatch) ->

url = register._studio_callback_url(session_id, workspace="demo")

assert url == f"http://platform:8080/studio/api/copilot/mcp/{session_id}?workspace=demo"
assert url == f"http://platform:8080/studio/api/assistant/mcp/{session_id}?workspace=demo"


def test_ask_user_question_rejects_invalid_payload() -> None:
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading