Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,10 @@ Key conventions for the two tiers:
and example URLs rather than a docs-site domain, so they keep working when
exported.
- **Supported consumer surface:** the consumer tier currently ships one skill,
`skills/integrations/consumer/nemo-fabric-integrate`, for typed Python SDK
integration. Future public harness adapter-authoring skills belong under
`skills/integrations/harness/`; repository-internal adapter maintenance stays
in `.agents/skills/`.
`skills/nemo-fabric-integrate`, for typed Python SDK integration. Future
consumer skills also live directly under `skills/` so they can be validated
and published independently; repository-internal adapter maintenance stays in
`.agents/skills/`.

## Documentation And Contribution Workflow

Expand Down
27 changes: 13 additions & 14 deletions skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,10 @@ documentation URLs, never on repository-internal paths.
## Using an Integration Skill in Your Project

For example, copy the individual skill directory
`integrations/consumer/nemo-fabric-integrate/`, including its `references/`,
into the place your coding agent discovers skills **in your own project**. Copy
the skill bundle itself, not its `consumer/` or `harness/` category directory.
Do not rely on this repository's maintainer wiring (its `.claude/skills` symlink
or `.agents/skills/` set); those serve NeMo Fabric's own contributors.
`nemo-fabric-integrate/`, including its `references/`, into the place your coding
agent discovers skills **in your own project**. Do not rely on this repository's
maintainer wiring (its `.claude/skills` symlink or `.agents/skills/` set); those
serve NeMo Fabric's own contributors.

- **Claude Code:** place it at `.claude/skills/nemo-fabric-integrate/` in your
project, or `~/.claude/skills/nemo-fabric-integrate/` to use it across
Expand All @@ -53,20 +52,20 @@ or `.agents/skills/` set); those serve NeMo Fabric's own contributors.

## Consumer Integrations

Consumer integration skills live under `integrations/consumer/`. The following
skill helps software on the consumer side call NeMo Fabric through its public
SDK:
Consumer integration skills live directly under `skills/` so each bundle can be
validated and published independently. The following skill helps software on
the consumer side call NeMo Fabric through its public SDK:

| Skill | Use it when |
| Skill | Use It When |
|---|---|
| [`nemo-fabric-integrate`](integrations/consumer/nemo-fabric-integrate/SKILL.md) | You are adding NeMo Fabric to a consumer application, service, evaluation harness, or platform through the typed Python SDK — building an in-memory `FabricConfig`, choosing the single-invocation convenience API or an explicitly started runtime, validating with `plan`/`doctor`, and consuming normalized results. |
| [`nemo-fabric-integrate`](nemo-fabric-integrate/SKILL.md) | You are adding NeMo Fabric to a consumer application, service, evaluation harness, or platform through the typed Python SDK — building an in-memory `FabricConfig`, choosing the single-invocation convenience API or an explicitly started runtime, validating with `plan`/`doctor`, and consuming normalized results. |

## Harness Integrations

Harness integration skills belong under `integrations/harness/`. A forthcoming
adapter-authoring skill will guide third-party harness authors through the
published adapter contract so they can build harness integrations that are
compatible with NeMo Fabric.
A forthcoming adapter-authoring skill can guide third-party harness authors
through the published adapter contract. Like other consumer skills, its bundle
will live directly under `skills/`; repository-internal adapter maintenance
remains in `.agents/skills/`.

## Conventions

Expand Down
Original file line number Diff line number Diff line change
@@ -1,23 +1,19 @@
---
name: nemo-fabric-integrate
description: Use this skill when integrating NeMo Fabric into a consumer application, service, evaluation harness, or platform through the typed Python SDK — translating the consumer's own application, job, or deployment config into an in-memory FabricConfig, choosing the single-invocation convenience API or an explicitly started runtime, validating with plan and doctor, and consuming normalized results, artifacts, and telemetry.
description: Use this skill when integrating NVIDIA NeMo Fabric into a consumer application, service, evaluation harness, or platform through the typed Python SDK — translating the consumer's own application, job, or deployment config into an in-memory FabricConfig, choosing the single-invocation convenience API or an explicitly started runtime, validating with plan and doctor, and consuming normalized results, artifacts, and telemetry.
license: Apache-2.0
metadata:
author: NVIDIA Corporation and Affiliates
---

# Integrate NeMo Fabric Through The Python SDK
# Integrate NVIDIA NeMo Fabric Through The Python SDK

Use this skill when a consumer codebase — an application, service, evaluation
harness, or platform — needs to run agent harnesses through NeMo Fabric's typed
Python SDK. The consumer owns its own configuration object and translates it
into an in-memory `FabricConfig`; NeMo Fabric owns adapter selection, the runtime
lifecycle, and normalized results.

Do not use this skill to author or modify NeMo Fabric adapters, change NeMo Fabric core or
its bindings, or maintain repository infrastructure. If you are contributing to
NeMo Fabric itself, use the maintainer skills in `.agents/skills/` instead.

## Integration Boundary

Stay on the public, in-memory contract. These rules keep a consumer integration
Expand All @@ -35,7 +31,7 @@ supported and upgrade-safe:
- Treat `runtime_id`, `invocation_id`, and `request_id` as opaque correlation
strings, not parsable or reusable state.

See [config-mapping.md](references/config-mapping.md) for how to translate a
Refer to [config-mapping.md](references/config-mapping.md) for how to translate a
consumer config object into `FabricConfig`, and for the full list of mechanics
that stay hidden behind this boundary.

Expand Down Expand Up @@ -296,9 +292,8 @@ else:
```

- Treat `status == "succeeded"` as the only success. Other terminal values
(`failed`, `cancelled`) are unsuccessful, and `error` may be `None` even then,
so branch on `status`, not on `error`. Read `status`, `error`, and `events`
before processing `output`.
(`failed`, `cancelled`) are unsuccessful, so branch on `status`, not on
`error`. Read `status`, `error`, and `events` before processing `output`.
- Capture `artifacts` and `telemetry` references as the returned evidence for
platforms and evaluations. Store and log `runtime_id`, `invocation_id`, and
`request_id` separately as opaque strings.
Expand All @@ -313,7 +308,7 @@ else:
invocation error the cleanup failure is attached to the original exception. Be
ready to handle a shutdown failure.

See [results-and-errors.md](references/results-and-errors.md) for the full
Refer to [results-and-errors.md](references/results-and-errors.md) for the full
result-field and error inventory, and
[sdk-api-inventory.md](references/sdk-api-inventory.md) for when to use each
`Fabric` and `Runtime` method.
Expand Down
58 changes: 58 additions & 0 deletions skills/nemo-fabric-integrate/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
[
{
"id": "nemo-fabric-integrate-001-python-service",
"question": "Add NeMo Fabric to this Python service so one request path can run either Codex or Hermes Agent. Keep our existing application configuration as the source of truth, validate the resolved setup before execution, and return normalized result data to the caller.",
"expected_skill": "nemo-fabric-integrate",
"expected_script": null,
"ground_truth": "The agent uses the public nemo_fabric Python SDK, maps the service configuration into an in-memory FabricConfig, selects the adapter through HarnessConfig, validates with plan and doctor, chooses the appropriate runtime lifecycle, and consumes the normalized invocation result without importing internal modules or reimplementing harness control.",
"expected_behavior": [
"Routes to nemo-fabric-integrate",
"Uses only public imports from nemo_fabric",
"Translates the service configuration into an in-memory FabricConfig",
"Selects Codex or Hermes Agent through HarnessConfig.adapter_id",
"Validates the resolved configuration with plan and doctor before execution",
"Consumes normalized status, output, artifacts, telemetry, and correlation identifiers",
"Does not import _native or manage adapter processes directly"
]
},
{
"id": "nemo-fabric-integrate-002-evaluation-platform",
"question": "Integrate NeMo Fabric into our evaluation runner. The runner needs repeated invocations, request correlation, artifact collection, telemetry, and an explicit shutdown point. Credentials must stay in environment variables.",
"expected_skill": "nemo-fabric-integrate",
"expected_script": null,
"ground_truth": "The agent follows the long-lived public SDK runtime workflow: construct a typed in-memory FabricConfig, configure credential environment-variable names, inspect plan and doctor results, start Fabric explicitly, use the runtime as an async context manager, invoke repeatedly with request IDs, read normalized results, and handle a possible FabricRuntimeError from shutdown.",
"expected_behavior": [
"Routes to nemo-fabric-integrate",
"Uses an explicitly started Fabric runtime as an async context manager for repeated invocations",
"Keeps credential values out of source and configuration literals",
"Calls plan and doctor before starting the runtime",
"Preserves request, invocation, and runtime identifiers as opaque correlation values",
"Collects normalized artifacts and telemetry from each result",
"Treats shutdown as attempted cleanup and handles a possible FabricRuntimeError"
]
},
{
"id": "nemo-fabric-integrate-003-adapter-contribution-negative",
"question": "Add a new FooHarness adapter package inside NeMo Fabric, including its descriptor, runner implementation, and repository tests.",
"expected_skill": null,
"expected_script": null,
"ground_truth": "This is a NeMo Fabric repository-maintainer task, not a consumer Python SDK integration. The agent must not route it to nemo-fabric-integrate and should use the repository's contribute-adapter maintainer workflow instead.",
Comment thread
exactlyallan marked this conversation as resolved.
"expected_behavior": [
"Does not route to nemo-fabric-integrate",
"Recognizes the request as repository-internal adapter development",
"Routes to the contribute-adapter maintainer workflow when it is available"
]
},
{
"id": "nemo-fabric-integrate-004-overview-negative",
"question": "Give me a short conceptual overview of NeMo Fabric and explain when a team would use it.",
"expected_skill": null,
"expected_script": null,
"ground_truth": "This is an explanation request rather than an implementation workflow. The agent answers from product documentation and does not invoke the focused nemo-fabric-integrate skill.",
"expected_behavior": [
"Does not route to nemo-fabric-integrate",
"Treats the request as product explanation rather than SDK implementation",
"Does not introduce integration code or configuration unless the user asks for it"
]
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ SPDX-License-Identifier: Apache-2.0

Translate the consumer's own application, job, or deployment object into a typed
`FabricConfig` in memory. The consumer keeps owning its configuration model;
NeMo Fabric only receives the validated slice it needs.
NVIDIA NeMo Fabric only receives the validated slice it needs.

## Public Config Models

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Every invocation that reaches the adapter boundary returns a normalized
| Field | Meaning |
| --- | --- |
| `status` | Terminal invocation status: `succeeded`, `failed`, or `cancelled`. Branch on this. |
| `error` | Structured `ErrorInfo`, or `None` — may be `None` even when `status` is not `succeeded`, so do not use it as the success signal. |
| `error` | Structured failure metadata when available. Branch on `status`, not on this field. |
| `output` | Harness output normalized to the configured output schema. |
| `artifacts` | Output files, logs, patches, and other materialized references. |
| `telemetry` | References to NVIDIA NeMo Relay or other telemetry streams from the run. |
Expand All @@ -35,16 +35,17 @@ else:

`runtime_id` identifies the runtime lifecycle, `invocation_id` identifies one
invocation within it, and `request_id` correlates the caller's request.
NeMo Fabric-generated values use type-specific prefixes such as `runtime-`,
`invocation-`, and `request-`; a caller may supply its own `request_id`. Store
Values generated by NVIDIA NeMo Fabric use type-specific prefixes such as
`runtime-`, `invocation-`, and `request-`; a caller can supply its own
`request_id`. Store
and log each field separately and treat every value as opaque — do not parse or
reuse the encoding.

## Error Hierarchy

All public SDK errors inherit from `FabricError`. NeMo Fabric raises these when it
cannot return a normalized result; it does not return a partial `RunResult`. See
the [errors reference](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/reference/api/python-library-reference/nemo_fabric.errors.md).
cannot return a normalized result; it does not return a partial `RunResult`.
Refer to the [errors reference](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/reference/api/python-library-reference/nemo_fabric.errors.md).

| Error | Meaning |
| --- | --- |
Expand All @@ -64,7 +65,7 @@ the [errors reference](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/refe
- The consumer owns job-level retries and rollout failure policy. NeMo Fabric marks a
runtime or invocation failed and returns structured error metadata when
possible, but does not retry by default.
- Transient failures may carry retryable error metadata. Capacity pressure
- Transient failures can carry retryable error metadata. Capacity pressure
surfaces as a structured error or event (busy, rate limited, backpressure).
The consumer decides whether to wait, retry, start a replacement runtime, or
escalate.
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ async with await fabric.start_runtime(config, base_dir=base) as runtime:

## Execution Model

NeMo Fabric separates configuration, planning, runtime lifecycle, and individual
invocations:
NVIDIA NeMo Fabric separates configuration, planning, runtime lifecycle, and
individual invocations:

```text
FabricConfig -> plan() -> RunPlan -> start_runtime() -> Runtime -> invoke() -> RunResult
Expand All @@ -74,6 +74,6 @@ FabricConfig -> plan() -> RunPlan -> start_runtime() -> Runtime -> invoke() -> R
Claude, Codex, Deep Agents, and Hermes Agent adapters retain their native runtime
resources in one local host. Local `process` and `python` adapters use the
same host lifecycle. A host crash or protocol timeout terminates the runtime;
the application may explicitly start a new runtime according to its policy.
the application can explicitly start a new runtime according to its policy.
- The application owns scheduling, queues, retries, and how many runtimes to
run. NeMo Fabric provides only the runtime contract.
Loading