Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 61 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,12 @@ uv add nemo-flow
npm install nemo-flow-node
```

The NeMo Flow CLI is offered as a separate crate:

```bash
cargo install nemo-flow-cli
```

For source builds, testing, and contribution workflow, see [CONTRIBUTING.md](CONTRIBUTING.md).

## Documentation
Expand All @@ -128,57 +134,88 @@ The table below summarizes the support level for each binding surface.
| Python | ✅ Fully Supported | Fully documented with Quick Start and Guides |
| Node.js | ✅ Fully Supported | Fully documented with Quick Start and Guides |
| Rust | ✅ Fully Supported | Fully documented with Quick Start and Guides |
| Coding-Agent CLI | 🚧 Experimental | Install with `cargo install nemo-flow-cli`. |
| NeMo Flow CLI | 🚧 Experimental | Install with `cargo install nemo-flow-cli`. |
| Go | 🚧 Experimental | Source-first under `go/nemo_flow`. |
| WebAssembly | 🚧 Experimental | Source-first under `crates/wasm`. |
| FFI | 🚧 Experimental | Source-first under `crates/ffi`. |

## Agent Harness Support

NeMo Flow CLI offers experimental support for several agent harnesses.
Refer to the NeMo Flow CLI documentation for additional information.
Comment thread
willkill07 marked this conversation as resolved.

Below is our support matrix for agent harnesses.

| Agent | Observability | Security | Optimization | Notes |
|:--|:--:|:--:|:--:|:--|
| Claude Code | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues |
| Codex | ✅ Yes | ❌ No | ❌ No | Observability only; missing some necessary hooks for full features |
| Hermes Agent | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues |
| Cursor | ✅ Yes | ❌ No | ❌ No | Observability only; not feature-rich, missing hooks under `cursor-agent` |

## Third-Party Integrations

Some framework integrations are maintained as packages in this repository. Other
sample integrations are maintained as patch sets against upstream projects.

### Public API-based Integrations
### Public API Integrations

Some integrations can be implemented using public APIs without patching. Public
API-based integrations live under language-specific integration packages such as
`python/nemo_flow/integrations/` and `integrations/`.

The OpenClaw observability plugin is available under `integrations/openclaw/`
and uses OpenClaw public plugin hooks plus the generic NeMo Flow plugin
Below is the support matrix for our public API integrations.

| Agent / Library | Observability | Security | Optimization | Notes |
|:--|:--:|:--:|:--:|:--|
| LangChain | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| LangGraph | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| Deep Agents | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| OpenClaw | ✅ Yes | ❌ No | ❌ No | Observability support; missing middleware for wrapped execution |

#### LangChain

The Python `nemo-flow` package ships several extras that offer comprehensive
middleware support for the following packages:

- LangChain
- LangGraph
- Deep Agents

See the [Python package README](python/nemo_flow/README.md) for more information.

#### OpenClaw

An OpenClaw plugin is available as a Node package `nemo-flow-openclaw`.
It relies on OpenClaw public plugin hooks plus the generic NeMo Flow plugin
configuration shape to export telemetry. See the
[OpenClaw package README](integrations/openclaw/README.md).
[OpenClaw package README](integrations/openclaw/README.md) for more information.

### Patch-based Integrations

Patch-based integrations offer experimental support. Our roadmap includes switching over to first-party plugins and packages where upstream extension points allow it.

Use [third_party/README.md](third_party/README.md) for the clone, checkout, and
patch-application workflow for those integrations.

### Support Matrix

The following table summarizes maintained third-party integrations and whether each provides observability, request intercepts, execution intercepts, and conditional execution.
The following table summarizes maintained third-party integrations and whether each provides observability, security, and optimization support.

| Integration | Method | Observability | Request Intercepts | Execution Intercepts | Conditional Execution |
|---|---|---|---|---|---|
| [LangChain](third_party/README-langchain.md), [LangGraph](third_party/README-langgraph.md), [LangChain NVIDIA](third_party/README-langchain-nvidia.md) | 🚧 Patch | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
| [opencode](third_party/README-opencode.md) | 🚧 Patch | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
| [OpenClaw](integrations/openclaw/README.md) | `nemo-flow-openclaw` package, `nemo-flow` plugin ID | ✅ Yes | ❌ No | ❌ No | ❌ No |
| [Coding-Agent CLI](docs/integrate-frameworks/coding-agent-gateway.md) | `nemo-flow-cli` package for closed harnesses | ✅ Yes | ❌ No | ❌ No | ❌ No |
| [Hermes Agent](third_party/README-hermes-agent.md) | 🚧 Patch | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |

Patch-based integrations offer experimental support. Our roadmap includes switching over to first-party plugins and packages where upstream extension points allow it.
| Integration | Observability | Security | Optimization | Notes |
|:---|:---:|:---:|:---:|:---|
| [LangChain](third_party/README-langchain.md), [LangGraph](third_party/README-langgraph.md), [LangChain NVIDIA](third_party/README-langchain-nvidia.md) | ✅ Yes | ✅ Yes | ✅ Yes | Directly patches behavior into code requiring no middleware |
| [opencode](third_party/README-opencode.md) | ✅ Yes | ✅ Yes | ✅ Yes | Directly patches behavior into code |
| [OpenClaw](third_party/README-openclaw.md) | ✅ Yes | ✅ Yes | ✅ Yes | Adds new middleware support to OpenClaw and a built-in plugin |
| [Hermes Agent](third_party/README-hermes-agent.md) | ✅ Yes | ✅ Yes | ✅ Yes | Directly patches behavior into code |

## Roadmap

The following roadmap outlines planned features and integrations for upcoming releases.

- NemoClaw support and integration for managed tool and LLM execution flows.
- Deeper NVIDIA NeMo ecosystem integration across agent, guardrail, evaluation, and
observability workflows.
- Expanded adaptive optimization capabilities for performance-aware scheduling,
hints, and cache behavior.
- First-party plugins and/or packages for common agent runtimes and frameworks.
- [ ] NemoClaw support and integration for managed tool and LLM execution flows.
- [ ] Deeper NVIDIA NeMo ecosystem integration across agent, guardrail, evaluation, and observability workflows.
- [ ] Expanded adaptive optimization capabilities for performance-aware scheduling, hints, and cache behavior.
- [ ] First-party plugins and/or packages for common agent runtimes and frameworks.

## License

NeMo Flow is licensed under the [Apache License 2.0](LICENSE). All source files must include SPDX license headers.
NeMo Flow is licensed under the [Apache License 2.0](LICENSE).
9 changes: 5 additions & 4 deletions crates/adaptive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0
[![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm)
[![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow)
[![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive)
[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow)

# nemo-flow-adaptive
# NeMo Flow
Comment thread
bbednarski9 marked this conversation as resolved.

`nemo-flow-adaptive` is the Rust companion crate for adaptive NeMo Flow
runtime behavior. Use it with `nemo-flow` when an agent runtime should learn
Expand Down Expand Up @@ -46,9 +47,9 @@ framework.
the `redis-backend` feature.
- ✅ **Learning primitives**: Runtime helpers and learners built on NeMo Flow
events.
- ✅ **ACG module surface**: The canonical `nemo_flow_adaptive::acg` module for
PromptIR, provider plugins, stability analysis, and cache telemetry
normalization.
- ✅ **Adaptive Cache Governor (ACG) module surface**: The canonical
`nemo_flow_adaptive::acg` module for PromptIR, provider plugins, stability
analysis, and cache telemetry normalization.

## Installation

Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg/anthropic_plugin.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Anthropic cache plugin for the ACG system.
//! Anthropic cache plugin for the Adaptive Cache Governor (ACG) system.
//!
//! Translates ACG stability classifications into Anthropic-specific
//! `cache_control` breakpoints on content blocks. Implements the
Expand Down
8 changes: 4 additions & 4 deletions crates/adaptive/src/acg/error.rs
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Error types for the ACG (Adaptive Cache Governor) crate.
//! Error types for the Adaptive Cache Governor (ACG) crate.
//!
//! All fallible operations in the ACG system return [`Result<T>`], which uses
//! [`AcgError`] as the error type.
//! All fallible operations in the Adaptive Cache Governor (ACG) system return
//! [`Result<T>`], which uses [`AcgError`] as the error type.

use thiserror::Error;

/// The error type for all ACG operations.
/// The error type for all Adaptive Cache Governor (ACG) operations.
#[derive(Debug, Error)]
pub enum AcgError {
/// An intent validation failed.
Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg/ir_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ use crate::acg::prompt_ir::{
///
/// The builder preserves prompt order, inserts tool-schema blocks before the
/// first non-system message when tools are present, and computes the request
/// hashes needed by downstream ACG analysis.
/// hashes needed by downstream Adaptive Cache Governor (ACG) analysis.
///
/// # Parameters
/// - `request`: Annotated LLM request to normalize.
Expand Down
6 changes: 4 additions & 2 deletions crates/adaptive/src/acg/mod.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Canonical ACG module surface exposed from the adaptive crate.
//! Canonical Adaptive Cache Governor (ACG) module surface exposed from the
//! adaptive crate.

/// Minimum observations required before ACG emits optimization intents.
/// Minimum observations required before Adaptive Cache Governor (ACG) emits
/// optimization intents.
pub const MIN_ACG_OBSERVATIONS: u32 = 2;

pub mod anthropic_plugin;
Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg/openai_plugin.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! OpenAI cache plugin for the ACG system.
//! OpenAI cache plugin for the Adaptive Cache Governor (ACG) system.
//!
//! Maximizes automatic prefix cache hits through deterministic JSON
//! serialization. OpenAI uses automatic prefix caching at 1024+ tokens
Expand Down
7 changes: 3 additions & 4 deletions crates/adaptive/src/acg/plugin.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Provider plugin trait and input/output types for the ACG system.
//! Provider plugin trait and input/output types for the Adaptive Cache Governor
//! (ACG) system.
//!
//! The [`ProviderPlugin`] trait defines the contract between ACG's
//! provider-agnostic optimization pipeline and backend-specific
Expand All @@ -13,16 +14,14 @@
//! # Design
//!
//! - **Synchronous**: `translate` is a pure data transform (JSON
//! restructuring), not an I/O operation. This matches the [`LlmCodec`]
//! restructuring), not an I/O operation. This matches the `LlmCodec`
//! pattern in `crates/core/src/codec/traits.rs`.
//! - **Compatibility facade**: provider plugins keep their existing
//! synchronous trait surface, but can internally split translation into
//! semantic hint planning plus request-surface application.
//! - **`Send + Sync`**: Required for storage as `Arc<dyn ProviderPlugin>`
//! in concurrent contexts.
//! - **Object-safe**: The trait is designed to be used as a trait object.
//!
//! [`LlmCodec`]: nemo_flow::codec::LlmCodec

use nemo_flow::api::llm::LlmRequest;

Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg/policy.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Policy types for the ACG system.
//! Policy types for the Adaptive Cache Governor (ACG) system.

use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg/profile.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Behavioral profile types for the ACG system.
//! Behavioral profile types for the Adaptive Cache Governor (ACG) system.

use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
Expand Down
3 changes: 2 additions & 1 deletion crates/adaptive/src/acg/prompt_ir.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Prompt Intermediate Representation (IR) types for the ACG system.
//! Prompt Intermediate Representation (IR) types for the Adaptive Cache
//! Governor (ACG) system.
//!
//! The Prompt IR decomposes LLM conversations into addressable blocks
//! with structural metadata for cache analysis and prompt rewriting.
Expand Down
3 changes: 2 additions & 1 deletion crates/adaptive/src/acg/telemetry.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Cache performance telemetry types for the ACG system.
//! Cache performance telemetry types for the Adaptive Cache Governor (ACG)
//! system.
//!
//! These types normalize provider-specific cache metrics (Anthropic
//! `cache_read_input_tokens`/`cache_creation_input_tokens`, OpenAI
Expand Down
5 changes: 3 additions & 2 deletions crates/adaptive/src/acg/types.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Core data types for the ACG (Adaptive Cache Governor) crate.
//! Core data types for the Adaptive Cache Governor (ACG) crate.
//!
//! This module defines the vocabulary types used by the ACG system:
//! This module defines the vocabulary types used by the Adaptive Cache
//! Governor (ACG) system:
//! [`OptimizationIntent`] enum with 9 variants, per-variant payload structs,
//! [`OptimizationIntentBundle`], [`AgentIdentity`], and supporting enums
//! ([`SharingScope`], [`RetentionTier`], [`PlacementTarget`], [`ModelClass`],
Expand Down
3 changes: 2 additions & 1 deletion crates/adaptive/src/acg_component.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! ACG request and execution intercept helpers for the adaptive runtime.
//! Adaptive Cache Governor (ACG) request and execution intercept helpers for
//! the adaptive runtime.

use std::fmt::Display;
use std::future::Future;
Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/acg_learner.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! ACG learner for the adaptive telemetry pipeline.
//! Adaptive Cache Governor (ACG) learner for the adaptive telemetry pipeline.

use std::collections::{HashMap, VecDeque};
use std::future::Future;
Expand Down
3 changes: 2 additions & 1 deletion crates/adaptive/src/acg_profile.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Derives stable ACG profile keys from structured LLM requests.
//! Derives stable Adaptive Cache Governor (ACG) profile keys from structured
//! LLM requests.

use crate::acg::canonicalize::{canonicalize_value, sha256_hex};
use nemo_flow::codec::request::{
Expand Down
2 changes: 1 addition & 1 deletion crates/adaptive/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ impl Default for ToolParallelismComponentConfig {
}
}

/// Typed helper for the built-in ACG component.
/// Typed helper for the built-in Adaptive Cache Governor (ACG) component.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AcgComponentConfig {
/// Which provider plugin to activate (e.g. "anthropic", "openai", "passthrough").
Expand Down
3 changes: 2 additions & 1 deletion crates/adaptive/src/intercepts.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! Intercept factories for the `nemo-flow-adaptive` crate.
//! Intercept factories for the `nemo-flow-adaptive` crate, including Adaptive
//! Cache Governor (ACG) intercepts.

use std::collections::HashMap;
use std::future::Future;
Expand Down
4 changes: 2 additions & 2 deletions crates/adaptive/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
//! Adaptive behavior is enabled through the generic core plugin system.
//!
//! This crate provides the adaptive runtime, persistence abstractions, learner
//! implementations, and ACG analysis types used to derive and apply runtime
//! hints from observed NeMo Flow executions.
//! implementations, and Adaptive Cache Governor (ACG) analysis types used to
//! derive and apply runtime hints from observed NeMo Flow executions.
pub mod acg;
pub mod acg_component;
pub mod acg_learner;
Expand Down
13 changes: 6 additions & 7 deletions crates/adaptive/src/plugin_component.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ use std::future::Future;
use std::pin::Pin;
use std::sync::{Arc, Mutex};

use nemo_flow::plugin::Result as CorePluginResult;
use nemo_flow::plugin::{
ConfigDiagnostic, ConfigPolicy, DiagnosticLevel, Plugin, PluginComponentSpec, PluginError,
PluginRegistration, PluginRegistrationContext, UnsupportedBehavior, deregister_plugin,
PluginRegistration, PluginRegistrationContext, Result, UnsupportedBehavior, deregister_plugin,
lookup_plugin, register_plugin,
};
use serde_json::{Map, Value as Json};
Expand Down Expand Up @@ -76,7 +75,7 @@ impl Plugin for AdaptivePlugin {
&'a self,
plugin_config: &Map<String, Json>,
ctx: &'a mut PluginRegistrationContext,
) -> Pin<Box<dyn Future<Output = CorePluginResult<()>> + Send + 'a>> {
) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'a>> {
let plugin_config = plugin_config.clone();
Box::pin(async move {
let config = parse_adaptive_config(&plugin_config)?;
Expand Down Expand Up @@ -112,8 +111,8 @@ impl Plugin for AdaptivePlugin {
/// that contain adaptive components.
///
/// # Returns
/// A core plugin [`Result`](CorePluginResult) that is `Ok(())` when the
/// adaptive component kind is available in the registry.
/// A core plugin [`Result`] that is `Ok(())` when the adaptive component kind
/// is available in the registry.
///
/// # Errors
/// Returns an error when registration fails for a reason other than an already
Expand All @@ -122,7 +121,7 @@ impl Plugin for AdaptivePlugin {
/// # Notes
/// Re-registering the adaptive component is treated as success when the
/// existing registration already resolves to the adaptive plugin kind.
pub fn register_adaptive_component() -> CorePluginResult<()> {
pub fn register_adaptive_component() -> Result<()> {
match register_plugin(Arc::new(AdaptivePlugin)) {
Ok(()) => Ok(()),
Err(PluginError::RegistrationFailed(message))
Expand Down Expand Up @@ -150,7 +149,7 @@ pub fn deregister_adaptive_component() -> bool {
deregister_plugin(ADAPTIVE_PLUGIN_KIND)
}

fn parse_adaptive_config(plugin_config: &Map<String, Json>) -> CorePluginResult<AdaptiveConfig> {
fn parse_adaptive_config(plugin_config: &Map<String, Json>) -> Result<AdaptiveConfig> {
serde_json::from_value(Json::Object(plugin_config.clone()))
.map_err(|err| PluginError::InvalidConfig(format!("invalid adaptive plugin config: {err}")))
}
Expand Down
Loading
Loading