diff --git a/README.md b/README.md index 3220f5a2..35bfcc4a 100644 --- a/README.md +++ b/README.md @@ -108,6 +108,12 @@ uv add nemo-flow npm install nemo-flow-node ``` +The NeMo Flow CLI is offered as a separate crate: + +```bash +cargo install nemo-flow-cli +``` + For source builds, testing, and contribution workflow, see [CONTRIBUTING.md](CONTRIBUTING.md). ## Documentation @@ -128,57 +134,88 @@ The table below summarizes the support level for each binding surface. | Python | βœ… Fully Supported | Fully documented with Quick Start and Guides | | Node.js | βœ… Fully Supported | Fully documented with Quick Start and Guides | | Rust | βœ… Fully Supported | Fully documented with Quick Start and Guides | -| Coding-Agent CLI | 🚧 Experimental | Install with `cargo install nemo-flow-cli`. | +| NeMo Flow CLI | 🚧 Experimental | Install with `cargo install nemo-flow-cli`. | | Go | 🚧 Experimental | Source-first under `go/nemo_flow`. | | WebAssembly | 🚧 Experimental | Source-first under `crates/wasm`. | | FFI | 🚧 Experimental | Source-first under `crates/ffi`. | +## Agent Harness Support + +NeMo Flow CLI offers experimental support for several agent harnesses. +Refer to the NeMo Flow CLI documentation for additional information. + +Below is our support matrix for agent harnesses. + +| Agent | Observability | Security | Optimization | Notes | +|:--|:--:|:--:|:--:|:--| +| Claude Code | βœ… Yes | ❌ No | ❌ No | Observability only; no known issues | +| Codex | βœ… Yes | ❌ No | ❌ No | Observability only; missing some necessary hooks for full features | +| Hermes Agent | βœ… Yes | ❌ No | ❌ No | Observability only; no known issues | +| Cursor | βœ… Yes | ❌ No | ❌ No | Observability only; not feature-rich, missing hooks under `cursor-agent` | + ## Third-Party Integrations Some framework integrations are maintained as packages in this repository. Other sample integrations are maintained as patch sets against upstream projects. -### Public API-based Integrations +### Public API Integrations Some integrations can be implemented using public APIs without patching. Public API-based integrations live under language-specific integration packages such as `python/nemo_flow/integrations/` and `integrations/`. -The OpenClaw observability plugin is available under `integrations/openclaw/` -and uses OpenClaw public plugin hooks plus the generic NeMo Flow plugin +Below is the support matrix for our public API integrations. + +| Agent / Library | Observability | Security | Optimization | Notes | +|:--|:--:|:--:|:--:|:--| +| LangChain | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped Tool and LLM calling | +| LangGraph | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped Tool and LLM calling | +| Deep Agents | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped Tool and LLM calling | +| OpenClaw | βœ… Yes | ❌ No | ❌ No | Observability support; missing middleware for wrapped execution | + +#### LangChain + +The Python `nemo-flow` package ships several extras that offer comprehensive +middleware support for the following packages: + +- LangChain +- LangGraph +- Deep Agents + +See the [Python package README](python/nemo_flow/README.md) for more information. + +#### OpenClaw + +An OpenClaw plugin is available as a Node package `nemo-flow-openclaw`. +It relies on OpenClaw public plugin hooks plus the generic NeMo Flow plugin configuration shape to export telemetry. See the -[OpenClaw package README](integrations/openclaw/README.md). +[OpenClaw package README](integrations/openclaw/README.md) for more information. ### Patch-based Integrations +Patch-based integrations offer experimental support. Our roadmap includes switching over to first-party plugins and packages where upstream extension points allow it. + Use [third_party/README.md](third_party/README.md) for the clone, checkout, and patch-application workflow for those integrations. -### Support Matrix - -The following table summarizes maintained third-party integrations and whether each provides observability, request intercepts, execution intercepts, and conditional execution. +The following table summarizes maintained third-party integrations and whether each provides observability, security, and optimization support. -| Integration | Method | Observability | Request Intercepts | Execution Intercepts | Conditional Execution | -|---|---|---|---|---|---| -| [LangChain](third_party/README-langchain.md), [LangGraph](third_party/README-langgraph.md), [LangChain NVIDIA](third_party/README-langchain-nvidia.md) | 🚧 Patch | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | -| [opencode](third_party/README-opencode.md) | 🚧 Patch | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | -| [OpenClaw](integrations/openclaw/README.md) | `nemo-flow-openclaw` package, `nemo-flow` plugin ID | βœ… Yes | ❌ No | ❌ No | ❌ No | -| [Coding-Agent CLI](docs/integrate-frameworks/coding-agent-gateway.md) | `nemo-flow-cli` package for closed harnesses | βœ… Yes | ❌ No | ❌ No | ❌ No | -| [Hermes Agent](third_party/README-hermes-agent.md) | 🚧 Patch | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | - -Patch-based integrations offer experimental support. Our roadmap includes switching over to first-party plugins and packages where upstream extension points allow it. +| Integration | Observability | Security | Optimization | Notes | +|:---|:---:|:---:|:---:|:---| +| [LangChain](third_party/README-langchain.md), [LangGraph](third_party/README-langgraph.md), [LangChain NVIDIA](third_party/README-langchain-nvidia.md) | βœ… Yes | βœ… Yes | βœ… Yes | Directly patches behavior into code requiring no middleware | +| [opencode](third_party/README-opencode.md) | βœ… Yes | βœ… Yes | βœ… Yes | Directly patches behavior into code | +| [OpenClaw](third_party/README-openclaw.md) | βœ… Yes | βœ… Yes | βœ… Yes | Adds new middleware support to OpenClaw and a built-in plugin | +| [Hermes Agent](third_party/README-hermes-agent.md) | βœ… Yes | βœ… Yes | βœ… Yes | Directly patches behavior into code | ## Roadmap The following roadmap outlines planned features and integrations for upcoming releases. -- NemoClaw support and integration for managed tool and LLM execution flows. -- Deeper NVIDIA NeMo ecosystem integration across agent, guardrail, evaluation, and - observability workflows. -- Expanded adaptive optimization capabilities for performance-aware scheduling, - hints, and cache behavior. -- First-party plugins and/or packages for common agent runtimes and frameworks. +- [ ] NemoClaw support and integration for managed tool and LLM execution flows. +- [ ] Deeper NVIDIA NeMo ecosystem integration across agent, guardrail, evaluation, and observability workflows. +- [ ] Expanded adaptive optimization capabilities for performance-aware scheduling, hints, and cache behavior. +- [ ] First-party plugins and/or packages for common agent runtimes and frameworks. ## License -NeMo Flow is licensed under the [Apache License 2.0](LICENSE). All source files must include SPDX license headers. +NeMo Flow is licensed under the [Apache License 2.0](LICENSE). diff --git a/crates/adaptive/README.md b/crates/adaptive/README.md index a8f95f78..7410c78b 100644 --- a/crates/adaptive/README.md +++ b/crates/adaptive/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow-adaptive +# NeMo Flow `nemo-flow-adaptive` is the Rust companion crate for adaptive NeMo Flow runtime behavior. Use it with `nemo-flow` when an agent runtime should learn @@ -46,9 +47,9 @@ framework. the `redis-backend` feature. - βœ… **Learning primitives**: Runtime helpers and learners built on NeMo Flow events. -- βœ… **ACG module surface**: The canonical `nemo_flow_adaptive::acg` module for - PromptIR, provider plugins, stability analysis, and cache telemetry - normalization. +- βœ… **Adaptive Cache Governor (ACG) module surface**: The canonical + `nemo_flow_adaptive::acg` module for PromptIR, provider plugins, stability + analysis, and cache telemetry normalization. ## Installation diff --git a/crates/adaptive/src/acg/anthropic_plugin.rs b/crates/adaptive/src/acg/anthropic_plugin.rs index 550742e5..885a73b1 100644 --- a/crates/adaptive/src/acg/anthropic_plugin.rs +++ b/crates/adaptive/src/acg/anthropic_plugin.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Anthropic cache plugin for the ACG system. +//! Anthropic cache plugin for the Adaptive Cache Governor (ACG) system. //! //! Translates ACG stability classifications into Anthropic-specific //! `cache_control` breakpoints on content blocks. Implements the diff --git a/crates/adaptive/src/acg/error.rs b/crates/adaptive/src/acg/error.rs index a9ef4af4..7b57c679 100644 --- a/crates/adaptive/src/acg/error.rs +++ b/crates/adaptive/src/acg/error.rs @@ -1,14 +1,14 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Error types for the ACG (Adaptive Cache Governor) crate. +//! Error types for the Adaptive Cache Governor (ACG) crate. //! -//! All fallible operations in the ACG system return [`Result`], which uses -//! [`AcgError`] as the error type. +//! All fallible operations in the Adaptive Cache Governor (ACG) system return +//! [`Result`], which uses [`AcgError`] as the error type. use thiserror::Error; -/// The error type for all ACG operations. +/// The error type for all Adaptive Cache Governor (ACG) operations. #[derive(Debug, Error)] pub enum AcgError { /// An intent validation failed. diff --git a/crates/adaptive/src/acg/ir_builder.rs b/crates/adaptive/src/acg/ir_builder.rs index 3c20dfe2..6f8584aa 100644 --- a/crates/adaptive/src/acg/ir_builder.rs +++ b/crates/adaptive/src/acg/ir_builder.rs @@ -21,7 +21,7 @@ use crate::acg::prompt_ir::{ /// /// The builder preserves prompt order, inserts tool-schema blocks before the /// first non-system message when tools are present, and computes the request -/// hashes needed by downstream ACG analysis. +/// hashes needed by downstream Adaptive Cache Governor (ACG) analysis. /// /// # Parameters /// - `request`: Annotated LLM request to normalize. diff --git a/crates/adaptive/src/acg/mod.rs b/crates/adaptive/src/acg/mod.rs index 2ad99193..fdd64a82 100644 --- a/crates/adaptive/src/acg/mod.rs +++ b/crates/adaptive/src/acg/mod.rs @@ -1,9 +1,11 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Canonical ACG module surface exposed from the adaptive crate. +//! Canonical Adaptive Cache Governor (ACG) module surface exposed from the +//! adaptive crate. -/// Minimum observations required before ACG emits optimization intents. +/// Minimum observations required before Adaptive Cache Governor (ACG) emits +/// optimization intents. pub const MIN_ACG_OBSERVATIONS: u32 = 2; pub mod anthropic_plugin; diff --git a/crates/adaptive/src/acg/openai_plugin.rs b/crates/adaptive/src/acg/openai_plugin.rs index 2c8fafd7..e54c113f 100644 --- a/crates/adaptive/src/acg/openai_plugin.rs +++ b/crates/adaptive/src/acg/openai_plugin.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! OpenAI cache plugin for the ACG system. +//! OpenAI cache plugin for the Adaptive Cache Governor (ACG) system. //! //! Maximizes automatic prefix cache hits through deterministic JSON //! serialization. OpenAI uses automatic prefix caching at 1024+ tokens diff --git a/crates/adaptive/src/acg/plugin.rs b/crates/adaptive/src/acg/plugin.rs index cd2d39aa..63d29157 100644 --- a/crates/adaptive/src/acg/plugin.rs +++ b/crates/adaptive/src/acg/plugin.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Provider plugin trait and input/output types for the ACG system. +//! Provider plugin trait and input/output types for the Adaptive Cache Governor +//! (ACG) system. //! //! The [`ProviderPlugin`] trait defines the contract between ACG's //! provider-agnostic optimization pipeline and backend-specific @@ -13,7 +14,7 @@ //! # Design //! //! - **Synchronous**: `translate` is a pure data transform (JSON -//! restructuring), not an I/O operation. This matches the [`LlmCodec`] +//! restructuring), not an I/O operation. This matches the `LlmCodec` //! pattern in `crates/core/src/codec/traits.rs`. //! - **Compatibility facade**: provider plugins keep their existing //! synchronous trait surface, but can internally split translation into @@ -21,8 +22,6 @@ //! - **`Send + Sync`**: Required for storage as `Arc` //! in concurrent contexts. //! - **Object-safe**: The trait is designed to be used as a trait object. -//! -//! [`LlmCodec`]: nemo_flow::codec::LlmCodec use nemo_flow::api::llm::LlmRequest; diff --git a/crates/adaptive/src/acg/policy.rs b/crates/adaptive/src/acg/policy.rs index b24d6257..32b8564b 100644 --- a/crates/adaptive/src/acg/policy.rs +++ b/crates/adaptive/src/acg/policy.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Policy types for the ACG system. +//! Policy types for the Adaptive Cache Governor (ACG) system. use chrono::{DateTime, Utc}; use serde::{Deserialize, Serialize}; diff --git a/crates/adaptive/src/acg/profile.rs b/crates/adaptive/src/acg/profile.rs index 013f2d69..f1f2e4a2 100644 --- a/crates/adaptive/src/acg/profile.rs +++ b/crates/adaptive/src/acg/profile.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Behavioral profile types for the ACG system. +//! Behavioral profile types for the Adaptive Cache Governor (ACG) system. use chrono::{DateTime, Utc}; use serde::{Deserialize, Serialize}; diff --git a/crates/adaptive/src/acg/prompt_ir.rs b/crates/adaptive/src/acg/prompt_ir.rs index caa66bec..89261e27 100644 --- a/crates/adaptive/src/acg/prompt_ir.rs +++ b/crates/adaptive/src/acg/prompt_ir.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Prompt Intermediate Representation (IR) types for the ACG system. +//! Prompt Intermediate Representation (IR) types for the Adaptive Cache +//! Governor (ACG) system. //! //! The Prompt IR decomposes LLM conversations into addressable blocks //! with structural metadata for cache analysis and prompt rewriting. diff --git a/crates/adaptive/src/acg/telemetry.rs b/crates/adaptive/src/acg/telemetry.rs index d2fb1e4c..6d3ae4a2 100644 --- a/crates/adaptive/src/acg/telemetry.rs +++ b/crates/adaptive/src/acg/telemetry.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Cache performance telemetry types for the ACG system. +//! Cache performance telemetry types for the Adaptive Cache Governor (ACG) +//! system. //! //! These types normalize provider-specific cache metrics (Anthropic //! `cache_read_input_tokens`/`cache_creation_input_tokens`, OpenAI diff --git a/crates/adaptive/src/acg/types.rs b/crates/adaptive/src/acg/types.rs index 66858f93..b13b0f5b 100644 --- a/crates/adaptive/src/acg/types.rs +++ b/crates/adaptive/src/acg/types.rs @@ -1,9 +1,10 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Core data types for the ACG (Adaptive Cache Governor) crate. +//! Core data types for the Adaptive Cache Governor (ACG) crate. //! -//! This module defines the vocabulary types used by the ACG system: +//! This module defines the vocabulary types used by the Adaptive Cache +//! Governor (ACG) system: //! [`OptimizationIntent`] enum with 9 variants, per-variant payload structs, //! [`OptimizationIntentBundle`], [`AgentIdentity`], and supporting enums //! ([`SharingScope`], [`RetentionTier`], [`PlacementTarget`], [`ModelClass`], diff --git a/crates/adaptive/src/acg_component.rs b/crates/adaptive/src/acg_component.rs index 31673429..666c80e8 100644 --- a/crates/adaptive/src/acg_component.rs +++ b/crates/adaptive/src/acg_component.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! ACG request and execution intercept helpers for the adaptive runtime. +//! Adaptive Cache Governor (ACG) request and execution intercept helpers for +//! the adaptive runtime. use std::fmt::Display; use std::future::Future; diff --git a/crates/adaptive/src/acg_learner.rs b/crates/adaptive/src/acg_learner.rs index 80044e38..5c0a31ae 100644 --- a/crates/adaptive/src/acg_learner.rs +++ b/crates/adaptive/src/acg_learner.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! ACG learner for the adaptive telemetry pipeline. +//! Adaptive Cache Governor (ACG) learner for the adaptive telemetry pipeline. use std::collections::{HashMap, VecDeque}; use std::future::Future; diff --git a/crates/adaptive/src/acg_profile.rs b/crates/adaptive/src/acg_profile.rs index 89170270..6551712b 100644 --- a/crates/adaptive/src/acg_profile.rs +++ b/crates/adaptive/src/acg_profile.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Derives stable ACG profile keys from structured LLM requests. +//! Derives stable Adaptive Cache Governor (ACG) profile keys from structured +//! LLM requests. use crate::acg::canonicalize::{canonicalize_value, sha256_hex}; use nemo_flow::codec::request::{ diff --git a/crates/adaptive/src/config.rs b/crates/adaptive/src/config.rs index 7907a07d..323446a4 100644 --- a/crates/adaptive/src/config.rs +++ b/crates/adaptive/src/config.rs @@ -149,7 +149,7 @@ impl Default for ToolParallelismComponentConfig { } } -/// Typed helper for the built-in ACG component. +/// Typed helper for the built-in Adaptive Cache Governor (ACG) component. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct AcgComponentConfig { /// Which provider plugin to activate (e.g. "anthropic", "openai", "passthrough"). diff --git a/crates/adaptive/src/intercepts.rs b/crates/adaptive/src/intercepts.rs index a9be6d05..df4a49d9 100644 --- a/crates/adaptive/src/intercepts.rs +++ b/crates/adaptive/src/intercepts.rs @@ -1,7 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Intercept factories for the `nemo-flow-adaptive` crate. +//! Intercept factories for the `nemo-flow-adaptive` crate, including Adaptive +//! Cache Governor (ACG) intercepts. use std::collections::HashMap; use std::future::Future; diff --git a/crates/adaptive/src/lib.rs b/crates/adaptive/src/lib.rs index 7ed1924d..3dd2d7af 100644 --- a/crates/adaptive/src/lib.rs +++ b/crates/adaptive/src/lib.rs @@ -7,8 +7,8 @@ //! Adaptive behavior is enabled through the generic core plugin system. //! //! This crate provides the adaptive runtime, persistence abstractions, learner -//! implementations, and ACG analysis types used to derive and apply runtime -//! hints from observed NeMo Flow executions. +//! implementations, and Adaptive Cache Governor (ACG) analysis types used to +//! derive and apply runtime hints from observed NeMo Flow executions. pub mod acg; pub mod acg_component; pub mod acg_learner; diff --git a/crates/adaptive/src/plugin_component.rs b/crates/adaptive/src/plugin_component.rs index 000c21db..15cc731d 100644 --- a/crates/adaptive/src/plugin_component.rs +++ b/crates/adaptive/src/plugin_component.rs @@ -7,10 +7,9 @@ use std::future::Future; use std::pin::Pin; use std::sync::{Arc, Mutex}; -use nemo_flow::plugin::Result as CorePluginResult; use nemo_flow::plugin::{ ConfigDiagnostic, ConfigPolicy, DiagnosticLevel, Plugin, PluginComponentSpec, PluginError, - PluginRegistration, PluginRegistrationContext, UnsupportedBehavior, deregister_plugin, + PluginRegistration, PluginRegistrationContext, Result, UnsupportedBehavior, deregister_plugin, lookup_plugin, register_plugin, }; use serde_json::{Map, Value as Json}; @@ -76,7 +75,7 @@ impl Plugin for AdaptivePlugin { &'a self, plugin_config: &Map, ctx: &'a mut PluginRegistrationContext, - ) -> Pin> + Send + 'a>> { + ) -> Pin> + Send + 'a>> { let plugin_config = plugin_config.clone(); Box::pin(async move { let config = parse_adaptive_config(&plugin_config)?; @@ -112,8 +111,8 @@ impl Plugin for AdaptivePlugin { /// that contain adaptive components. /// /// # Returns -/// A core plugin [`Result`](CorePluginResult) that is `Ok(())` when the -/// adaptive component kind is available in the registry. +/// A core plugin [`Result`] that is `Ok(())` when the adaptive component kind +/// is available in the registry. /// /// # Errors /// Returns an error when registration fails for a reason other than an already @@ -122,7 +121,7 @@ impl Plugin for AdaptivePlugin { /// # Notes /// Re-registering the adaptive component is treated as success when the /// existing registration already resolves to the adaptive plugin kind. -pub fn register_adaptive_component() -> CorePluginResult<()> { +pub fn register_adaptive_component() -> Result<()> { match register_plugin(Arc::new(AdaptivePlugin)) { Ok(()) => Ok(()), Err(PluginError::RegistrationFailed(message)) @@ -150,7 +149,7 @@ pub fn deregister_adaptive_component() -> bool { deregister_plugin(ADAPTIVE_PLUGIN_KIND) } -fn parse_adaptive_config(plugin_config: &Map) -> CorePluginResult { +fn parse_adaptive_config(plugin_config: &Map) -> Result { serde_json::from_value(Json::Object(plugin_config.clone())) .map_err(|err| PluginError::InvalidConfig(format!("invalid adaptive plugin config: {err}"))) } diff --git a/crates/adaptive/src/storage/traits.rs b/crates/adaptive/src/storage/traits.rs index b9de57aa..94e2a000 100644 --- a/crates/adaptive/src/storage/traits.rs +++ b/crates/adaptive/src/storage/traits.rs @@ -78,7 +78,8 @@ pub trait StorageBackendDyn: Send + Sync + 'static { Ok(()) } - /// Persist prompt IR observations for an agent or derived ACG profile. + /// Persist prompt IR observations for an agent or derived Adaptive Cache + /// Governor (ACG) profile. /// /// # Notes /// The default implementation is a no-op. @@ -90,7 +91,8 @@ pub trait StorageBackendDyn: Send + Sync + 'static { Box::pin(async move { Ok(()) }) } - /// Load prompt IR observations for an agent or derived ACG profile. + /// Load prompt IR observations for an agent or derived Adaptive Cache + /// Governor (ACG) profile. /// /// # Notes /// The default implementation returns `Ok(None)`. diff --git a/crates/adaptive/src/types/cache.rs b/crates/adaptive/src/types/cache.rs index a5c04fba..cf02dc20 100644 --- a/crates/adaptive/src/types/cache.rs +++ b/crates/adaptive/src/types/cache.rs @@ -14,8 +14,8 @@ use crate::types::plan::ExecutionPlan; /// /// The adaptive runtime keeps this structure in an [`std::sync::RwLock`] so /// intercepts and event-processing tasks can exchange recently learned plans, -/// trie state, and ACG summaries without hitting the configured backend on -/// every request. +/// trie state, and Adaptive Cache Governor (ACG) summaries without hitting the +/// configured backend on every request. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct HotCache { /// Current execution plan used for tool parallelism hints. diff --git a/crates/adaptive/src/types/records.rs b/crates/adaptive/src/types/records.rs index 4d04f969..aae750e6 100644 --- a/crates/adaptive/src/types/records.rs +++ b/crates/adaptive/src/types/records.rs @@ -51,10 +51,12 @@ pub struct CallRecord { /// Number of tool calls issued by the provider, when available. #[serde(skip_serializing_if = "Option::is_none", default)] pub tool_call_count: Option, - /// Annotated request captured for ACG analysis, when available. + /// Annotated request captured for Adaptive Cache Governor (ACG) analysis, + /// when available. #[serde(skip_serializing_if = "Option::is_none", default)] pub annotated_request: Option>, - /// Annotated response captured for ACG analysis, when available. + /// Annotated response captured for Adaptive Cache Governor (ACG) analysis, + /// when available. #[serde(skip_serializing_if = "Option::is_none", default)] pub annotated_response: Option>, } diff --git a/crates/cli/README.md b/crates/cli/README.md index 39313e1a..848898ae 100644 --- a/crates/cli/README.md +++ b/crates/cli/README.md @@ -15,12 +15,12 @@ SPDX-License-Identifier: Apache-2.0 [![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow-cli +# NeMo Flow -`nemo-flow-cli` is the coding-agent gateway CLI for NeMo Flow observability. -It installs the `nemo-flow` binary, which can configure supported coding-agent -hooks, run agents through an ephemeral gateway, and diagnose local agent and -exporter readiness. +`nemo-flow-cli` installs the NeMo Flow CLI, the `nemo-flow` binary for local +coding-agent observability. It can configure supported coding-agent hooks, run +agents through an ephemeral gateway, and diagnose local agent and exporter +readiness. The CLI is a Rust package in this repository, but most users should interact with the installed `nemo-flow` command rather than link against the crate. @@ -28,7 +28,8 @@ with the installed `nemo-flow` command rather than link against the crate. ## Why Use It? - 🧭 **Observe existing coding agents**: Run Claude Code, Codex, Cursor, or - Hermes through a local NeMo Flow gateway without changing the agent itself. + Hermes Agent through a local NeMo Flow gateway without changing the agent + itself. - πŸ› οΈ **Configure hooks interactively**: Use the setup wizard to write project or user config and install the hook files needed by supported agents. - πŸ“‘ **Export local sessions**: Write ATIF trajectory files, ATOF event JSONL @@ -63,13 +64,6 @@ That command installs the binary as: nemo-flow --version ``` -For local development, build and test the package directly: - -```bash -cargo build -p nemo-flow-cli -cargo test -p nemo-flow-cli -``` - ## Getting Started Run the first-time setup wizard: @@ -104,6 +98,12 @@ Project config lives at `./.nemo-flow/config.toml`; user config lives at The project layer overrides system config, and the user layer overrides the project layer. +General options are configured through the top-level config. Edit the config with: + +```bash +nemo-flow config +``` + Observability exporters are configured through the plugin config. Edit the user plugin config with: @@ -132,4 +132,4 @@ output_directory = "./atif" ## Documentation -NeMo Flow Documentation: https://nvidia.github.io/NeMo-Flow +NeMo Flow Documentation: https://nvidia.github.io/NeMo-Flow/ diff --git a/crates/cli/src/launcher.rs b/crates/cli/src/launcher.rs index 6ff4b5e7..e3601076 100644 --- a/crates/cli/src/launcher.rs +++ b/crates/cli/src/launcher.rs @@ -372,7 +372,7 @@ impl PreparedRun { // Injects Codex hook and provider configuration through repeated `--config` flags. Codex // reserves built-in provider IDs, so run mode installs a temporary provider alias instead of // overriding `model_providers.openai`. Uses `features.hooks=true` introduced in codex-cli - // 0.129; the older `features.codex_hooks` is deprecated. Requires codex-cli >= 0.129.0. + // 0.129. Requires codex-cli >= 0.129.0. fn prepare_codex(&mut self, gateway_url: &str) { // Codex resolves auth via `CodexAuth::from_auth_dot_json` (`codex-rs/login/src/auth/ // manager.rs`): `auth_mode=ApiKey` uses `OPENAI_API_KEY`, `auth_mode=Chatgpt` uses the diff --git a/crates/core/README.md b/crates/core/README.md index 02db9192..b0ec834c 100644 --- a/crates/core/README.md +++ b/crates/core/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow +# NeMo Flow `nemo-flow` is the core Rust SDK for NeMo Flow, a portable execution runtime for agent systems. Use it when a Rust application, framework adapter, @@ -31,7 +32,8 @@ Node.js bindings mirror the semantics exposed by this crate. - πŸ›‘οΈ **Put policy around real calls**: Guardrails and intercepts can block work, sanitize observability payloads, rewrite requests, or wrap execution. - πŸ“‘ **Emit one lifecycle stream**: Subscribers can consume canonical runtime - events in-process or export them to ATIF, OpenTelemetry, and OpenInference. + events in-process or export them to Agent Trajectory Interchange Format + (ATIF), OpenTelemetry, and OpenInference. - 🧩 **Integrate without changing orchestration**: Wrap framework and provider callbacks while leaving scheduling, retries, memory, and result handling in the owning application. @@ -45,8 +47,8 @@ Node.js bindings mirror the semantics exposed by this crate. - βœ… **Plugin primitives**: Register reusable runtime behavior configured from one shared plugin system. - βœ… **Built-in observability plugin**: Configure first-party Agent Trajectory - Observability Format (ATOF), ATIF, OpenTelemetry, and OpenInference exporters - from the core crate. + Observability Format (ATOF), Agent Trajectory Interchange Format (ATIF), + OpenTelemetry, and OpenInference exporters from the core crate. - βœ… **Codec and typed helpers**: Normalize provider requests and responses for framework integrations. - βœ… **Binding source of truth**: Use the runtime semantics mirrored by the diff --git a/crates/core/src/api/event.rs b/crates/core/src/api/event.rs index ae85eb3e..e7748a17 100644 --- a/crates/core/src/api/event.rs +++ b/crates/core/src/api/event.rs @@ -13,6 +13,8 @@ // See the License for the specific language governing permissions and // limitations under the License. +//! Event types for Agent Trajectory Observability Format (ATOF) runtime events. + use std::collections::BTreeMap; use std::sync::Arc; @@ -28,7 +30,7 @@ use crate::codec::request::AnnotatedLlmRequest; use crate::codec::response::AnnotatedLlmResponse; use crate::json::Json; -/// ATOF protocol version emitted by this runtime. +/// Agent Trajectory Observability Format (ATOF) protocol version emitted by this runtime. pub const ATOF_VERSION: &str = "0.1"; /// Identifier for the schema that describes an event's opaque `data` payload. @@ -158,7 +160,7 @@ impl From<&EventCategory> for ScopeType { } } -/// ATOF lifecycle phase for a scope event. +/// Agent Trajectory Observability Format (ATOF) lifecycle phase for a scope event. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] pub enum ScopeCategory { @@ -171,7 +173,8 @@ pub enum ScopeCategory { /// Category-specific profile data. /// /// Unknown wire keys are preserved in `extra`. LLM annotations are runtime-only -/// enrichment used by internal adaptive/ATIF logic and are never serialized. +/// enrichment used by internal adaptive and Agent Trajectory Interchange Format +/// (ATIF) logic and are never serialized. #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize, TypedBuilder)] #[builder(field_defaults(setter(into, strip_option(ignore_invalid, fallback_suffix = "_opt"))))] pub struct CategoryProfile { diff --git a/crates/core/src/api/llm.rs b/crates/core/src/api/llm.rs index e95242b6..b3b1554e 100644 --- a/crates/core/src/api/llm.rs +++ b/crates/core/src/api/llm.rs @@ -143,7 +143,8 @@ pub struct LlmCallParams<'a> { /// LLM attribute bitflags applied to the span. #[builder(default = LlmAttributes::empty())] pub attributes: LlmAttributes, - /// Optional application payload stored on the handle but not emitted as ATOF data. + /// Optional application payload stored on the handle but not emitted as + /// Agent Trajectory Observability Format (ATOF) data. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on the start event. @@ -178,7 +179,8 @@ pub struct LlmCallExecuteParams { /// LLM attribute bitflags applied to the managed span. #[builder(default = LlmAttributes::empty())] pub attributes: LlmAttributes, - /// Optional application payload stored on the handle but not emitted as ATOF data. + /// Optional application payload stored on the handle but not emitted as + /// Agent Trajectory Observability Format (ATOF) data. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on emitted events. @@ -216,7 +218,8 @@ pub struct LlmStreamCallExecuteParams { /// LLM attribute bitflags applied to the managed span. #[builder(default = LlmAttributes::empty())] pub attributes: LlmAttributes, - /// Optional application payload stored on the handle but not emitted as ATOF data. + /// Optional application payload stored on the handle but not emitted as + /// Agent Trajectory Observability Format (ATOF) data. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on emitted events. @@ -241,7 +244,8 @@ pub struct LlmCallEndParams<'a> { pub handle: &'a LlmHandle, /// Raw provider response associated with the end event. pub response: Json, - /// Optional application payload retained for compatibility; ATOF data is the response. + /// Optional application payload retained for compatibility; Agent + /// Trajectory Observability Format (ATOF) data is the response. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on the end event. diff --git a/crates/core/src/api/scope.rs b/crates/core/src/api/scope.rs index 4c79c8c3..dc8c75f0 100644 --- a/crates/core/src/api/scope.rs +++ b/crates/core/src/api/scope.rs @@ -254,7 +254,8 @@ pub fn get_handle() -> Result { /// - `attributes`: Bitflags that modify scope behavior and observability. /// - `data`: Optional application payload stored on the returned handle. /// - `metadata`: Optional JSON metadata recorded on the emitted start event. -/// - `input`: Optional JSON payload exported as the ATOF data payload. +/// - `input`: Optional JSON payload exported as the Agent Trajectory +/// Observability Format (ATOF) data payload. /// - `timestamp`: Optional timestamp recorded as the handle start time and on /// the emitted start event. When `None`, the current UTC time is used. /// diff --git a/crates/core/src/api/tool.rs b/crates/core/src/api/tool.rs index b123d629..4a9d3331 100644 --- a/crates/core/src/api/tool.rs +++ b/crates/core/src/api/tool.rs @@ -117,7 +117,8 @@ pub struct ToolCallParams<'a> { /// Tool attribute bitflags applied to the span. #[builder(default = ToolAttributes::empty())] pub attributes: ToolAttributes, - /// Optional application payload stored on the handle but not emitted as ATOF data. + /// Optional application payload stored on the handle but not emitted as + /// Agent Trajectory Observability Format (ATOF) data. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on the start event. @@ -149,7 +150,8 @@ pub struct ToolCallExecuteParams { /// Tool attribute bitflags applied to the managed span. #[builder(default = ToolAttributes::empty())] pub attributes: ToolAttributes, - /// Optional application payload stored on the handle but not emitted as ATOF data. + /// Optional application payload stored on the handle but not emitted as + /// Agent Trajectory Observability Format (ATOF) data. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on emitted events. @@ -165,7 +167,8 @@ pub struct ToolCallEndParams<'a> { pub handle: &'a ToolHandle, /// Raw tool result associated with the end event. pub result: Json, - /// Optional application payload retained for compatibility; ATOF data is the result. + /// Optional application payload retained for compatibility; Agent + /// Trajectory Observability Format (ATOF) data is the result. #[builder(default)] pub data: Option, /// Optional JSON metadata recorded on the end event. diff --git a/crates/core/src/observability/atif.rs b/crates/core/src/observability/atif.rs index a1c55578..ed2a1ec4 100644 --- a/crates/core/src/observability/atif.rs +++ b/crates/core/src/observability/atif.rs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! ATIF (Agent Trajectory Interchange Format) exporter. +//! Agent Trajectory Interchange Format (ATIF) exporter. //! //! This module provides types and an exporter that collects lifecycle events //! from the NeMo Flow runtime and converts them into ATIF trajectories conforming diff --git a/crates/core/src/observability/atof.rs b/crates/core/src/observability/atof.rs index 94c93b8d..bb7b35d7 100644 --- a/crates/core/src/observability/atof.rs +++ b/crates/core/src/observability/atof.rs @@ -1,10 +1,12 @@ // SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! ATOF JSONL exporter support for NeMo Flow. +//! Agent Trajectory Observability Format (ATOF) JSONL exporter support for NeMo +//! Flow. //! //! The [`AtofExporter`] registers as an event subscriber and writes each raw -//! NeMo Flow ATOF event as one JSON object per JSONL line. +//! NeMo Flow Agent Trajectory Observability Format (ATOF) event as one JSON +//! object per JSONL line. use std::fs::{File, OpenOptions}; use std::io::{BufWriter, Write}; @@ -144,7 +146,7 @@ struct AtofExporterState { last_error: Option, } -/// Filesystem-backed ATOF JSONL event exporter. +/// Filesystem-backed Agent Trajectory Observability Format (ATOF) JSONL event exporter. pub struct AtofExporter { path: PathBuf, state: Arc>, diff --git a/crates/core/src/observability/plugin_component.rs b/crates/core/src/observability/plugin_component.rs index 8b0e3c4a..a1ba7274 100644 --- a/crates/core/src/observability/plugin_component.rs +++ b/crates/core/src/observability/plugin_component.rs @@ -9,10 +9,11 @@ //! register subscribers or construct exporters. //! //! The plugin intentionally infers subscriber names from the component namespace -//! so configuration remains portable across bindings. ATOF, OpenTelemetry, and -//! OpenInference each register one global subscriber when enabled. ATIF uses a -//! global dispatcher that detects direct child agent scopes and creates one -//! scope-local exporter for each top-level agent run. +//! so configuration remains portable across bindings. Agent Trajectory +//! Observability Format (ATOF), OpenTelemetry, and OpenInference each register +//! one global subscriber when enabled. Agent Trajectory Interchange Format +//! (ATIF) uses a global dispatcher that detects direct child agent scopes and +//! creates one scope-local exporter for each top-level agent run. use std::collections::HashMap; use std::future::Future; diff --git a/crates/core/src/stream.rs b/crates/core/src/stream.rs index cda5de29..d412d059 100644 --- a/crates/core/src/stream.rs +++ b/crates/core/src/stream.rs @@ -77,7 +77,8 @@ impl LlmStreamWrapper { /// forward chunks elsewhere. Returning `Err` terminates the stream. /// - `finalizer`: One-shot callback invoked when the stream finishes to /// synthesize the aggregated response payload. - /// - `data`: Retained compatibility payload; ATOF end data is the finalized response. + /// - `data`: Retained compatibility payload; Agent Trajectory + /// Observability Format (ATOF) end data is the finalized response. /// - `metadata`: Optional event metadata merged into the emitted LLM-end event. /// - `response_codec`: Optional codec used to derive annotated response /// metadata from the aggregated final payload. diff --git a/crates/ffi/README.md b/crates/ffi/README.md index 0df90c93..e2c53361 100644 --- a/crates/ffi/README.md +++ b/crates/ffi/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow-ffi +# NeMo Flow `nemo-flow-ffi` provides the C-compatible ABI for NeMo Flow. Use it when a native integration or downstream language binding needs direct access to the diff --git a/crates/ffi/nemo_flow.h b/crates/ffi/nemo_flow.h index 9e4446c3..72149c27 100644 --- a/crates/ffi/nemo_flow.h +++ b/crates/ffi/nemo_flow.h @@ -2336,7 +2336,8 @@ char *nemo_flow_event_attributes_json(const struct FfiEvent *ptr); char *nemo_flow_event_category_profile(const struct FfiEvent *ptr); /** - * Return the ATOF data schema as a JSON C string, or null if absent. + * Return the Agent Trajectory Observability Format (ATOF) data schema as a + * JSON C string, or null if absent. * * # Safety * `ptr` must be a valid `FfiEvent` pointer or null. diff --git a/crates/ffi/src/types/mod.rs b/crates/ffi/src/types/mod.rs index 8f86f472..25e8b6ca 100644 --- a/crates/ffi/src/types/mod.rs +++ b/crates/ffi/src/types/mod.rs @@ -679,7 +679,8 @@ pub unsafe extern "C" fn nemo_flow_event_category_profile(ptr: *const FfiEvent) } } -/// Return the ATOF data schema as a JSON C string, or null if absent. +/// Return the Agent Trajectory Observability Format (ATOF) data schema as a +/// JSON C string, or null if absent. /// /// # Safety /// `ptr` must be a valid `FfiEvent` pointer or null. diff --git a/crates/node/README.md b/crates/node/README.md index 1f53cb1f..ccba1cd9 100644 --- a/crates/node/README.md +++ b/crates/node/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow-node +# NeMo Flow `nemo-flow-node` is the NeMo Flow package for Node.js applications. It gives JavaScript and TypeScript code access to the same execution scopes, middleware, @@ -30,7 +31,8 @@ should install it from npm rather than depend on the Rust crate directly. - πŸ›‘οΈ **Put policy around callbacks**: Register guardrails and intercepts for request rewriting, blocking, sanitization, and execution wrapping. - πŸ“‘ **Emit one lifecycle stream**: Send runtime events to in-process - subscribers, ATIF, OpenTelemetry, or OpenInference workflows. + subscribers, Agent Trajectory Interchange Format (ATIF), OpenTelemetry, or + OpenInference workflows. - 🧩 **Use package entry points by need**: Import the main runtime surface plus typed, plugin, adaptive, and observability helpers from npm. diff --git a/crates/node/adaptive.d.ts b/crates/node/adaptive.d.ts index 102cc655..391bde77 100644 --- a/crates/node/adaptive.d.ts +++ b/crates/node/adaptive.d.ts @@ -145,10 +145,10 @@ export declare function toolParallelismConfig(config?: ToolParallelismConfig): T /** * Create adaptive cache-governor settings with defaults applied. * - * Merges caller-supplied overrides onto the ACG config shape used by the - * adaptive plugin's LLM execution intercept. + * Merges caller-supplied overrides onto the Adaptive Cache Governor (ACG) + * config shape used by the adaptive plugin's LLM execution intercept. * - * @param config - Partial ACG settings to override. + * @param config - Partial Adaptive Cache Governor (ACG) settings to override. * @returns A normalized adaptive cache-governor config object. * @remarks Nested `stability_thresholds` values are defaulted individually so * callers can override only the thresholds they need. diff --git a/crates/node/observability.d.ts b/crates/node/observability.d.ts index 811bce12..4a66c7ee 100644 --- a/crates/node/observability.d.ts +++ b/crates/node/observability.d.ts @@ -56,9 +56,9 @@ export interface ComponentSpec { export declare const OBSERVABILITY_PLUGIN_KIND: 'observability'; /** Create a default observability component config. */ export declare function defaultConfig(): Config; -/** Create filesystem-backed ATOF JSONL settings with defaults applied. */ +/** Create filesystem-backed Agent Trajectory Observability Format (ATOF) JSONL settings with defaults applied. */ export declare function atofConfig(config?: AtofConfig): AtofConfig; -/** Create per-agent ATIF trajectory settings with defaults applied. */ +/** Create per-agent Agent Trajectory Interchange Format (ATIF) trajectory settings with defaults applied. */ export declare function atifConfig(config?: AtifConfig): AtifConfig; /** Create OTLP exporter settings for OpenTelemetry or OpenInference. */ export declare function otlpConfig(config?: OtlpConfig): OtlpConfig; diff --git a/crates/node/src/api/mod.rs b/crates/node/src/api/mod.rs index 68815281..be5d9a22 100644 --- a/crates/node/src/api/mod.rs +++ b/crates/node/src/api/mod.rs @@ -2887,11 +2887,11 @@ pub fn llm_conditional_execution(env: Env, request: Json) -> Result { } // --------------------------------------------------------------------------- -// ATIF Exporter +// Agent Trajectory Interchange Format (ATIF) Exporter // --------------------------------------------------------------------------- -/// An ATIF (Agent Trajectory Interchange Format) exporter that collects lifecycle events -/// and exports them as a structured trajectory. +/// An Agent Trajectory Interchange Format (ATIF) exporter that collects lifecycle +/// events and exports them as a structured trajectory. /// /// Create an instance with session and agent metadata, then register it as an event subscriber. /// When ready, call `exportJson()` to serialize the collected trajectory. @@ -2972,7 +2972,7 @@ pub struct AtofExporterConfig { pub filename: Option, } -/// Filesystem-backed ATOF JSONL event exporter. +/// Filesystem-backed Agent Trajectory Observability Format (ATOF) JSONL event exporter. #[napi] pub struct AtofExporter { inner: nemo_flow::observability::atof::AtofExporter, @@ -2980,7 +2980,8 @@ pub struct AtofExporter { #[napi] impl AtofExporter { - /// Create a new ATOF JSONL exporter from a config object. + /// Create a new Agent Trajectory Observability Format (ATOF) JSONL exporter + /// from a config object. #[napi(constructor)] pub fn new(config: Option) -> napi::Result { let inner = nemo_flow::observability::atof::AtofExporter::new(build_atof_config(config)?) diff --git a/crates/python/README.md b/crates/python/README.md index 214a6fc6..e15b3b9b 100644 --- a/crates/python/README.md +++ b/crates/python/README.md @@ -12,6 +12,7 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) # NeMo Flow Python Bindings diff --git a/crates/wasm/README.md b/crates/wasm/README.md index 20ac0e8a..d6b53e28 100644 --- a/crates/wasm/README.md +++ b/crates/wasm/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# nemo-flow-wasm +# NeMo Flow `nemo-flow-wasm` is the NeMo Flow WebAssembly package for JavaScript environments that load the runtime through WebAssembly. It exposes the same execution @@ -25,6 +26,14 @@ The Rust crate in this directory is build machinery for the generated npm package. JavaScript users should install the npm package rather than depend on the Rust crate directly. +This surface is experimental and source-first. Use the repository source tree +and WebAssembly tests when validating behavior, and prefer Rust, Python, or +Node.js for primary documented application integrations. + +Observability support is also experimental. The WebAssembly target does not +support `grpc` OTLP transport, and file-backed observability plugin sinks +require a host runtime with filesystem access. + ## Why Use It? - 🌐 **Bring NeMo Flow to WebAssembly**: Use the shared runtime model from @@ -45,7 +54,8 @@ the Rust crate directly. - βœ… **Middleware registration**: Guardrail and intercept APIs for JavaScript callbacks. - βœ… **Additional entry points**: `nemo-flow-wasm/typed`, - `nemo-flow-wasm/plugin`, and `nemo-flow-wasm/adaptive`. + `nemo-flow-wasm/plugin`, `nemo-flow-wasm/adaptive`, and + `nemo-flow-wasm/observability`. - βœ… **Generated npm package**: A `wasm-pack` build prepared for JavaScript package consumption. @@ -57,7 +67,7 @@ Install the npm package in a JavaScript project: npm install nemo-flow-wasm ``` -For local source development from the repository root: +For local source validation from the repository root: ```bash npm run build:pkg --workspace=nemo-flow-wasm @@ -96,8 +106,8 @@ main().catch((error) => { ``` The main runtime API is exported from `nemo-flow-wasm`. Additional entry points -are available at `nemo-flow-wasm/typed`, `nemo-flow-wasm/plugin`, and -`nemo-flow-wasm/adaptive`. +are available at `nemo-flow-wasm/typed`, `nemo-flow-wasm/plugin`, +`nemo-flow-wasm/adaptive`, and `nemo-flow-wasm/observability`. ## Documentation diff --git a/docs/about/concepts/plugins.md b/docs/about/concepts/plugins.md index e383ab20..7eb0e6e0 100644 --- a/docs/about/concepts/plugins.md +++ b/docs/about/concepts/plugins.md @@ -144,16 +144,21 @@ through the same component lifecycle as other plugins: - Initialize the plugin system - Inspect the activation result if needed -Detailed adaptive configuration belongs in [Configure Adaptive Optimization](../../use-adaptive-optimization/configure.md), [Adaptive Code Examples](../../use-adaptive-optimization/code-examples.md), and [Advanced Guide: Configure Adaptive Components](../../use-adaptive-optimization/adaptive-components.md). +Detailed adaptive configuration belongs in +[Adaptive Configuration](../../plugins/adaptive/configuration.md), +[Adaptive Cache Governor (ACG)](../../plugins/adaptive/acg.md), and +[Adaptive Hints](../../plugins/adaptive/adaptive-hints.md). ### Observability -The core crate ships a built-in `observability` plugin component for ATOF, -ATIF, OpenTelemetry, and OpenInference exporters. Each exporter section is +The core crate ships a built-in `observability` plugin component for Agent +Trajectory Observability Format (ATOF), Agent Trajectory Interchange Format +(ATIF), OpenTelemetry, and OpenInference exporters. Each exporter section is disabled unless its section sets `enabled: true`, and subscriber names are inferred from the plugin namespace instead of exposed in public config. -Detailed observability plugin configuration belongs in [Configure the Observability Plugin](../../export-observability-data/observability-plugin.md). +Detailed observability plugin configuration belongs in +[Observability Configuration](../../plugins/observability/configuration.md). For the CLI gateway's `plugins.toml` discovery, precedence, merge, and editing rules, see [Plugin Configuration Files](../../build-plugins/plugin-configuration-files.md). diff --git a/docs/about/concepts/subscribers.md b/docs/about/concepts/subscribers.md index 5bb48d8b..71780d81 100644 --- a/docs/about/concepts/subscribers.md +++ b/docs/about/concepts/subscribers.md @@ -81,15 +81,17 @@ exporters. A plain custom subscriber is the right choice when you want in-process handling of the canonical event stream. -### ATIF Exporter +### Agent Trajectory Interchange Format (ATIF) Exporter -The ATIF exporter collects lifecycle events and emits trajectory artifacts for -offline analysis, replay, or debugging. +The [Agent Trajectory Interchange Format (ATIF) exporter](../../plugins/observability/atif.md) +collects lifecycle events and emits trajectory artifacts for offline analysis, +replay, or debugging. -### ATOF JSONL Exporter +### Agent Trajectory Observability Format (ATOF) JSONL Exporter -The ATOF JSONL exporter writes the canonical event stream to a native -filesystem path as one raw ATOF event per line. +The [Agent Trajectory Observability Format (ATOF) JSONL exporter](../../plugins/observability/atof.md) +writes the canonical event stream to a native filesystem path as one raw ATOF +event per line. ### OpenTelemetry Subscriber @@ -102,10 +104,9 @@ The OpenInference subscriber maps runtime events into OTLP traces using OpenInference semantics for model-centric observability. Detailed setup, configuration, and API shape for these subscribers belongs in -[Export Observability Data](../../export-observability-data/basic-guide.md) -and [Observability Code Examples](../../export-observability-data/code-examples.md). +[Observability](../../plugins/observability/about.md). For configuration-driven setup, use the built-in -[`observability` plugin](../../export-observability-data/observability-plugin.md) +[`observability` plugin](../../plugins/observability/configuration.md) to install ATOF, ATIF, OpenTelemetry, and OpenInference subscribers from one plugin component. diff --git a/docs/about/ecosystem.md b/docs/about/ecosystem.md index 08ec1df3..bbaea3f5 100644 --- a/docs/about/ecosystem.md +++ b/docs/about/ecosystem.md @@ -31,7 +31,7 @@ systems can call when actual work crosses a scope, tool, or model boundary. | NeMo Agent Toolkit and agent application frameworks | Build, run, profile, and optimize agent workflows across tools, data sources, and framework choices. | NeMo Flow can sit below these systems as the shared runtime contract for scopes, middleware, lifecycle events, subscribers, and plugins. | | NeMo Guardrails and policy systems | Define safety, control, and compliance behavior for LLM applications. | NeMo Flow can host runtime guardrails and intercepts around managed tool and LLM calls, while higher-level guardrail systems can still own policy authoring and orchestration. | | Application harnesses and workflow code | Decide the agent pattern, planner, memory, retries, scheduling, and user-facing behavior. | NeMo Flow instruments the execution boundaries that the harness already owns. | -| Observability and evaluation backends | Store traces, trajectories, metrics, and analysis data. | NeMo Flow emits lifecycle events and exports them to in-process subscribers, ATIF, OpenTelemetry, OpenInference-compatible traces, or other backends. | +| Observability and evaluation backends | Store traces, trajectories, metrics, and analysis data. | NeMo Flow emits lifecycle events and exports them to in-process subscribers, Agent Trajectory Observability Format (ATOF), Agent Trajectory Interchange Format (ATIF), OpenTelemetry, OpenInference-compatible traces, or other backends. | In practical terms, NeMo Flow answers a different question than higher-level agent products. A framework asks, "What should the agent do next?" NeMo Flow diff --git a/docs/about/release-notes/highlights.md b/docs/about/release-notes/highlights.md index 3544eca5..9defd9d9 100644 --- a/docs/about/release-notes/highlights.md +++ b/docs/about/release-notes/highlights.md @@ -7,12 +7,32 @@ SPDX-License-Identifier: Apache-2.0 This page summarizes the notable capabilities in the current release documentation set. -## Initial Release +## NeMo Flow 0.2 -The initial NeMo Flow release introduces: +This release of NeMo Flow release introduces several new components and capabilities. +The complete changelog and release notes can be viewed on [GitHub](https://github.com/NVIDIA/NeMo-Flow/releases). -- A Rust core runtime for scopes, middleware registries, lifecycle events, subscribers, plugins, tool calls, and LLM calls -- Primary Rust, Python, and Node.js documentation tracks -- ATIF, OpenTelemetry, and OpenInference subscriber/exporter surfaces -- An adaptive component for plugin-driven optimization behavior -- Sample third-party integration patches for selected agent frameworks and tools +### NeMo Flow CLI + +- Added CLI support for coding-agent harnesses, including Claude Code, Codex, Cursor, and Hermes Agent. +- Enhanced setup and configuration workflows with guided onboarding, environment diagnostics, plugin configuration loading, and plugin editing. + +### Integrations + +- Added OpenClaw observability support through a first-party `nemo-flow-openclaw` plugin. +- Added middleware and runtime integrations for LangChain, LangGraph, and Deep Agents. These are accessed through the Python `nemo-flow` package through extras. + +### Observability + +- Introduced a core Observability plugin that globally configures all built-in observability exporters and subscribers. +- Added an Agent Trajectory Observability Format (ATOF) JSONL exporter for structured event output. +- Expanded Codec support for optional annotated LLM request and response fields. + +### Security + +- Added an example NeMo Guardrails plugin. + +### Agent Skill Improvements + +- Updated Python binding test guidance for agent skills. +- Added PR template requirements to agent contribution guidance. diff --git a/docs/about/release-notes/known-issues.md b/docs/about/release-notes/known-issues.md index d8c19cf8..b3a2bcb9 100644 --- a/docs/about/release-notes/known-issues.md +++ b/docs/about/release-notes/known-issues.md @@ -7,11 +7,14 @@ SPDX-License-Identifier: Apache-2.0 This page lists current limitations and support notes for the release documentation set. -## Initial Release +## NeMo Flow 0.2 -These notes apply to the initial public NeMo Flow release. +These notes apply to the NeMo Flow 0.2 Release. - Go, WebAssembly, and the raw C FFI surface are experimental and source-first. - Generated API pages cover Rust, Python, and Node.js. Experimental bindings do not yet have the same generated documentation depth. +- The NeMo Flow CLI is experimental. Coding agent observability support varies due to capabilities of hooks. Any encountered problems should be filed as bugs. -No fixed known issues are documented for the initial release. +### Fixed issues from NeMo Flow 0.1: +- Enabled TLS support for OTLP HTTP export. +- Preserved Go scope stacks across OS threads. diff --git a/docs/about/release-notes/related-topics.md b/docs/about/release-notes/related-topics.md index 17da2929..b1316688 100644 --- a/docs/about/release-notes/related-topics.md +++ b/docs/about/release-notes/related-topics.md @@ -9,5 +9,3 @@ Use these links to continue into adjacent concepts and workflows. - [Ecosystem](../ecosystem.md) - [Support and FAQs](../../resources/support-and-faqs.md) - -No migration guide is required for the initial NeMo Flow release. Future public API, binding behavior, plugin configuration, or integration workflow changes should add migration guidance to the relevant release note and link it from this page. diff --git a/docs/build-plugins/about.md b/docs/build-plugins/about.md index 53d338bf..bd66cfa8 100644 --- a/docs/build-plugins/about.md +++ b/docs/build-plugins/about.md @@ -31,11 +31,11 @@ If the behavior applies to only one request or tenant, consider scope-local midd Use these guide links to move from the overview into task-specific instructions. -- [Basic Guide: Define a Plugin](basic-guide.md) explains plugin kinds, shape, runtime ownership, and the activation lifecycle. -- [Basic Guide: Validate Plugin Configuration](validate-configuration.md) covers JSON-compatible config, validation rules, and structured diagnostics. -- [Basic Guide: Plugin Configuration Files](plugin-configuration-files.md) documents `plugins.toml` file discovery, precedence, merge behavior, and editor controls for the CLI gateway. -- [Basic Guide: Register Plugin Behavior](register-behavior.md) shows how to initialize config and install subscribers or middleware through `PluginContext`. -- [Advanced Guide: Design Plugin Configuration](advanced-configuration.md) covers validation rules, advanced configuration patterns, rollout controls, and `PluginContext` usage. +- [Define a Plugin](basic-guide.md) explains plugin kinds, shape, runtime ownership, and the activation lifecycle. +- [Validate Plugin Configuration](validate-configuration.md) covers JSON-compatible config, validation rules, and structured diagnostics. +- [Plugin Configuration Files](plugin-configuration-files.md) documents `plugins.toml` file discovery, precedence, merge behavior, and editor controls for the CLI gateway. +- [Register Plugin Behavior](register-behavior.md) shows how to initialize config and install subscribers or middleware through `PluginContext`. +- [Design Plugin Configuration](advanced-configuration.md) covers validation rules, advanced configuration patterns, rollout controls, and `PluginContext` usage. - [NeMo Guardrails Example Plugin](nemoguardrails.md) shows an external Python plugin that applies NeMo Guardrails checks around NeMo Flow LLM and tool calls. - [Code Examples](code-examples.md) provides patterns for dynamic header injection, subscriber-oriented export, multi-surface bundles, and framework-facing plugins. diff --git a/docs/build-plugins/advanced-configuration.md b/docs/build-plugins/advanced-configuration.md index dd0f5f88..08d223f0 100644 --- a/docs/build-plugins/advanced-configuration.md +++ b/docs/build-plugins/advanced-configuration.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Design Plugin Configuration +# Design Plugin Configuration Use this guide when a plugin needs more than a single flag or string to configure it safely. @@ -172,7 +172,7 @@ Before publishing a plugin config contract: Use these links to continue from this workflow into the next related task. -- Build the first plugin with [Basic Guide: Define a Plugin](basic-guide.md). -- Validate plugin config with [Basic Guide: Validate Plugin Configuration](validate-configuration.md). -- Register runtime behavior with [Basic Guide: Register Plugin Behavior](register-behavior.md). +- Build the first plugin with [Define a Plugin](basic-guide.md). +- Validate plugin config with [Validate Plugin Configuration](validate-configuration.md). +- Register runtime behavior with [Register Plugin Behavior](register-behavior.md). - Review reusable patterns in [Code Examples](code-examples.md). diff --git a/docs/build-plugins/basic-guide.md b/docs/build-plugins/basic-guide.md index 414cd8a8..a76828d2 100644 --- a/docs/build-plugins/basic-guide.md +++ b/docs/build-plugins/basic-guide.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Define a Plugin +# Define a Plugin Use this guide when you want to package reusable NeMo Flow behavior as a plugin that can be activated from configuration. @@ -142,7 +142,7 @@ Before you write the plugin implementation, answer these questions: Use these links to continue from this workflow into the next related task. -- Define validation behavior with [Basic Guide: Validate Plugin Configuration](validate-configuration.md). -- Register runtime behavior with [Basic Guide: Register Plugin Behavior](register-behavior.md). -- Add rollout controls with [Advanced Guide: Design Plugin Configuration](advanced-configuration.md). +- Define validation behavior with [Validate Plugin Configuration](validate-configuration.md). +- Register runtime behavior with [Register Plugin Behavior](register-behavior.md). +- Add rollout controls with [Design Plugin Configuration](advanced-configuration.md). - Review complete examples in [Code Examples](code-examples.md). diff --git a/docs/build-plugins/plugin-configuration-files.md b/docs/build-plugins/plugin-configuration-files.md index a8cc0cac..18504c88 100644 --- a/docs/build-plugins/plugin-configuration-files.md +++ b/docs/build-plugins/plugin-configuration-files.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Plugin Configuration Files +# Plugin Configuration Files Use `plugins.toml` when the `nemo-flow` CLI gateway should activate plugins at startup. The file contains the same generic plugin configuration document used @@ -176,8 +176,9 @@ kind = "observability" mode = "overwrite" ``` -The effective ATOF config keeps `enabled` and `output_directory` from the system -file and uses `mode = "overwrite"` from the user file. +The effective Agent Trajectory Observability Format (ATOF) config keeps +`enabled` and `output_directory` from the system file and uses +`mode = "overwrite"` from the user file. The top-level `components` array is special. Components are matched by `kind` across files. A higher-precedence component with the same `kind` merges into the @@ -233,8 +234,8 @@ Common validation failures include: - Unsupported field values, such as an invalid exporter mode or transport. - Duplicate singleton components. - Enabled components whose build-time features are unavailable. -- Component-specific semantic failures, such as an ATIF filename template that - does not contain `{session_id}`. +- Component-specific semantic failures, such as an Agent Trajectory Interchange + Format (ATIF) filename template that does not contain `{session_id}`. Use `nemo-flow doctor` to inspect the resolved gateway configuration and plugin diagnostics. For Observability, doctor also reports enabled exporter sections and @@ -260,6 +261,7 @@ Observability exporters through `plugins.toml`. Use the component guides for field-level configuration: -- [Configure the Observability Plugin](../export-observability-data/observability-plugin.md) -- [Configure Adaptive Optimization](../use-adaptive-optimization/configure.md) -- [Advanced Guide: Configure Adaptive Components](../use-adaptive-optimization/adaptive-components.md) +- [Observability Configuration](../plugins/observability/configuration.md) +- [Adaptive Configuration](../plugins/adaptive/configuration.md) +- [Adaptive Cache Governor (ACG)](../plugins/adaptive/acg.md) +- [Adaptive Hints](../plugins/adaptive/adaptive-hints.md) diff --git a/docs/build-plugins/register-behavior.md b/docs/build-plugins/register-behavior.md index befc07f7..7c08471f 100644 --- a/docs/build-plugins/register-behavior.md +++ b/docs/build-plugins/register-behavior.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Register Plugin Behavior +# Register Plugin Behavior Use this guide when plugin config validation is in place and you need the plugin to install real NeMo Flow runtime behavior. @@ -289,5 +289,5 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Add advanced validation and rollout controls with [Advanced Guide: Design Plugin Configuration](advanced-configuration.md). +- Add advanced validation and rollout controls with [Design Plugin Configuration](advanced-configuration.md). - Review concrete authoring patterns in [Code Examples](code-examples.md). diff --git a/docs/build-plugins/validate-configuration.md b/docs/build-plugins/validate-configuration.md index 570d794a..31b148ad 100644 --- a/docs/build-plugins/validate-configuration.md +++ b/docs/build-plugins/validate-configuration.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Validate Plugin Configuration +# Validate Plugin Configuration Use this guide when you have a plugin kind and need predictable diagnostics before the plugin installs runtime behavior. @@ -201,6 +201,6 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Register runtime behavior with [Basic Guide: Register Plugin Behavior](register-behavior.md). -- Add rollout controls with [Advanced Guide: Design Plugin Configuration](advanced-configuration.md). +- Register runtime behavior with [Register Plugin Behavior](register-behavior.md). +- Add rollout controls with [Design Plugin Configuration](advanced-configuration.md). - Review concrete validation patterns in [Code Examples](code-examples.md). diff --git a/docs/contribute/development-setup.md b/docs/contribute/development-setup.md index bb1219b8..9151b029 100644 --- a/docs/contribute/development-setup.md +++ b/docs/contribute/development-setup.md @@ -11,11 +11,9 @@ changes. ## Package Installation If you are consuming NeMo Flow rather than developing this repository, install -the published package for your language: - -- Rust: `cargo add nemo-flow` -- Python: `uv add nemo-flow` or `pip install nemo-flow` -- Node.js: `npm install nemo-flow-node` +the published package for your language. Use +[Installation](../getting-started/installation.md) for package-manager commands +covering the CLI, Python, Node.js, Rust, and supported integrations. Go, WebAssembly, and the raw FFI surface are currently experimental and remain source-first. diff --git a/docs/export-observability-data/about.md b/docs/export-observability-data/about.md deleted file mode 100644 index 4822b8f4..00000000 --- a/docs/export-observability-data/about.md +++ /dev/null @@ -1,60 +0,0 @@ - - -# About - -Use this section when you need to inspect NeMo Flow lifecycle events in process -or export agent activity to tracing, trajectory, or analysis systems. - -Observability in NeMo Flow starts with events. Scopes, marks, managed tool -calls, managed LLM calls, middleware, and manual lifecycle APIs emit a canonical -event stream. Subscribers consume that stream inside the process, and -exporter-oriented subscribers write raw ATOF JSONL or translate it into formats -such as ATIF, OpenTelemetry, and OpenInference. - -For standard exporters, use the built-in `observability` plugin to configure -ATOF, per-agent ATIF, OpenTelemetry, and OpenInference from one plugin document. -Each section is disabled unless it explicitly sets `enabled: true`. - -Use these guides to confirm what ran, where it belonged, which model or tool was -involved, and what sanitized payload was observed across Rust, Python, and -Node.js. - -## Start Here When - -Start here when you need to perform one of the following checks: - -- Verify that instrumentation is attached to the right scope -- Inspect tool and LLM inputs and outputs after sanitization -- Correlate concurrent agent runs by root scope -- Export traces to existing OTLP-compatible infrastructure -- Produce trajectory data for analysis, replay, or evaluation workflows - -If you have not instrumented any scopes, tools, or LLM calls yet, start with [Instrument Applications](../instrument-applications/about.md). - -## Guides - -The following guides describe available tutorials and exporters: - -- [Basic Guide: Register a Subscriber](basic-guide.md) shows a simple subscriber lifecycle and validation workflow. -- [Basic Guide: Configure the Observability Plugin](observability-plugin.md) shows the built-in exporter plugin and its config schema. -- [Code Examples](code-examples.md#atof-jsonl-export) shows how to write raw ATOF events as JSONL. -- [Advanced Guide: Export OpenTelemetry Data](opentelemetry.md) shows how to export generic OTLP spans. -- [Advanced Guide: Export OpenInference Data](advanced-guide.md) shows how to configure and operate the OpenInference exporter. -- [Advanced Guide: Export ATIF](atif.md) shows how to collect and export trajectory artifacts. -- [Code Examples](code-examples.md) shows event shape, scope-local subscribers, ATOF JSONL export, ATIF export, OpenTelemetry export, and exporter selection snippets. - -Begin with a local subscriber so you can confirm the application emits the -expected scope, tool, LLM, and mark events. Add exporters only after the event -stream is correct and sensitive payloads are sanitized. - -For production export, register subscribers before the first instrumented -request, use stable service identity fields, keep credentials outside source -code, flush during graceful shutdown, and filter by `root_uuid` when analyzing -concurrent agent runs. - -The filesystem-backed ATOF JSONL exporter and ATIF plugin file sink require -native filesystem access. Use explicit plugin teardown or exporter shutdown to -flush files during graceful shutdown. diff --git a/docs/export-observability-data/advanced-guide.md b/docs/export-observability-data/advanced-guide.md deleted file mode 100644 index 56404d4c..00000000 --- a/docs/export-observability-data/advanced-guide.md +++ /dev/null @@ -1,181 +0,0 @@ - - -# Advanced Guide: Export OpenInference Data - -Use this guide when you want NeMo Flow lifecycle events exported as OTLP trace spans with OpenInference-oriented semantics. - -## What You Build - -You will configure the OpenInference subscriber, register it before scoped work starts, run instrumented work, flush spans, and shut down the exporter. - -The OpenInference subscriber maps NeMo Flow lifecycle payloads to trace attributes: - -- Scope, tool, and LLM start inputs become `input.value`. -- Scope, tool, and LLM end outputs become `output.value`. -- LLM usage metadata maps to OpenInference token-count attributes when the provider response includes usage information. - -## Before You Start - -Complete these steps: - -1. Instrument at least one scope, tool call, or LLM call. -2. Decide where OTLP traces should be sent. -3. Start an OTLP/HTTP collector or tracing backend. -4. Redact sensitive event payloads with sanitize guardrails before production export. - -## Configure the Exporter - -Set these fields first: - -| Field | Purpose | -|---|---| -| `endpoint` | OTLP trace endpoint, such as `http://localhost:4318/v1/traces` | -| `service_name` | Logical service name shown in traces | -| `service_namespace` | Optional namespace for grouping services | -| `service_version` | Optional application or package version | -| `headers` | Optional auth or routing headers | -| `resource_attributes` | Deployment metadata such as environment or region | -| `timeout_millis` | Export timeout in milliseconds | - -OpenInference exports semantic lifecycle payloads directly: - -- Scope, tool, and LLM start inputs become `input.value` -- Scope, tool, and LLM end outputs become `output.value` -- LLM usage metadata maps token counters when the provider response includes usage information - -`OTEL_*` variables may be used by the underlying OpenTelemetry exporter when values are not set directly in config. Prefer explicit config fields for endpoint, headers, resource attributes, and service identity in application code. - -## Register and Export - -The examples below show how to register the exporter and emit data from instrumented -work. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import OpenInferenceConfig, OpenInferenceSubscriber - -config = OpenInferenceConfig() -config.transport = "http_binary" -config.endpoint = "http://localhost:4318/v1/traces" -config.service_name = "agent-service" -config.service_namespace = "nemo" -config.service_version = "1.0.0" -config.instrumentation_scope = "nemo-flow-openinference" -config.timeout_millis = 3000 -config.headers = {"authorization": "Bearer token"} -config.resource_attributes = {"deployment.environment": "dev"} - -subscriber = OpenInferenceSubscriber(config) -subscriber.register("openinference-exporter") - -# Run instrumented application work here. - -subscriber.force_flush() -subscriber.deregister("openinference-exporter") -subscriber.shutdown() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```js -const { OpenInferenceSubscriber } = require("nemo-flow-node"); - -const subscriber = new OpenInferenceSubscriber({ - transport: "http_binary", - endpoint: "http://localhost:4318/v1/traces", - serviceName: "agent-service", - serviceNamespace: "nemo", - serviceVersion: "1.0.0", - instrumentationScope: "nemo-flow-openinference", - timeoutMillis: 3000, - headers: { authorization: "Bearer token" }, - resourceAttributes: { "deployment.environment": "dev" }, -}); - -subscriber.register("openinference-exporter"); - -// Run instrumented application work here. - -subscriber.forceFlush(); -subscriber.deregister("openinference-exporter"); -subscriber.shutdown(); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::openinference::{OpenInferenceConfig, OpenInferenceSubscriber}; - -let subscriber = OpenInferenceSubscriber::new( - OpenInferenceConfig::new() - .with_service_name("agent-service") - .with_endpoint("http://localhost:4318/v1/traces") - .with_header("authorization", "Bearer token") - .with_resource_attribute("deployment.environment", "dev") - .with_service_namespace("nemo") - .with_service_version("1.0.0") - .with_instrumentation_scope("nemo-flow-openinference"), -)?; - -subscriber.register("openinference-exporter")?; - -// Run instrumented application work here. - -subscriber.force_flush()?; -subscriber.deregister("openinference-exporter")?; -subscriber.shutdown()?; -``` -::: - -:::: - -## Validate the Export - -Check the export in three places: - -1. Application logs should not show exporter construction or transport errors. -2. The collector should receive OTLP/HTTP trace export requests. -3. The tracing backend should show spans for scopes, tools, and LLM calls from the same `root_uuid`. - -If spans arrive without useful payloads, check that tool and LLM calls pass JSON-compatible `input` and `output` values. If payloads contain sensitive fields, add sanitize guardrails before exporting. - -## Production Checklist - -Use this checklist before running the pattern in production traffic. - -- Register the exporter before the first instrumented request. -- Flush during graceful shutdown. -- Use stable service identity fields across deployments. -- Keep headers and endpoint configuration outside source code. -- Filter or redact sensitive payloads before export. -- Use `root_uuid` to isolate concurrent agent runs in trace analysis. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **No spans appear**: Confirm that the OTLP endpoint is reachable and that application work emits events. -- **Exporter fails on startup**: Confirm the transport is `http_binary`. -- **Only scope spans appear**: Route tools and LLM calls through the managed execute helpers. -- **Sensitive data appears in the backend**: Add sanitize guardrails for event payloads. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Add custom redaction with [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md). -- Compare generic OTLP export with [Advanced Guide: Export OpenTelemetry Data](opentelemetry.md). -- Export trajectory artifacts with [Advanced Guide: Export ATIF](atif.md). -- Review [Code Examples](code-examples.md) for event shape, ATIF, OpenTelemetry, and exporter selection snippets. diff --git a/docs/export-observability-data/atif.md b/docs/export-observability-data/atif.md deleted file mode 100644 index 1dd640bc..00000000 --- a/docs/export-observability-data/atif.md +++ /dev/null @@ -1,157 +0,0 @@ - - -# Advanced Guide: Export ATIF - -Use this guide when you want to collect NeMo Flow lifecycle events and export them as an Agent Trajectory Interchange Format (ATIF) trajectory for offline analysis, replay, or evaluation workflows. - -## What You Build - -You will create an ATIF exporter, register it as a subscriber, run instrumented work, export the collected trajectory, and clear or deregister the exporter when the collection window ends. - -Unlike OpenTelemetry and OpenInference export, ATIF export is in-process and buffered. The exporter collects events until you call `export`, `export_json`, or `clear`. - -For automatic per-agent file export, use the built-in -[Observability plugin](observability-plugin.md). Its `atif` section creates one -scope-local exporter for each top-level agent scope and writes each trajectory -when that agent scope ends. - -## Before You Start - -Complete these steps: - -1. Instrument scope, tool, or LLM work so the runtime emits events. -2. Choose a stable `session_id` for the trajectory. -3. Set agent metadata such as name, version, and optional model name. -4. Decide when the collection window starts and ends. -5. Sanitize sensitive event payloads before exporting trajectories outside the process. - -## How Events Map to ATIF - -The exporter translates NeMo Flow events into ATIF v1.6 trajectory data: - -| NeMo Flow event | ATIF output | -|---|---| -| LLM start and end | Agent steps and model metadata. | -| Tool start and end | Tool calls and observations. | -| Scope nesting | Parent-child lineage in trajectory metadata. | -| Event payloads | Step input, output, tool call, or observation content. | - -The exporter preserves the collected event order and uses lifecycle pairing to reconstruct the trajectory. Use `root_uuid` or separate collection windows when concurrent agent runs should produce separate trajectories. - -## Register and Export - -The examples below show how to register the exporter and emit data from instrumented -work. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import AtifExporter - -exporter = AtifExporter("session-1", "agent", "1.0.0", model_name="demo-model") -exporter.register("atif-exporter") - -# Run instrumented application work here. - -trajectory = exporter.export() -trajectory_json = exporter.export_json() -exporter.deregister("atif-exporter") -exporter.clear() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```js -const { AtifExporter } = require("nemo-flow-node"); - -const exporter = new AtifExporter("session-1", "agent", "1.0.0", "demo-model"); -exporter.register("atif-exporter"); - -// Run instrumented application work here. - -const trajectory = JSON.parse(exporter.exportJson()); -exporter.deregister("atif-exporter"); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::api::subscriber::{deregister_subscriber, register_subscriber}; -use nemo_flow::observability::atif::{AtifAgentInfo, AtifExporter}; - -let exporter = AtifExporter::new( - "session-1".into(), - AtifAgentInfo { - name: "agent".into(), - version: "1.0.0".into(), - model_name: Some("demo-model".into()), - tool_definitions: None, - extra: None, - }, -); - -let subscriber = exporter.subscriber(); -register_subscriber("atif-exporter", subscriber.clone())?; - -// Run instrumented application work here. - -let trajectory = exporter.export(); -let removed = deregister_subscriber("atif-exporter")?; -exporter.clear(); -``` -::: - -:::: - -## Validate the Trajectory - -Inspect the exported trajectory before using it in evaluation workflows: - -1. Confirm `schema_version` is `ATIF-v1.6`. -2. Confirm agent metadata matches the intended workflow. -3. Confirm the expected LLM and tool steps are present. -4. Confirm tool observations appear after their tool calls. -5. Confirm sensitive fields are absent or sanitized. - -If a trajectory is missing tool or LLM steps, verify that those calls use managed execution helpers or explicit lifecycle APIs. - -## Collection Boundaries - -Choose one of these patterns: - -| Pattern | Use When | -|---|---| -| One exporter per run | Each agent run should produce one trajectory. | -| Long-lived exporter with `clear` | A test or local tool exports multiple trajectories in one process. | -| Filtered analysis by root scope | Concurrent runs share one process but can be separated later. | -| Observability plugin ATIF section | Each direct child agent scope should write its own trajectory file automatically. | - -For production services, prefer bounded collection windows. Long-lived unbounded exporters can accumulate more event data than expected. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **Trajectory has only scope events**: Route tool and LLM calls through managed execute helpers. -- **Multiple runs appear in one trajectory**: Use one exporter per run or clear the exporter between runs. -- **Payloads are too large**: Sanitize or summarize event payloads before export. -- **Model name is missing**: Pass model metadata to managed LLM calls or set exporter agent metadata when constructing the exporter. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Review event fields in [Code Examples](code-examples.md#event-shape). -- Export trace spans with [Advanced Guide: Export OpenTelemetry Data](opentelemetry.md). -- Add redaction with [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md). diff --git a/docs/export-observability-data/basic-guide.md b/docs/export-observability-data/basic-guide.md deleted file mode 100644 index 79da45ed..00000000 --- a/docs/export-observability-data/basic-guide.md +++ /dev/null @@ -1,170 +0,0 @@ - - -# Basic Guide: Register a Subscriber - -Use this guide when you want to observe NeMo Flow lifecycle events inside the same process. Subscribers are the simplest way to verify instrumentation, collect debug logs, update counters, or forward events to another exporter. - -## What You Build - -You will register a subscriber, run scoped work, inspect emitted events, and deregister the subscriber when it is no longer needed. - -Subscribers receive events from instrumented scopes, tool calls, LLM calls, middleware, and mark events. They do not change runtime behavior. Use middleware when you need to transform or block execution. - -## Before You Start - -Instrument at least one scope, tool call, or LLM call. A subscriber only sees events that the runtime emits. - -Good starting points: - -- [Basic Guide: Adding Scopes and Marks](../instrument-applications/adding-scopes-and-marks.md) -- [Basic Guide: Instrument a Tool Call](../instrument-applications/instrument-tool-call.md) -- [Basic Guide: Instrument an LLM Call](../instrument-applications/instrument-llm-call.md) -- [Python Quick Start](../getting-started/python/index.md) -- [Node.js Quick Start](../getting-started/nodejs.md) -- [Rust Quick Start](../getting-started/rust.md) - -## Subscriber Lifecycle - -Follow this lifecycle to register, use, and clean up a subscriber safely. - -1. Define a callback that accepts one event. -2. Register the callback under a stable subscriber name. -3. Run scoped application work. -4. Inspect or export the event fields you need. -5. Deregister the subscriber during teardown or test cleanup. - -Use global subscribers for process-level export. Use scope-local subscribers when only one request or tenant should be observed. - -## Register a Global Subscriber - -The examples below register one process-wide subscriber and remove it during cleanup. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -import nemo_flow - - -def log_event(event) -> None: - print({ - "kind": event.kind, - "name": event.name, - "root_uuid": getattr(event, "root_uuid", None), - }) - - -nemo_flow.subscribers.register("debug-logger", log_event) - -# Run instrumented application work here. - -removed = nemo_flow.subscribers.deregister("debug-logger") -``` -::: - -:::{tab-item} Node.js -:sync: node - -```js -const { deregisterSubscriber, registerSubscriber } = require("nemo-flow-node"); - -registerSubscriber("debug-logger", (event) => { - console.log({ - kind: event.kind, - name: event.name, - root_uuid: event.root_uuid, - }); -}); - -// Run instrumented application work here. - -const removed = deregisterSubscriber("debug-logger"); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::api::subscriber::{deregister_subscriber, register_subscriber}; -use std::sync::Arc; - -register_subscriber( - "debug-logger", - Arc::new(|event| { - println!("{} {}", event.kind(), event.name()); - }), -)?; - -// Run instrumented application work here. - -let removed = deregister_subscriber("debug-logger")?; -``` -::: - -:::: - -## Decide What to Capture - -Start with a small event record: - -- `kind`: The lifecycle event type. -- `name`: The scope, tool, LLM provider, subscriber, or mark name. -- `root_uuid`: The root scope identifier for isolating concurrent agents. -- `input`: The post-guardrail input payload for start events. -- `output`: The post-guardrail output payload for end events. -- `model_name`: The model name for LLM events when provided. -- `tool_call_id`: The tool call identifier for tool events when provided. - -Do not serialize full payloads by default in production. Use sanitize guardrails to redact sensitive fields before subscribers or exporters receive events. - -## Subscriber Options - -The table below compares subscriber and exporter options for common observability needs. - -| Subscriber / Exporter | Purpose | -|---|---| -| Custom subscriber | Consume events in process. | -| Scope-local subscriber | Observe one request or tenant and clean up when its scope closes. | -| ATOF JSONL exporter | Write raw ATOF events as one JSON object per line. | -| ATIF exporter | Collect events and export ATIF v1.6 trajectories. | -| OpenTelemetry subscriber | Export lifecycle events as OTLP spans. | -| OpenInference subscriber | Export lifecycle events as OTLP spans with OpenInference-oriented semantics. | -| Observability plugin | Configure ATOF, per-agent ATIF, OpenTelemetry, and OpenInference from one built-in plugin component. | - -## Validate the Subscriber - -Run one known instrumented path and check for the expected lifecycle sequence: - -1. A scope start event for the active agent, request, or workflow. -2. Tool or LLM start events for managed calls. -3. Tool or LLM end events after callbacks return. -4. A scope end event when the scope closes. - -If the subscriber receives no events, verify that the work uses `tools.execute`, `toolCallExecute`, `tool_call_execute`, `llm.execute`, `llmCallExecute`, or `llm_call_execute`. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **Duplicate subscriber name**: Subscriber names are registry keys. Reusing a name replaces or conflicts with the existing registration depending on the binding behavior. -- **Events from unrelated requests appear**: Filter by `root_uuid` or use scope-local subscribers. -- **Sensitive payloads appear in logs**: Add sanitize guardrails before registering production exporters. -- **Tests leak subscribers**: Deregister test subscribers in teardown or a `finally` block. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Export generic OTLP spans with [Advanced Guide: Export OpenTelemetry Data](opentelemetry.md). -- Export traces with [Advanced Guide: Export OpenInference Data](advanced-guide.md). -- Export trajectory artifacts with [Advanced Guide: Export ATIF](atif.md). -- Configure standard exporters with [Basic Guide: Configure the Observability Plugin](observability-plugin.md). -- Use [Code Examples](code-examples.md) for event shape, scope-local subscribers, ATIF, and OpenTelemetry snippets. -- Add redaction with [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md). diff --git a/docs/export-observability-data/code-examples.md b/docs/export-observability-data/code-examples.md deleted file mode 100644 index 99872ba3..00000000 --- a/docs/export-observability-data/code-examples.md +++ /dev/null @@ -1,533 +0,0 @@ - - -# Code Examples - -Use these examples when you need to inspect or export the lifecycle event stream. - -## Event Shape - -NeMo Flow emits Agent Trajectory Observability Format (ATOF) `0.1` events. The -wire format has two event kinds: - -- `scope`: start or end lifecycle events for agent, function, tool, LLM, retrieval, embedding, reranking, guardrail, evaluator, custom, or unknown work -- `mark`: point-in-time checkpoints that do not represent a full lifecycle span - -Every event includes a shared envelope: - -- `kind` -- `atof_version` -- `parent_uuid` -- `uuid` -- `timestamp` -- `name` -- `data` -- `data_schema` -- `metadata` - -Scope events add: - -- `scope_category`: `start` or `end` -- `category`: semantic work category, such as `agent`, `function`, `tool`, or `llm` -- `attributes`: behavioral flags, such as `remote`, `stateful`, `streaming`, `parallel`, or `relocatable` -- `category_profile`: fields such as `model_name`, `tool_call_id`, or `subtype` - -Start and end events for the same lifecycle use the same `uuid`. The `data` field is the semantic input on start events and the semantic output on end events. - -## Inspect Events - -The examples below show how to inspect emitted event payloads from each binding. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import Event - - -def _profile_value(profile, field: str): - if isinstance(profile, dict): - return profile.get(field) - return getattr(profile, field, None) - - -def on_event(event: Event) -> None: - print(event.kind, event.name, getattr(event, "uuid", None)) - - if event.kind == "scope": - print(event.scope_category, event.category, event.data) - print(_profile_value(event.category_profile, "model_name")) - print(_profile_value(event.category_profile, "tool_call_id")) - elif event.kind == "mark": - print(event.data, event.metadata) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import { registerSubscriber } from 'nemo-flow-node'; - -registerSubscriber('logger', (event) => { - console.log(event.kind, event.name, event.uuid); - - if (event.kind === 'scope') { - console.log(event.scope_category, event.category, event.data); - console.log(event.category_profile?.model_name, event.category_profile?.tool_call_id); - } else if (event.kind === 'mark') { - console.log(event.data, event.metadata); - } -}); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::api::event::{Event, ScopeCategory}; - -match event { - Event::Scope(e) => { - let input = (e.scope_category == ScopeCategory::Start).then_some(&e.base.data); - let output = (e.scope_category == ScopeCategory::End).then_some(&e.base.data); - let _ = (&e.base.uuid, &e.category, &e.attributes, input, output); - } - Event::Mark(e) => { - let _ = (&e.base.uuid, &e.base.data, &e.category, &e.category_profile); - } -} -``` -::: - -:::: - -## Scope-Local Subscriber - -Use scope-local subscribers when observation should be owned by one request and removed when that scope closes. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -import nemo_flow - -scope = nemo_flow.scope.push("request", nemo_flow.ScopeType.Agent) -nemo_flow.scope_local.register_subscriber( - scope, - "scoped-logger", - lambda event: print(event.kind, event.name), -) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import { ScopeType, pushScope, scopeRegisterSubscriber } from 'nemo-flow-node'; - -const scope = pushScope('request', ScopeType.Agent, null, null, null, null, null); -scopeRegisterSubscriber(scope.uuid, 'scoped-logger', (event) => { - console.log(event.kind, event.name); -}); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::api::scope::{push_scope, PushScopeParams, ScopeAttributes, ScopeType}; -use nemo_flow::api::subscriber::scope_register_subscriber; -use std::sync::Arc; - -let scope = push_scope( - PushScopeParams::builder() - .name("request") - .scope_type(ScopeType::Agent) - .attributes(ScopeAttributes::empty()) - .build(), -)?; - -scope_register_subscriber(&scope.uuid, "scoped-logger", Arc::new(|event| { - let _ = (event.kind(), event.name()); -}))?; -``` -::: - -:::: - -## Observability Plugin Configuration - -Use the built-in plugin when one config document should own standard exporter -setup and teardown. Each exporter section stays disabled unless it explicitly -sets `enabled: true`. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import plugin -from nemo_flow.observability import ( - AtifConfig, - AtofConfig, - ComponentSpec, - ObservabilityConfig, -) - -await plugin.initialize( - plugin.PluginConfig( - components=[ - ComponentSpec( - ObservabilityConfig( - atof=AtofConfig( - enabled=True, - output_directory="logs", - filename="events.jsonl", - mode="overwrite", - ), - atif=AtifConfig( - enabled=True, - output_directory="logs", - filename_template="trajectory-{session_id}.json", - ), - ) - ) - ] - ) -) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import * as plugin from 'nemo-flow-node/plugin'; -import * as observability from 'nemo-flow-node/observability'; - -await plugin.initialize({ - version: 1, - components: [ - observability.ComponentSpec({ - version: 1, - atof: observability.atofConfig({ - enabled: true, - output_directory: 'logs', - filename: 'events.jsonl', - mode: 'overwrite', - }), - atif: observability.atifConfig({ - enabled: true, - output_directory: 'logs', - filename_template: 'trajectory-{session_id}.json', - }), - }), - ], -}); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::plugin_component::{ - AtifSectionConfig, AtofSectionConfig, ComponentSpec, ObservabilityConfig, -}; -use nemo_flow::plugin::{initialize_plugins, PluginConfig}; - -let component = ComponentSpec::new(ObservabilityConfig { - atof: Some(AtofSectionConfig { - enabled: true, - output_directory: Some("logs".into()), - filename: Some("events.jsonl".into()), - mode: "overwrite".into(), - }), - atif: Some(AtifSectionConfig { - enabled: true, - output_directory: Some("logs".into()), - filename_template: "trajectory-{session_id}.json".into(), - ..AtifSectionConfig::default() - }), - ..ObservabilityConfig::default() -}); - -initialize_plugins(PluginConfig { - version: 1, - components: vec![component.into()], - policy: Default::default(), -}) -.await?; -``` -::: - -:::: - -## ATOF JSONL Export - -Use the ATOF JSONL exporter when you want the raw canonical event stream on -disk. The exporter writes one ATOF event JSON object per line, opens files in -append mode by default, and flushes after every event. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import AtofExporter, AtofExporterConfig, AtofExporterMode - -config = AtofExporterConfig() -config.output_directory = "logs" -config.mode = AtofExporterMode.Append -config.filename = "nemo-flow-events.jsonl" - -exporter = AtofExporter(config) -exporter.register("atof-jsonl") - -# Run instrumented application work here. - -exporter.deregister("atof-jsonl") -exporter.shutdown() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import { AtofExporter } from 'nemo-flow-node'; - -const exporter = new AtofExporter({ - outputDirectory: 'logs', - mode: 'append', - filename: 'nemo-flow-events.jsonl', -}); - -exporter.register('atof-jsonl'); - -// Run instrumented application work here. - -exporter.deregister('atof-jsonl'); -exporter.shutdown(); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::atof::{ - AtofExporter, AtofExporterConfig, AtofExporterMode, -}; - -let exporter = AtofExporter::new( - AtofExporterConfig::new() - .with_output_directory("logs") - .with_mode(AtofExporterMode::Append) - .with_filename("nemo-flow-events.jsonl"), -)?; - -exporter.register("atof-jsonl")?; - -// Run instrumented application work here. - -exporter.deregister("atof-jsonl")?; -exporter.shutdown()?; -``` -::: - -:::: - -## ATIF Export - -The ATIF exporter collects lifecycle events and exports an ATIF trajectory for offline analysis, replay, or debugging. - -For operational guidance, see [Advanced Guide: Export ATIF](atif.md). - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import AtifExporter - -exporter = AtifExporter("session-1", "agent", "1.0.0", model_name="demo-model") -exporter.register("atif-exporter") - -# Run instrumented application work here. - -trajectory = exporter.export() -trajectory_json = exporter.export_json() -exporter.deregister("atif-exporter") -exporter.clear() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import { AtifExporter } from 'nemo-flow-node'; - -const exporter = new AtifExporter('session-1', 'agent', '1.0.0', 'demo-model'); -exporter.register('atif-exporter'); - -// Run instrumented application work here. - -const trajectory = JSON.parse(exporter.exportJson()); -exporter.deregister('atif-exporter'); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::api::subscriber::{deregister_subscriber, register_subscriber}; -use nemo_flow::observability::atif::{AtifAgentInfo, AtifExporter}; - -let exporter = AtifExporter::new( - "session-1".into(), - AtifAgentInfo { - name: "agent".into(), - version: "1.0.0".into(), - model_name: Some("demo-model".into()), - tool_definitions: None, - extra: None, - }, -); - -let subscriber = exporter.subscriber(); -register_subscriber("atif-exporter", subscriber.clone())?; - -// Run instrumented application work here. - -let trajectory = exporter.export(); -let removed = deregister_subscriber("atif-exporter")?; -exporter.clear(); -``` -::: - -:::: - -## OpenTelemetry Export - -Use the OpenTelemetry subscriber when you want generic OTLP spans from NeMo Flow lifecycle events. - -For setup and validation guidance, see [Advanced Guide: Export OpenTelemetry Data](opentelemetry.md). - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import OpenTelemetryConfig, OpenTelemetrySubscriber - -config = OpenTelemetryConfig() -config.transport = "http_binary" -config.endpoint = "http://localhost:4318/v1/traces" -config.service_name = "agent" -config.service_namespace = "nemo" -config.service_version = "1.0.0" -config.instrumentation_scope = "nemo-flow-otel" -config.timeout_millis = 3000 -config.headers = {"authorization": "Bearer token"} -config.resource_attributes = {"deployment.environment": "dev"} - -subscriber = OpenTelemetrySubscriber(config) -subscriber.register("otel-exporter") -subscriber.force_flush() -subscriber.deregister("otel-exporter") -subscriber.shutdown() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import { OpenTelemetrySubscriber } from 'nemo-flow-node'; - -const subscriber = new OpenTelemetrySubscriber({ - transport: 'http_binary', - endpoint: 'http://localhost:4318/v1/traces', - serviceName: 'agent', - serviceNamespace: 'nemo', - serviceVersion: '1.0.0', - instrumentationScope: 'nemo-flow-otel', - timeoutMillis: 3000, - headers: { authorization: 'Bearer token' }, - resourceAttributes: { 'deployment.environment': 'dev' }, -}); - -subscriber.register('otel-exporter'); -subscriber.forceFlush(); -subscriber.deregister('otel-exporter'); -subscriber.shutdown(); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::otel::{OpenTelemetryConfig, OpenTelemetrySubscriber}; - -let subscriber = OpenTelemetrySubscriber::new( - OpenTelemetryConfig::http_binary("agent") - .with_endpoint("http://localhost:4318/v1/traces") - .with_header("authorization", "Bearer token") - .with_resource_attribute("deployment.environment", "dev") - .with_service_namespace("nemo") - .with_service_version("1.0.0") - .with_instrumentation_scope("nemo-flow-otel"), -)?; - -subscriber.register("otel-exporter")?; -subscriber.force_flush()?; -subscriber.deregister("otel-exporter")?; -subscriber.shutdown()?; -``` -::: - -:::: - -## Exporter Selection - -The table below summarizes which exporter or subscriber to start with for each goal. - -| Subscriber / Exporter | Purpose | -|---|---| -| Custom subscriber | Consume events in process. | -| ATOF JSONL exporter | Write raw ATOF events as one JSON object per line. | -| ATIF exporter | Collect events and export ATIF v1.6 trajectories. | -| OpenTelemetry subscriber | Export lifecycle events as OTLP spans. | -| OpenInference subscriber | Export lifecycle events as OTLP spans with OpenInference-oriented semantics. | - -OpenInference maps lifecycle payloads directly: - -- Start inputs become `input.value` -- End outputs become `output.value` -- LLM usage metadata maps to OpenInference token-count attributes when the response includes usage - -`OTEL_*` variables may be used by the underlying OpenTelemetry exporter when endpoint settings are not passed directly in config, but prefer explicit config objects for application code. diff --git a/docs/export-observability-data/observability-plugin.md b/docs/export-observability-data/observability-plugin.md deleted file mode 100644 index dc0d2351..00000000 --- a/docs/export-observability-data/observability-plugin.md +++ /dev/null @@ -1,349 +0,0 @@ - - -# Basic Guide: Configure the Observability Plugin - -Use the built-in Observability plugin when an application should install -standard exporters from one plugin configuration document instead of manually -registering each subscriber. - -The plugin kind is `observability`. It is registered by the core runtime, so -applications do not need to register a plugin implementation before validation -or initialization. - -:::{note} -Observability plugin configuration uses the generic NeMo Flow plugin document -shape, so field names are `snake_case` in every binding. This differs from -Node.js runtime classes such as `OpenTelemetrySubscriber`, which use -Node-native `camelCase` option names outside the plugin system. -::: - -## What It Installs - -The component accepts four optional sections: - -| Section | Runtime behavior | -|---|---| -| `atof` | Registers a global ATOF JSONL exporter for raw lifecycle events. | -| `atif` | Registers one ATIF dispatcher that writes one trajectory file for each top-level agent scope. | -| `opentelemetry` | Registers a global OpenTelemetry OTLP subscriber. | -| `openinference` | Registers a global OpenInference OTLP subscriber. | - -Every section defaults to disabled. A section is active only when it includes -`enabled: true`. - -## Top-Level Shape - -The generic plugin config wraps the observability component: - -```json -{ - "version": 1, - "components": [ - { - "kind": "observability", - "enabled": true, - "config": { - "version": 1, - "atof": { "enabled": true, "filename": "events.jsonl" } - } - } - ] -} -``` - -`subscriber_name` is not part of this config. The runtime infers subscriber -names from the plugin namespace: - -- ATOF: `atof` -- ATIF dispatcher: `atif` -- Per-agent ATIF scope subscriber: `atif-{agent_scope_uuid}` -- OpenTelemetry: `opentelemetry` -- OpenInference: `openinference` - -The active runtime names include the component namespace prefix used by the -plugin system. - -## CLI Gateway `plugins.toml` - -The `nemo-flow` CLI gateway can activate one process-level plugin config at -startup from `plugins.toml`. Use the interactive editor for the Observability -component: - -```bash -nemo-flow plugins edit -nemo-flow plugins edit --project -``` - -See [Plugin Configuration Files](../build-plugins/plugin-configuration-files.md) -for discovery locations, precedence, merge behavior, editor controls, conflicts -with `[plugins].config` or `--plugin-config`, and validation behavior. - -`plugins.toml` uses the generic plugin config shape at the file root. The -example below shows every observability section; include only the sections you -want to configure. Missing sections behave like disabled sections when no -lower-precedence `plugins.toml` supplies that section. In a layered -`plugins.toml` setup, omission inherits lower-precedence values; write -`enabled = false` to disable an inherited section. - -`version = 1` is recommended for clarity but not required. The root plugin -config version and observability component config version both default to `1` -when omitted; unsupported non-`1` versions fail validation by default. - -```toml -version = 1 - -[[components]] -kind = "observability" -enabled = true - -[components.config] -version = 1 - -[components.config.atof] -enabled = true -output_directory = "logs" -filename = "events.jsonl" -mode = "overwrite" - -[components.config.atif] -enabled = true -output_directory = "logs" -filename_template = "trajectory-{session_id}.json" - -[components.config.opentelemetry] -enabled = true -transport = "http_binary" -endpoint = "http://localhost:4318/v1/traces" -service_name = "nemo-flow" -service_namespace = "agent" -service_version = "0.2.0" -instrumentation_scope = "nemo-flow-observability" -timeout_millis = 3000 - -[components.config.opentelemetry.headers] -authorization = "Bearer " - -[components.config.opentelemetry.resource_attributes] -"deployment.environment" = "dev" -"service.instance.id" = "local" - -[components.config.openinference] -enabled = true -transport = "http_binary" -endpoint = "http://localhost:6006/v1/traces" -service_name = "nemo-flow" -service_namespace = "agent" -service_version = "0.2.0" -instrumentation_scope = "nemo-flow-openinference" -timeout_millis = 3000 - -[components.config.openinference.headers] -authorization = "Bearer " - -[components.config.openinference.resource_attributes] -"deployment.environment" = "dev" -"service.instance.id" = "local" - -[components.config.policy] -unknown_component = "warn" -unknown_field = "warn" -unsupported_value = "error" -``` - -The file format is generic. Other plugin kinds can use the same `components` -array when their plugin implementation is registered in the gateway process. - -## ATOF Section - -Use ATOF when you want the raw ATOF `0.1` event stream as JSONL. - -| Field | Default | Notes | -|---|---|---| -| `enabled` | `false` | Must be `true` to write events. | -| `output_directory` | Current working directory | Directory containing the JSONL file. | -| `filename` | Timestamped `nemo-flow-events-*.jsonl` | Explicit output filename. | -| `mode` | `append` | `append` or `overwrite`. | - -## ATIF Section - -Use ATIF when you want one trajectory artifact per top-level agent run. - -| Field | Default | Notes | -|---|---|---| -| `enabled` | `false` | Must be `true` to write trajectories. | -| `agent_name` | `NeMo Flow` | Agent metadata written into the trajectory. | -| `agent_version` | NeMo Flow crate version | Agent version metadata. | -| `model_name` | `unknown` | Default model metadata when no call-level model is present. | -| `tool_definitions` | Omitted | Optional ATIF tool metadata. | -| `extra` | Omitted | Optional ATIF agent metadata. | -| `output_directory` | Current working directory | Directory containing trajectory files. | -| `filename_template` | `nemo-flow-atif-{session_id}.json` | Must contain `{session_id}`. | - -A top-level agent is a scope start event with category `agent` whose parent is -the implicit root scope. The ATIF plugin creates a separate exporter for each -direct child agent scope, records that start event, attaches a scope-local -subscriber to the agent scope, and writes the file when the agent scope ends. -If the plugin is cleared while an agent is still open, teardown flushes the -partial trajectory. - -Nested agent scopes under a top-level agent remain in the parent trajectory. -Direct child scopes that are not `agent` scopes do not create ATIF files. - -## OpenTelemetry and OpenInference Sections - -OpenTelemetry and OpenInference use the same section shape: - -| Field | Default | Notes | -|---|---|---| -| `enabled` | `false` | Must be `true` to construct and register the subscriber. | -| `transport` | `http_binary` | `http_binary` or `grpc`. | -| `endpoint` | Exporter default | OTLP endpoint. | -| `headers` | `{}` | String-to-string exporter headers. | -| `resource_attributes` | `{}` | String-to-string OTLP resource attributes. | -| `service_name` | `nemo-flow` | `service.name` resource attribute. | -| `service_namespace` | Omitted | Optional `service.namespace`. | -| `service_version` | Omitted | Optional `service.version`. | -| `instrumentation_scope` | Omitted | Optional instrumentation scope name. | -| `timeout_millis` | `3000` | Export timeout. | - -Disabled OTLP sections do not construct exporters and do not contact endpoints. - -## Configure - -:::::{tab-set} -:sync-group: language - -::::{tab-item} Python -:sync: python - -```python -from nemo_flow import plugin, scope, ScopeType -from nemo_flow.observability import ( - AtifConfig, - AtofConfig, - ComponentSpec, - ObservabilityConfig, -) - -config = plugin.PluginConfig( - components=[ - ComponentSpec( - ObservabilityConfig( - atof=AtofConfig( - enabled=True, - output_directory="logs", - filename="events.jsonl", - mode="overwrite", - ), - atif=AtifConfig( - enabled=True, - output_directory="logs", - filename_template="trajectory-{session_id}.json", - ), - ) - ) - ] -) - -report = plugin.validate(config) -if report["diagnostics"]: - raise RuntimeError(report["diagnostics"]) - -await plugin.initialize(config) -try: - with scope.scope("agent", ScopeType.Agent): - pass -finally: - plugin.clear() -``` - -:::: - -::::{tab-item} Node.js -:sync: node - -```js -const plugin = require('nemo-flow-node/plugin'); -const observability = require('nemo-flow-node/observability'); - -await plugin.initialize({ - version: 1, - components: [ - observability.ComponentSpec({ - version: 1, - atof: observability.atofConfig({ - enabled: true, - output_directory: 'logs', - filename: 'events.jsonl', - mode: 'overwrite', - }), - atif: observability.atifConfig({ - enabled: true, - output_directory: 'logs', - filename_template: 'trajectory-{session_id}.json', - }), - }), - ], -}); - -try { - // Run instrumented application work here. -} finally { - plugin.clear(); -} -``` - -:::: - -::::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::plugin_component::{ - AtifSectionConfig, AtofSectionConfig, ComponentSpec, ObservabilityConfig, -}; -use nemo_flow::plugin::{PluginConfig, initialize_plugins}; - -let component = ComponentSpec::new(ObservabilityConfig { - atof: Some(AtofSectionConfig { - enabled: true, - output_directory: Some("logs".into()), - filename: Some("events.jsonl".into()), - mode: "overwrite".into(), - }), - atif: Some(AtifSectionConfig { - enabled: true, - output_directory: Some("logs".into()), - filename_template: "trajectory-{session_id}.json".into(), - ..AtifSectionConfig::default() - }), - ..ObservabilityConfig::default() -}); - -let config = PluginConfig { - version: 1, - components: vec![component.into()], - policy: Default::default(), -}; - -let report = initialize_plugins(config).await?; -assert!(!report.has_errors()); -``` - -:::: - -::::: - -## Validation and Teardown - -Validate plugin configuration before activating it. The plugin reports -unsupported transports, unsupported ATOF modes, unsafe ATIF filename templates, -unknown fields according to policy, and enabled exporters that are unavailable -in the current build or target. - -Call `plugin.clear()` or `clear_plugin_configuration()` during teardown. Clearing -the plugin config deregisters inferred subscribers, flushes file exporters, and -shuts down owned OTLP subscribers. diff --git a/docs/export-observability-data/opentelemetry.md b/docs/export-observability-data/opentelemetry.md deleted file mode 100644 index 8f9c36ec..00000000 --- a/docs/export-observability-data/opentelemetry.md +++ /dev/null @@ -1,174 +0,0 @@ - - -# Advanced Guide: Export OpenTelemetry Data - -Use this guide when you want NeMo Flow lifecycle events exported as generic OpenTelemetry Protocol (OTLP) trace spans. - -## What You Build - -You will configure the OpenTelemetry subscriber, register it before instrumented work starts, run scoped work, flush spans, and shut down the exporter. - -Use OpenTelemetry export when your tracing backend already expects OTLP spans and you want NeMo Flow scopes, tool calls, LLM calls, and marks to appear in the same tracing pipeline as the rest of the application. - -## Before You Start - -Complete these steps: - -1. Instrument at least one scope, tool call, or LLM call. -2. Start or identify an OTLP trace collector endpoint. -3. Decide the `service_name`, namespace, and deployment attributes for this process. -4. Add sanitize guardrails for sensitive event payloads before production export. - -For OpenInference-specific span semantics, use [Advanced Guide: Export OpenInference Data](advanced-guide.md). - -## Configure the Exporter - -Set these fields first: - -| Field | Purpose | -|---|---| -| `transport` | OTLP transport. Start with `http_binary`. | -| `endpoint` | OTLP trace endpoint, such as `http://localhost:4318/v1/traces`. | -| `service_name` | Logical service name shown in traces. | -| `service_namespace` | Optional namespace for grouping related services. | -| `service_version` | Optional application or package version. | -| `instrumentation_scope` | Scope name for NeMo Flow spans. | -| `headers` | Optional auth or routing headers. | -| `resource_attributes` | Deployment metadata such as environment or region. | -| `timeout_millis` | Export timeout in milliseconds. | - -`OTEL_*` variables may be used by the underlying OpenTelemetry exporter when values are not set directly in config. Prefer explicit config objects in application code so the docs, tests, and deployment manifests show the active export settings. - -## Register and Export - -The examples below show how to register the exporter and emit data from instrumented -work. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import OpenTelemetryConfig, OpenTelemetrySubscriber - -config = OpenTelemetryConfig() -config.transport = "http_binary" -config.endpoint = "http://localhost:4318/v1/traces" -config.service_name = "agent" -config.service_namespace = "nemo" -config.service_version = "1.0.0" -config.instrumentation_scope = "nemo-flow-otel" -config.timeout_millis = 3000 -config.headers = {"authorization": "Bearer token"} -config.resource_attributes = {"deployment.environment": "dev"} - -subscriber = OpenTelemetrySubscriber(config) -subscriber.register("otel-exporter") - -# Run instrumented application work here. - -subscriber.force_flush() -subscriber.deregister("otel-exporter") -subscriber.shutdown() -``` -::: - -:::{tab-item} Node.js -:sync: node - -```js -const { OpenTelemetrySubscriber } = require("nemo-flow-node"); - -const subscriber = new OpenTelemetrySubscriber({ - transport: "http_binary", - endpoint: "http://localhost:4318/v1/traces", - serviceName: "agent", - serviceNamespace: "nemo", - serviceVersion: "1.0.0", - instrumentationScope: "nemo-flow-otel", - timeoutMillis: 3000, - headers: { authorization: "Bearer token" }, - resourceAttributes: { "deployment.environment": "dev" }, -}); - -subscriber.register("otel-exporter"); - -// Run instrumented application work here. - -subscriber.forceFlush(); -subscriber.deregister("otel-exporter"); -subscriber.shutdown(); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::observability::otel::{OpenTelemetryConfig, OpenTelemetrySubscriber}; - -let subscriber = OpenTelemetrySubscriber::new( - OpenTelemetryConfig::http_binary("agent") - .with_endpoint("http://localhost:4318/v1/traces") - .with_header("authorization", "Bearer token") - .with_resource_attribute("deployment.environment", "dev") - .with_service_namespace("nemo") - .with_service_version("1.0.0") - .with_instrumentation_scope("nemo-flow-otel"), -)?; - -subscriber.register("otel-exporter")?; - -// Run instrumented application work here. - -subscriber.force_flush()?; -subscriber.deregister("otel-exporter")?; -subscriber.shutdown()?; -``` -::: - -:::: - -## Validate the Export - -Check the export in this order: - -1. Application startup should not report exporter construction errors. -2. The collector should receive OTLP trace export requests. -3. The tracing backend should show spans for NeMo Flow scopes, tools, and LLM calls. -4. Span grouping should match the same root scope for one agent run. - -If spans arrive without useful payloads, verify that the application uses managed execution helpers or explicit lifecycle APIs. If payloads contain sensitive fields, add sanitize guardrails before registering the exporter in production. - -## Production Checklist - -Use this checklist before running the pattern in production traffic. - -- Register the subscriber before the first instrumented request. -- Use stable service identity across deployments. -- Keep auth headers and endpoints outside source code. -- Flush during graceful shutdown. -- Redact sensitive payloads before export. -- Filter by `root_uuid` when analyzing concurrent agent runs. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **No spans appear**: Confirm the OTLP endpoint is reachable and the application emits events. -- **Only scope spans appear**: Route tools and LLM calls through managed execute helpers or explicit lifecycle APIs. -- **Exporter hangs during shutdown**: Lower `timeout_millis` and make shutdown flush bounded. -- **Sensitive data appears in traces**: Add sanitize-request and sanitize-response guardrails before registering production exporters. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Add custom redaction with [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md). -- Compare exporter semantics with [Advanced Guide: Export OpenInference Data](advanced-guide.md). -- Review concrete snippets in [Code Examples](code-examples.md#opentelemetry-export). diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md index c7e6c306..23feda2c 100644 --- a/docs/getting-started/configuration.md +++ b/docs/getting-started/configuration.md @@ -26,7 +26,7 @@ Plugins use a structured plugin configuration with: - One or more component definitions - Optional component policy -Start with [Basic Guide: Define a Plugin](../build-plugins/basic-guide.md) when you need reusable middleware, subscribers, or adaptive behavior. +Start with [Define a Plugin](../build-plugins/basic-guide.md) when you need reusable middleware, subscribers, or adaptive behavior. The `nemo-flow` CLI gateway reads plugin files named `plugins.toml`. See [Plugin Configuration Files](../build-plugins/plugin-configuration-files.md) @@ -35,12 +35,13 @@ rules. ## Observability Setup -ATOF exporters, ATIF exporters, OpenTelemetry subscribers, and OpenInference -subscribers can be configured directly through binding-native config objects. -Use the built-in `observability` plugin when you want one plugin component to -own standard exporter setup and teardown. See -[Configure the Observability Plugin](../export-observability-data/observability-plugin.md) -and [Export Observability Data](../export-observability-data/code-examples.md) +Agent Trajectory Observability Format (ATOF) exporters, Agent Trajectory +Interchange Format (ATIF) exporters, OpenTelemetry subscribers, and +OpenInference subscribers can be configured directly through binding-native +config objects. Use the built-in `observability` plugin when you want one +plugin component to own standard exporter setup and teardown. See +[Observability Configuration](../plugins/observability/configuration.md) +and [Observability](../plugins/observability/about.md) for the supported export paths. NeMo Flow does not require application-level environment variables for normal @@ -54,4 +55,4 @@ deployment manifests. ## Adaptive Setup -Adaptive optimization is enabled through the adaptive plugin component and binding helper APIs. See [Configure Adaptive Optimization](../use-adaptive-optimization/configure.md). +Adaptive optimization is enabled through the adaptive plugin component and binding helper APIs. See [Adaptive Configuration](../plugins/adaptive/configuration.md). diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 1beabf2a..374b8ce1 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -5,19 +5,29 @@ SPDX-License-Identifier: Apache-2.0 # Installation -Choose the installation path that matches how you plan to use NeMo Flow. +Use this page when you are consuming a published NeMo Flow release from a +package manager. -## Install from a Package Manager +If you are working from a source checkout, validating unpublished changes, or +contributing to the repository, use +[Development Setup](../contribute/development-setup.md) instead. -Use a package manager when you are consuming a published release in an application workspace. +## CLI -### Python +Install the NeMo Flow CLI when you want the `nemo-flow` executable for +coding-agent hook and LLM gateway observability. + +```bash +cargo install nemo-flow-cli@0.2.0 +``` + +## Python Install the Python package when your application uses NeMo Flow through the Python wrapper. ```bash -uv add nemo-flow +uv add nemo-flow@0.2.0 ``` Use `uv add` from an application project that has a `pyproject.toml`; it records @@ -26,7 +36,7 @@ virtual environment and do not have project metadata, use `uv pip install nemo-flow` instead. You can also use `pip install nemo-flow` if you are not managing the environment with `uv`. -### Node.js +## Node.js Install the Node.js package when your application uses NeMo Flow through the JavaScript API. @@ -35,144 +45,48 @@ JavaScript API. npm install nemo-flow-node ``` -### Rust +## Rust Add the Rust crates when your application uses NeMo Flow directly from Rust. -```toml -[dependencies] -nemo-flow = "0.1.*" -nemo-flow-adaptive = "0.1.*" +```bash +cargo add nemo-flow@0.2.0 +cargo add nemo-flow-adaptive@0.2.0 ``` - `nemo-flow` provides the core runtime APIs for scopes, middleware, subscribers, plugins, tool calls, and LLM calls. - `nemo-flow-adaptive` provides adaptive runtime primitives and Redis-backed learning components when you want adaptive optimization behavior in Rust. -- `nemo-flow-cli` is a published binary crate for coding-agent hook and LLM - gateway observability. Install it with `cargo install nemo-flow-cli` when - you need the `nemo-flow` executable. - -## Install from Source - -Use a source checkout when you want an application to consume a local NeMo Flow repository: - -```bash -git clone https://github.com/NVIDIA/NeMo-Flow.git ../NeMo-Flow -``` - -### Python - -Use an editable Python install when your application should import a local -checkout. - -```bash -pip install -e ../NeMo-Flow -``` - -If your application is a `uv` project, add the local checkout as an editable -dependency from the application directory: - -```bash -uv add --editable ../NeMo-Flow -``` - -That command records the source path in your application's `pyproject.toml`. - -Run `uv sync` from the cloned `../NeMo-Flow` checkout when you are developing -the NeMo Flow repository itself. - -### Node.js - -Build and install the local Node.js package when your application should consume -the checkout version. - -```bash -cd ../NeMo-Flow -npm install --ignore-scripts -npm run build --workspace=nemo-flow-node -cd - -npm install ../NeMo-Flow/crates/node -``` - -### Rust - -Use path dependencies from your application: - -```toml -[dependencies] -nemo-flow = { path = "../NeMo-Flow/crates/core" } -nemo-flow-adaptive = { path = "../NeMo-Flow/crates/adaptive" } -``` -Install the published gateway binary when you need coding-agent hook and LLM -gateway observability: +## Integrations -```bash -cargo install nemo-flow-cli -``` - -## Install from the Repository - -Use the repository workflow when you are developing against local source, validating unpublished changes, or working across multiple bindings. - -### Development Tooling - -Install `uv` by following the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/), then verify that it is available: - -```bash -uv --version -``` - -Install the required development task runner before you use the repository build, test, or documentation commands: - -```bash -cargo install just --locked -``` - -Verify that `just` is available: - -```bash -just --version -``` +Install integration packages when your application already uses one of the +supported framework or agent harness surfaces. -### Python Development Setup +### OpenClaw -Use this setup when you need the Python development environment for tests or docs -tooling. +Install the OpenClaw plugin through OpenClaw so OpenClaw can register and manage +the package: ```bash -uv sync +openclaw plugins install npm:nemo-flow-openclaw@0.2.0 +openclaw gateway restart ``` -This command installs the Python package in editable mode, builds the native extension through `maturin`, and installs the docs, test, and development dependencies used across the repo. - -### Node.js Development Setup - -Use this setup when you need the Node.js package dependencies for binding work. +Use the package name `nemo-flow-openclaw` for installation. Use the plugin ID +`nemo-flow` in OpenClaw configuration, inspection, and gateway status commands. +See the [OpenClaw Plugin Guide](../integrations/openclaw-plugin.md) for +configuration and verification steps. -```bash -npm install --ignore-scripts -npm run build --workspace=nemo-flow-node -``` +### Python Framework Integrations -### Rust Development Setup - -Use this setup when you need Rust tooling for core runtime or native binding work. +Install the Python package with the supported framework extras when your +application uses LangChain, LangGraph, or Deep Agents. ```bash -cargo build --workspace +uv add "nemo-flow[langchain,langgraph,deepagents]@0.2.0" ``` -If you only need the core Rust crate: - -```bash -cargo build -p nemo-flow -``` - -## Documentation Tooling - -Use these commands when you work on the documentation site: - -```bash -just docs -just docs-linkcheck -``` +The extras install the NeMo Flow Python package plus the dependencies needed by +the maintained public integrations. See +[Supported Integrations](../integrations/about.md) for guide links and support +levels. diff --git a/docs/getting-started/nodejs.md b/docs/getting-started/nodejs.md index 68332cac..0279dc2f 100644 --- a/docs/getting-started/nodejs.md +++ b/docs/getting-started/nodejs.md @@ -10,8 +10,16 @@ events. ## Choose an Install Path -Pick the installation path that matches whether you are using a local checkout or a -published package. +Pick the installation path that matches whether you are using a published package or a +local checkout. + +### Install from a Package Manager + +Use this path when you want the published package for application development. + +```bash +npm install nemo-flow-node +``` ### Install from the Repository @@ -25,14 +33,6 @@ npm run build --workspace=nemo-flow-node This path is for local source development when you need to build the binding from the repository checkout. -### Install from a Package Manager - -Use this path when you want the published package for application development. - -```bash -npm install nemo-flow-node -``` - ## Run One Scope, One Tool Call, and One LLM Call The example below runs one minimal instrumented workflow through the binding. @@ -118,4 +118,4 @@ Use these links to continue from the quick start into the core runtime concepts. - [Instrument Applications Code Examples](../instrument-applications/code-examples.md) - [Subscribers](../about/concepts/subscribers.md) -- [Advanced Guide: Using Codecs](../integrate-frameworks/using-codecs.md) +- [Using Codecs](../integrate-frameworks/using-codecs.md) diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index 2be96641..0b707629 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -12,8 +12,8 @@ Install the tooling for the binding you plan to use. | Rust | 1.86 or newer | Rust builds, local workspace builds, and the Rust core runtime | | Python | 3.11 or newer | Python bindings, Python tests, and docs tooling | | Node.js | 20 or newer | Node.js bindings and generated Node.js API docs | -| `uv` | see [Installation](installation.md#development-tooling) | Python environments, docs builds, and repository setup | -| `just` | see [Installation](installation.md) | Repository development, test, build, and docs task aliases | +| `uv` | see [Development Setup](../contribute/development-setup.md) | Python environments, docs builds, and repository setup | +| `just` | see [Development Setup](../contribute/development-setup.md) | Repository development, test, build, and docs task aliases | The primary documentation track covers Rust, Python, and Node.js. Go, WebAssembly, and the raw FFI surface are experimental and source-first. diff --git a/docs/getting-started/python/index.md b/docs/getting-started/python/index.md index ef0b7544..8b2e10d5 100644 --- a/docs/getting-started/python/index.md +++ b/docs/getting-started/python/index.md @@ -7,12 +7,25 @@ SPDX-License-Identifier: Apache-2.0 This quick start shows the smallest Python workflow that emits scope, tool, and LLM events. -[LangChain](https://www.langchain.com/langchain), [LangGraph](https://www.langchain.com/langgraph), and [Deep Agents](https://www.langchain.com/deep-agents) users should start with the [LangChain integration](langchain.md), [LangGraph integration](langgraph.md), and [Deep Agents integration](deepagents.md) guides for the best experience in those frameworks. +[LangChain](https://www.langchain.com/langchain), [LangGraph](https://www.langchain.com/langgraph), and [Deep Agents](https://www.langchain.com/deep-agents) users should start with the [LangChain integration](../../integrations/langchain.md), [LangGraph integration](../../integrations/langgraph.md), and [Deep Agents integration](../../integrations/deepagents.md) guides for the best experience in those frameworks. ## Choose an Install Path -Pick the installation path that matches whether you are using a local checkout or a -published package. +Pick the installation path that matches whether you are using a published package or a +local checkout. + +### Install from a Package Manager + +Use this path when you want the published package for application development. + +```bash +uv add nemo-flow@0.2.0 +``` + +Run `uv add` from an application project that has a `pyproject.toml`; it records +`nemo-flow` as a dependency. If you are only installing into an active virtual +environment, use `uv pip install nemo-flow`. If you are not using `uv`, install +the published package with `pip install nemo-flow`. ### Install from the Repository @@ -35,19 +48,6 @@ uv add --editable ../NeMo-Flow This records the local source in the application's `pyproject.toml` through `[tool.uv.sources]`. -### Install from a Package Manager - -Use this path when you want the published package for application development. - -```bash -uv add nemo-flow -``` - -Run `uv add` from an application project that has a `pyproject.toml`; it records -`nemo-flow` as a dependency. If you are only installing into an active virtual -environment, use `uv pip install nemo-flow`. If you are not using `uv`, install -the published package with `pip install nemo-flow`. - ## Run One Scope, One Tool Call, and One LLM Call The example below runs one minimal instrumented workflow through the binding. @@ -127,23 +127,7 @@ These modules are the main Python APIs to use from applications and integrations Use these links to continue from the quick start into the core runtime concepts. -- [LangChain integration](langchain.md) -- [LangGraph integration](langgraph.md) -- [Deep Agents integration](deepagents.md) +- [Supported Integrations](../../integrations/about.md) - [Scopes](../../about/concepts/scopes.md) - [Middleware](../../about/concepts/middleware.md) - [Plugins](../../about/concepts/plugins.md) - -## Framework Integrations - -Use these guides when your Python application already uses LangChain, -LangGraph, or Deep Agents and you want NeMo Flow observability through their -public APIs. - -```{toctree} -:maxdepth: 1 - -LangChain Integration -LangGraph Integration -Deep Agents Integration -``` diff --git a/docs/getting-started/rust.md b/docs/getting-started/rust.md index 478c1649..e4420cd8 100644 --- a/docs/getting-started/rust.md +++ b/docs/getting-started/rust.md @@ -9,8 +9,25 @@ This quick start shows the smallest Rust workflow that emits scope and mark even ## Choose an Install Path -Pick the installation path that matches whether you are using a local checkout or a -published package. +Pick the installation path that matches whether you are using a published package or a +local checkout. + +### Install from a Package Manager + +Use the published crates when you are consuming a release: + +```bash +cargo add nemo-flow@0.2.0 +cargo add nemo-flow-adaptive@0.2.0 +cargo add serde_json +``` + +Install the published NeMo Flow CLI separately when you need coding-agent hook +and LLM gateway observability: + +```bash +cargo install nemo-flow-cli@0.2.0 +``` ### Install from the Repository @@ -25,26 +42,8 @@ serde_json = "1" - `nemo-flow` is the core Rust runtime surface. - `nemo-flow-adaptive` is the companion crate for adaptive runtime primitives and Redis-backed learning components. -- `nemo-flow-cli` is a binary crate. Use `cargo install nemo-flow-cli` when - you need the coding-agent gateway. - -### Install from a Package Manager - -Use the published crate when you are consuming a release: - -```toml -[dependencies] -nemo-flow = "0.1.*" -nemo-flow-adaptive = "0.1.*" -serde_json = "1" -``` - -Install the published gateway binary separately when you need coding-agent hook -and LLM gateway observability: - -```bash -cargo install nemo-flow-cli -``` +- `nemo-flow-cli` is a binary crate. Use `cargo install nemo-flow-cli@0.2.0` when + you need the NeMo Flow CLI. ## Push a Scope and Emit a Mark diff --git a/docs/index.md b/docs/index.md index adf5cd62..a51f42b8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -39,9 +39,9 @@ targets. execution. Plugins package that behavior so applications and framework integrations can install it from configuration. - **Export one event stream to many backends**: Subscribers consume the canonical - lifecycle stream in-process or translate it to ATIF trajectories, - OpenTelemetry traces, and OpenInference-compatible traces for debugging, - evaluation, and production observability. + lifecycle stream in-process or translate it to Agent Trajectory Interchange + Format (ATIF) trajectories, OpenTelemetry traces, and OpenInference-compatible + traces for debugging, evaluation, and production observability. - **Adopt without replacing the stack**: NeMo Flow can sit below NeMo ecosystem components, third-party agent frameworks, provider adapters, or direct application code, so teams can add shared runtime semantics without a @@ -51,15 +51,24 @@ targets. authors, plugin authors, and application teams reason about behavior consistently. -## Use Cases - -These paths map common reader goals to the most relevant documentation entry points. - -- **End Users**: Start with [Prerequisites](getting-started/prerequisites.md) and [Quick Start](getting-started/quick-start.md). -- **Coding-Agent CLI Users**: Start with [Advanced Guide: Coding-Agent Gateway](integrate-frameworks/coding-agent-gateway.md), then use the per-agent guide for Claude Code, Codex, Cursor, or Hermes. -- **Agent Framework Developers**: Start with [Integrate into Frameworks](integrate-frameworks/about.md). -- **Plugin Writers**: Start with [Build Plugins](build-plugins/about.md), then continue to [Basic Guide: Define a Plugin](build-plugins/basic-guide.md). -- **Contributors**: Start with [Contribute](contribute/about.md) and the repository root `CONTRIBUTING.md` guide. +## What Should I Read First? + +Use the reading path that matches your task: + +| Task | Start With | +|---|---| +| Run a minimal example | [Quick Start](getting-started/quick-start.md) | +| Install packages | [Installation](getting-started/installation.md) | +| Develop from source | [Development Setup](contribute/development-setup.md) | +| Understand the runtime model | [Concepts](about/concepts/index.md) | +| Instrument an application | [Instrument Applications](instrument-applications/about.md) | +| Use a maintained integration | [Supported Integrations](integrations/about.md) | +| Integrate a framework | [Integrate into Frameworks](integrate-frameworks/about.md) | +| Observe a local coding-agent CLI | [NeMo Flow CLI](nemo-flow-cli/about.md) | +| Package reusable behavior | [Build Plugins](build-plugins/about.md) | +| Export traces or trajectories | [Observability](plugins/observability/about.md) | +| Configure adaptive behavior | [Adaptive](plugins/adaptive/about.md) | +| Look up symbols | [APIs](reference/api/index.md) | ## Conceptual Diagram @@ -140,80 +149,97 @@ Configuration / Setup Quick Start ``` +```{toctree} +:hidden: +:caption: NeMo Flow CLI +:maxdepth: 2 + +About +Basic Usage +Claude Code +Codex +Cursor +Hermes Agent +``` + +```{toctree} +:hidden: +:caption: Supported Integrations +:maxdepth: 2 + +About +OpenClaw Plugin Guide +LangChain Integration Guide +LangGraph Integration Guide +Deep Agents Integration Guide +``` + ```{toctree} :hidden: :caption: Instrument Applications :maxdepth: 2 About -Basic Guide: Adding Scopes and Marks -Basic Guide: Instrument a Tool Call -Basic Guide: Instrument an LLM Call -Advanced Guide: Add Middleware +Adding Scopes and Marks +Instrument a Tool Call +Instrument an LLM Call +Add Middleware Code Examples ``` ```{toctree} :hidden: -:caption: Integrate into Frameworks +:caption: Observability Plugin :maxdepth: 2 -About -Basic Guide: Adding Scopes -Basic Guide: Wrap Tool Calls -Basic Guide: Wrap LLM Calls -Advanced Guide: Coding-Agent Gateway -OpenClaw Plugin Guide -Claude Code Gateway Guide -Codex Gateway Guide -Cursor Gateway Guide -Hermes Gateway Guide -Advanced Guide: Handle Non-Serializable Data -Advanced Guide: Using Codecs -Advanced Guide: Provider Codecs -Advanced Guide: Provider Response Codecs -Code Examples +About +Configuration +Agent Trajectory Interchange Format (ATIF) +Agent Trajectory Observability Format (ATOF) +OpenTelemetry +OpenInference ``` ```{toctree} :hidden: -:caption: Build Plugins +:caption: Adaptive Plugin :maxdepth: 2 -About -Basic Guide: Define a Plugin -Basic Guide: Validate Plugin Configuration -Basic Guide: Plugin Configuration Files -Basic Guide: Register Plugin Behavior -Advanced Guide: Design Plugin Configuration -NeMo Guardrails Example Plugin -Code Examples +About +Configuration +Adaptive Cache Governor (ACG) +Adaptive Hints ``` ```{toctree} :hidden: -:caption: Export Observability Data +:caption: Integrate into Frameworks :maxdepth: 2 -About -Basic Guide: Register a Subscriber -Basic Guide: Configure the Observability Plugin -Advanced Guide: Export OpenTelemetry Data -Advanced Guide: Export OpenInference Data -Advanced Guide: Export ATIF -Code Examples +About +Adding Scopes +Wrap Tool Calls +Wrap LLM Calls +Handle Non-Serializable Data +Using Codecs +Provider Codecs +Provider Response Codecs +Code Examples ``` ```{toctree} :hidden: -:caption: Use Adaptive Optimization +:caption: Build Plugins :maxdepth: 2 -About -Basic Guide: Configure Adaptive Optimization -Advanced Guide: Configure Adaptive Components -Advanced Guide: Tune Adaptive Behavior -Code Examples +About +Define a Plugin +Validate Plugin Configuration +Plugin Configuration Files +Register Plugin Behavior +Design Plugin Configuration +NeMo Guardrails Example Plugin +Code Examples ``` ```{toctree} @@ -232,7 +258,7 @@ Testing and Documentation :caption: Reference :maxdepth: 2 -API +APIs reference/performance ``` diff --git a/docs/instrument-applications/about.md b/docs/instrument-applications/about.md index 2f318b09..da65adcf 100644 --- a/docs/instrument-applications/about.md +++ b/docs/instrument-applications/about.md @@ -29,10 +29,10 @@ If the tool or LLM boundary is owned by a framework, use [Integrate into Framewo These guides show how to instrument applications with scopes, tool calls, LLM calls, middleware, and direct API examples. -- [Basic Guide: Adding Scopes and Marks](adding-scopes-and-marks.md) shows how to create ownership boundaries and checkpoint events before adding call instrumentation. -- [Basic Guide: Instrument a Tool Call](instrument-tool-call.md) shows the smallest managed tool wrapper with event validation. -- [Basic Guide: Instrument an LLM Call](instrument-llm-call.md) shows the smallest managed model-provider wrapper with event validation. -- [Advanced Guide: Add Middleware](advanced-guide.md) shows how to add guardrails, request intercepts, execution intercepts, and scope-local behavior. +- [Adding Scopes and Marks](adding-scopes-and-marks.md) shows how to create ownership boundaries and checkpoint events before adding call instrumentation. +- [Instrument a Tool Call](instrument-tool-call.md) shows the smallest managed tool wrapper with event validation. +- [Instrument an LLM Call](instrument-llm-call.md) shows the smallest managed model-provider wrapper with event validation. +- [Add Middleware](advanced-guide.md) shows how to add guardrails, request intercepts, execution intercepts, and scope-local behavior. - [Code Examples](code-examples.md) collects direct API examples for tools, LLMs, streaming calls, scopes, and partial middleware helpers. Start with scopes and marks, then instrument the call boundaries your diff --git a/docs/instrument-applications/adding-scopes-and-marks.md b/docs/instrument-applications/adding-scopes-and-marks.md index f983a965..e0d964b0 100644 --- a/docs/instrument-applications/adding-scopes-and-marks.md +++ b/docs/instrument-applications/adding-scopes-and-marks.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Adding Scopes and Marks +# Adding Scopes and Marks Use this guide when you want NeMo Flow to identify one agent run, request, workflow, or operation before you instrument individual tool and LLM calls. @@ -206,6 +206,6 @@ Use this checklist before running the pattern in production traffic. Use these links to continue from this workflow into the next related task. -- Add tool instrumentation with [Basic Guide: Instrument a Tool Call](instrument-tool-call.md). -- Add model-provider instrumentation with [Basic Guide: Instrument an LLM Call](instrument-llm-call.md). +- Add tool instrumentation with [Instrument a Tool Call](instrument-tool-call.md). +- Add model-provider instrumentation with [Instrument an LLM Call](instrument-llm-call.md). - Use [Code Examples](code-examples.md#scope-and-context-helpers) for explicit scope-stack propagation helpers. diff --git a/docs/instrument-applications/advanced-guide.md b/docs/instrument-applications/advanced-guide.md index b1c48a21..a68cd111 100644 --- a/docs/instrument-applications/advanced-guide.md +++ b/docs/instrument-applications/advanced-guide.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Add Middleware +# Add Middleware Use this guide when instrumentation is working and you want NeMo Flow to enforce policy, transform requests, wrap execution, or sanitize observability payloads around tool and LLM calls. @@ -18,7 +18,7 @@ You will add middleware to an instrumented application and verify that it runs i ## Before You Start -Complete [Basic Guide: Instrument a Tool Call](instrument-tool-call.md) or [Basic Guide: Instrument an LLM Call](instrument-llm-call.md). Middleware only runs when the call goes through a NeMo Flow managed lifecycle API. +Complete [Instrument a Tool Call](instrument-tool-call.md) or [Instrument an LLM Call](instrument-llm-call.md). Middleware only runs when the call goes through a NeMo Flow managed lifecycle API. ## Choose the Middleware Type @@ -232,4 +232,4 @@ Use these links to continue from this workflow into the next related task. - Use [Middleware](../about/concepts/middleware.md) to review execution order. - Use [Code Examples](code-examples.md) for direct registration and partial-execution examples. -- Use [Advanced Guide: Handle Non-Serializable Data](../integrate-frameworks/non-serializable-data.md) if middleware needs to work with framework objects. +- Use [Handle Non-Serializable Data](../integrate-frameworks/non-serializable-data.md) if middleware needs to work with framework objects. diff --git a/docs/instrument-applications/code-examples.md b/docs/instrument-applications/code-examples.md index 327515b4..7b138b59 100644 --- a/docs/instrument-applications/code-examples.md +++ b/docs/instrument-applications/code-examples.md @@ -429,4 +429,4 @@ Every family also has a scope-local surface: `nemo_flow::api::registry`; subscriber scope registration under `nemo_flow::api::subscriber` -Use [Advanced Guide: Add Middleware](advanced-guide.md) for an end-to-end policy example and [API Reference](../reference/api/index.md) for symbol-level details. +Use [Add Middleware](advanced-guide.md) for an end-to-end policy example and [API Reference](../reference/api/index.md) for symbol-level details. diff --git a/docs/instrument-applications/instrument-llm-call.md b/docs/instrument-applications/instrument-llm-call.md index d78223b8..5129c69f 100644 --- a/docs/instrument-applications/instrument-llm-call.md +++ b/docs/instrument-applications/instrument-llm-call.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Instrument an LLM Call +# Instrument an LLM Call Use this guide when you own the model-provider callback and want NeMo Flow to emit lifecycle events, apply LLM middleware, and preserve the active agent scope around the call. @@ -26,7 +26,7 @@ Complete one binding Quick Start guide first: - [Node.js Quick Start](../getting-started/nodejs.md) - [Rust Quick Start](../getting-started/rust.md) -Create a scope for the active request or agent run before adding LLM instrumentation. If you have not done that yet, start with [Basic Guide: Adding Scopes and Marks](adding-scopes-and-marks.md). +Create a scope for the active request or agent run before adding LLM instrumentation. If you have not done that yet, start with [Adding Scopes and Marks](adding-scopes-and-marks.md). The request and response payloads must be JSON-compatible. If your provider SDK uses clients, streams, callbacks, or other opaque objects, keep those objects in the provider callback and pass only a serializable request projection into NeMo Flow. @@ -248,7 +248,7 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Instrument tools with [Basic Guide: Instrument a Tool Call](instrument-tool-call.md). -- Add policy or transformation with [Advanced Guide: Add Middleware](advanced-guide.md). -- Use [Advanced Guide: Provider Codecs](../integrate-frameworks/provider-codecs.md) when middleware needs normalized LLM request and response data. -- Export events with [Basic Guide: Register a Subscriber](../export-observability-data/basic-guide.md). +- Instrument tools with [Instrument a Tool Call](instrument-tool-call.md). +- Add policy or transformation with [Add Middleware](advanced-guide.md). +- Use [Provider Codecs](../integrate-frameworks/provider-codecs.md) when middleware needs normalized LLM request and response data. +- Export events with [Observability](../plugins/observability/about.md). diff --git a/docs/instrument-applications/instrument-tool-call.md b/docs/instrument-applications/instrument-tool-call.md index edec2678..76d286aa 100644 --- a/docs/instrument-applications/instrument-tool-call.md +++ b/docs/instrument-applications/instrument-tool-call.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Instrument a Tool Call +# Instrument a Tool Call Use this guide when you have an application tool callback and want NeMo Flow to emit lifecycle events, apply middleware, and preserve the active agent scope around the call. @@ -25,9 +25,9 @@ Complete one binding Quick Start guide first: - [Node.js Quick Start](../getting-started/nodejs.md) - [Rust Quick Start](../getting-started/rust.md) -Create a scope for the active request or agent run before adding tool instrumentation. If you have not done that yet, start with [Basic Guide: Adding Scopes and Marks](adding-scopes-and-marks.md). +Create a scope for the active request or agent run before adding tool instrumentation. If you have not done that yet, start with [Adding Scopes and Marks](adding-scopes-and-marks.md). -The tool arguments and result must be JSON-compatible. If your framework passes clients, sockets, streams, callbacks, or other opaque objects, use [Advanced Guide: Handle Non-Serializable Data](../integrate-frameworks/non-serializable-data.md) before you instrument the callback. +The tool arguments and result must be JSON-compatible. If your framework passes clients, sockets, streams, callbacks, or other opaque objects, use [Handle Non-Serializable Data](../integrate-frameworks/non-serializable-data.md) before you instrument the callback. ## Integration Pattern @@ -226,7 +226,7 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Add model-provider instrumentation with [Basic Guide: Instrument an LLM Call](instrument-llm-call.md). -- Add policy or transformation with [Advanced Guide: Add Middleware](advanced-guide.md). -- Export events with [Basic Guide: Register a Subscriber](../export-observability-data/basic-guide.md). +- Add model-provider instrumentation with [Instrument an LLM Call](instrument-llm-call.md). +- Add policy or transformation with [Add Middleware](advanced-guide.md). +- Export events with [Observability](../plugins/observability/about.md). - Use [Code Examples](code-examples.md) for manual lifecycle, streaming, scope, and partial middleware API examples. diff --git a/docs/integrate-frameworks/about.md b/docs/integrate-frameworks/about.md index 9ae886dd..c131928b 100644 --- a/docs/integrate-frameworks/about.md +++ b/docs/integrate-frameworks/about.md @@ -29,28 +29,27 @@ Use these signals to decide whether this documentation path matches your current - Are building or reviewing third-party integration patches If you own the application call sites directly, use [Instrument Applications](../instrument-applications/about.md) first. -If your application uses [LangChain](https://www.langchain.com/langchain), -[LangGraph](https://www.langchain.com/langgraph), or [Deep Agents](https://www.langchain.com/deep-agents), start with [LangChain Integration](../getting-started/python/langchain.md), [LangGraph Integration](../getting-started/python/langgraph.md), or [Deep Agents Integration](../getting-started/python/deepagents.md). +If your application uses a maintained public integration such as LangChain, +LangGraph, Deep Agents, or OpenClaw, start with +[Supported Integrations](../integrations/about.md). ## Guides Use these guide links to move from the overview into task-specific instructions. -- [Basic Guide: Adding Scopes](adding-scopes.md) shows how framework request and run hooks become NeMo Flow ownership boundaries. -- [Basic Guide: Wrap Tool Calls](wrap-tool-calls.md) explains where to place managed tool wrappers and tool lifecycle fallbacks. -- [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md) explains where to place managed provider wrappers, model names, streaming behavior, and LLM lifecycle fallbacks. -- [Advanced Guide: Coding-Agent Gateway](coding-agent-gateway.md) describes the Rust gateway for observing Codex, Claude Code, Cursor, and Hermes through canonical hooks plus a passthrough LLM gateway. -- [OpenClaw Plugin Guide](openclaw-plugin.md) covers configuring the OpenClaw plugin, mapping OpenClaw hooks to NeMo Flow telemetry, and understanding current LLM replay fidelity boundaries. -- [Claude Code Gateway Guide](coding-agent-claude-code.md) covers transparent Claude Code runs, Anthropic gateway routing, ATIF verification, and unsupported Claude application modes. -- [Codex Gateway Guide](coding-agent-codex.md) covers transparent Codex CLI runs, local GUI/app caveats, model provider routing, and remote-task limits. -- [Cursor Gateway Guide](coding-agent-cursor.md) covers transparent Cursor runs, temporary hook patching, GUI and CLI smoke tests, and gateway routing limits. -- [Hermes Gateway Guide](coding-agent-hermes.md) covers Hermes shell hook installation, dynamic gateway URL handling, session-finalize behavior, and hook consent caveats. -- [Advanced Guide: Handle Non-Serializable Data](non-serializable-data.md) shows how to keep clients, streams, callbacks, and SDK objects outside JSON payloads. -- [Advanced Guide: Using Codecs](using-codecs.md) explains typed value codecs for framework-facing wrappers. -- [Advanced Guide: Provider Codecs](provider-codecs.md) explains provider request and response codecs for normalized middleware and event annotations. -- [Advanced Guide: Provider Response Codecs](provider-response-codecs.md) focuses on response-only annotations for subscribers and exporters. +- [Adding Scopes](adding-scopes.md) shows how framework request and run hooks become NeMo Flow ownership boundaries. +- [Wrap Tool Calls](wrap-tool-calls.md) explains where to place managed tool wrappers and tool lifecycle fallbacks. +- [Wrap LLM Calls](wrap-llm-calls.md) explains where to place managed provider wrappers, model names, streaming behavior, and LLM lifecycle fallbacks. +- [Handle Non-Serializable Data](non-serializable-data.md) shows how to keep clients, streams, callbacks, and SDK objects outside JSON payloads. +- [Using Codecs](using-codecs.md) explains typed value codecs for framework-facing wrappers. +- [Provider Codecs](provider-codecs.md) explains provider request and response codecs for normalized middleware and event annotations. +- [Provider Response Codecs](provider-response-codecs.md) focuses on response-only annotations for subscribers and exporters. - [Code Examples](code-examples.md) collects fallback APIs, mark events, and repository patch workflow examples. +For coding-agent hook and LLM gateway observability, use +[NeMo Flow CLI](../nemo-flow-cli/about.md). That section covers Claude Code, +Codex, Cursor, and Hermes Agent support. + Start by identifying the framework's stable tool and LLM boundaries. Prefer managed execution wrappers wherever the framework exposes a callback that NeMo Flow can own. Use explicit API calls only when the framework owns invocation diff --git a/docs/integrate-frameworks/adding-scopes.md b/docs/integrate-frameworks/adding-scopes.md index 62e8fa8c..5254b429 100644 --- a/docs/integrate-frameworks/adding-scopes.md +++ b/docs/integrate-frameworks/adding-scopes.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Adding Scopes +# Adding Scopes Use this guide when a framework needs a durable NeMo Flow ownership boundary for one request, agent run, workflow, or framework task. @@ -188,6 +188,6 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Wrap tools with [Basic Guide: Wrap Tool Calls](wrap-tool-calls.md). -- Wrap model providers with [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md). +- Wrap tools with [Wrap Tool Calls](wrap-tool-calls.md). +- Wrap model providers with [Wrap LLM Calls](wrap-llm-calls.md). - Use [Code Examples](../instrument-applications/code-examples.md#scope-and-context-helpers) for explicit scope-stack propagation helpers. diff --git a/docs/integrate-frameworks/code-examples.md b/docs/integrate-frameworks/code-examples.md index d04cacba..9703c208 100644 --- a/docs/integrate-frameworks/code-examples.md +++ b/docs/integrate-frameworks/code-examples.md @@ -7,13 +7,13 @@ SPDX-License-Identifier: Apache-2.0 Use these examples for implementation surfaces: -- [Basic Guide: Adding Scopes](adding-scopes.md) -- [Basic Guide: Wrap Tool Calls](wrap-tool-calls.md) -- [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md) +- [Adding Scopes](adding-scopes.md) +- [Wrap Tool Calls](wrap-tool-calls.md) +- [Wrap LLM Calls](wrap-llm-calls.md) - [Non-Serializable Data](non-serializable-data.md) -- [Advanced Guide: Using Codecs](using-codecs.md) -- [Advanced Guide: Provider Codecs](provider-codecs.md) -- [Advanced Guide: Provider Response Codecs](provider-response-codecs.md) +- [Using Codecs](using-codecs.md) +- [Provider Codecs](provider-codecs.md) +- [Provider Response Codecs](provider-response-codecs.md) ## Preferred Integration Order @@ -289,7 +289,8 @@ event( NeMo Flow keeps sample third-party integrations as patch sets under `patches/` and pinned upstream checkouts under `third_party/`. For the current OpenClaw -end-user integration, use the [OpenClaw Plugin Guide](openclaw-plugin.md). +end-user integration, use the +[OpenClaw Plugin Guide](../integrations/openclaw-plugin.md). The following table lists maintained patch checkouts: diff --git a/docs/integrate-frameworks/non-serializable-data.md b/docs/integrate-frameworks/non-serializable-data.md index 239d20b5..76e745bf 100644 --- a/docs/integrate-frameworks/non-serializable-data.md +++ b/docs/integrate-frameworks/non-serializable-data.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Handle Non-Serializable Data +# Handle Non-Serializable Data Use this guide when a framework exposes SDK clients, streams, callbacks, file handles, or custom classes at the same boundary where you need NeMo Flow instrumentation. @@ -196,7 +196,7 @@ contract. Use these links to continue from this workflow into the next related task. -- Wrap the framework boundary with [Basic Guide: Wrap Tool Calls](wrap-tool-calls.md) or [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md). -- Use [Advanced Guide: Provider Codecs](provider-codecs.md) when provider payloads need normalized request or response annotations. -- Use the typed value codec examples in [Advanced Guide: Using Codecs](using-codecs.md#typed-value-codecs) for structured conversion helpers. -- Use [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md) before adding request transforms. +- Wrap the framework boundary with [Wrap Tool Calls](wrap-tool-calls.md) or [Wrap LLM Calls](wrap-llm-calls.md). +- Use [Provider Codecs](provider-codecs.md) when provider payloads need normalized request or response annotations. +- Use the typed value codec examples in [Using Codecs](using-codecs.md#typed-value-codecs) for structured conversion helpers. +- Use [Add Middleware](../instrument-applications/advanced-guide.md) before adding request transforms. diff --git a/docs/integrate-frameworks/provider-codecs.md b/docs/integrate-frameworks/provider-codecs.md index 48ef37de..b1ebc31f 100644 --- a/docs/integrate-frameworks/provider-codecs.md +++ b/docs/integrate-frameworks/provider-codecs.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Provider Codecs +# Provider Codecs Use this guide when a framework integration needs NeMo Flow middleware, intercepts, or subscribers to reason about provider-specific LLM payloads through a stable annotated shape. @@ -31,7 +31,7 @@ A provider codec is a pure data translator at the NeMo Flow LLM boundary. - An LLM request codec converts a raw provider request into a normalized annotated request, then encodes any annotated edits back into the original provider request. - An LLM response codec converts a raw provider response into a normalized response annotation for lifecycle events. -Provider codecs let framework code keep using provider-native payloads while NeMo Flow middleware works against a shared annotated model. For application-facing type conversion, use [Advanced Guide: Using Codecs](using-codecs.md). +Provider codecs let framework code keep using provider-native payloads while NeMo Flow middleware works against a shared annotated model. For application-facing type conversion, use [Using Codecs](using-codecs.md). ## How Provider Codecs Work @@ -43,7 +43,7 @@ When a managed LLM call has a request codec: 4. NeMo Flow calls `encode` to merge the annotated request back into the original raw request. 5. Execution intercepts and the provider callback receive the encoded provider request. -When a managed LLM call has a response codec, NeMo Flow decodes the raw provider response for observability and attaches the result to the emitted LLM end event. The response codec does not rewrite the value returned to the application. Use [Advanced Guide: Provider Response Codecs](provider-response-codecs.md) for response-only behavior and custom response codec examples. +When a managed LLM call has a response codec, NeMo Flow decodes the raw provider response for observability and attaches the result to the emitted LLM end event. The response codec does not rewrite the value returned to the application. Use [Provider Response Codecs](provider-response-codecs.md) for response-only behavior and custom response codec examples. Codec implementations should preserve fields they do not understand. Treat `encode` as a merge operation over the original provider payload, not as a full replacement. @@ -385,7 +385,7 @@ contract. Use these links to continue from this workflow into the next related task. -- Use [Advanced Guide: Using Codecs](using-codecs.md) for typed value codecs. -- Use [Advanced Guide: Provider Response Codecs](provider-response-codecs.md) for response-only annotations and subscriber examples. -- Use [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md) before adding request transforms. -- Use [Advanced Guide: Handle Non-Serializable Data](non-serializable-data.md) when the framework boundary includes SDK objects or streams that cannot pass through JSON payloads. +- Use [Using Codecs](using-codecs.md) for typed value codecs. +- Use [Provider Response Codecs](provider-response-codecs.md) for response-only annotations and subscriber examples. +- Use [Add Middleware](../instrument-applications/advanced-guide.md) before adding request transforms. +- Use [Handle Non-Serializable Data](non-serializable-data.md) when the framework boundary includes SDK objects or streams that cannot pass through JSON payloads. diff --git a/docs/integrate-frameworks/provider-response-codecs.md b/docs/integrate-frameworks/provider-response-codecs.md index 02e1c37a..81d865ae 100644 --- a/docs/integrate-frameworks/provider-response-codecs.md +++ b/docs/integrate-frameworks/provider-response-codecs.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Provider Response Codecs +# Provider Response Codecs Use this guide when subscribers, exporters, or diagnostics need a provider-neutral view of raw LLM responses. @@ -22,7 +22,7 @@ Response codecs are observability-only: You need: -- A managed LLM boundary from [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md). +- A managed LLM boundary from [Wrap LLM Calls](wrap-llm-calls.md). - A raw provider response that is JSON-compatible. - A built-in response codec or a custom response codec for the provider response shape. - A subscriber or exporter that consumes `annotated_response` from LLM end events. @@ -362,6 +362,6 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Use [Advanced Guide: Provider Codecs](provider-codecs.md) for request-side provider codecs and full request/response examples. -- Use [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md) to add the managed LLM boundary first. -- Use [Export Observability Data](../export-observability-data/about.md) after annotations are visible in local subscribers. +- Use [Provider Codecs](provider-codecs.md) for request-side provider codecs and full request/response examples. +- Use [Wrap LLM Calls](wrap-llm-calls.md) to add the managed LLM boundary first. +- Use [Observability](../plugins/observability/about.md) after annotations are visible in local subscribers. diff --git a/docs/integrate-frameworks/using-codecs.md b/docs/integrate-frameworks/using-codecs.md index b25aab65..df30adb2 100644 --- a/docs/integrate-frameworks/using-codecs.md +++ b/docs/integrate-frameworks/using-codecs.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Using Codecs +# Using Codecs Use this guide when a framework integration needs typed application values at its public boundary while NeMo Flow still records JSON-compatible payloads. @@ -16,7 +16,7 @@ You will choose typed value codecs for framework-facing wrappers so that: - Middleware and subscribers receive predictable serialized values - The framework callback still receives the application type it expects -For provider-native LLM payload normalization, use [Advanced Guide: Provider Codecs](provider-codecs.md). +For provider-native LLM payload normalization, use [Provider Codecs](provider-codecs.md). ## Before You Start @@ -189,6 +189,6 @@ contract. Use these links to continue from this workflow into the next related task. -- Use [Advanced Guide: Provider Codecs](provider-codecs.md) when provider payloads need normalized request or response annotations. -- Use [Advanced Guide: Handle Non-Serializable Data](non-serializable-data.md) when framework objects cannot pass through JSON payloads. +- Use [Provider Codecs](provider-codecs.md) when provider payloads need normalized request or response annotations. +- Use [Handle Non-Serializable Data](non-serializable-data.md) when framework objects cannot pass through JSON payloads. - Use [Integrate into Frameworks Code Examples](code-examples.md) for explicit fallback APIs. diff --git a/docs/integrate-frameworks/wrap-llm-calls.md b/docs/integrate-frameworks/wrap-llm-calls.md index 673122d1..86b327a4 100644 --- a/docs/integrate-frameworks/wrap-llm-calls.md +++ b/docs/integrate-frameworks/wrap-llm-calls.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Wrap LLM Calls +# Wrap LLM Calls Use this guide when a framework, SDK, or provider adapter owns model invocation and you need NeMo Flow to observe and control those provider calls. @@ -15,7 +15,7 @@ You will place a managed NeMo Flow LLM execution wrapper at the provider boundar You need: -- A framework request or run scope. If the framework does not create one yet, start with [Basic Guide: Adding Scopes](adding-scopes.md). +- A framework request or run scope. If the framework does not create one yet, start with [Adding Scopes](adding-scopes.md). - A stable model-provider boundary, such as a provider adapter or client dispatch method. - A JSON-compatible request projection inside `LLMRequest`. - A JSON-compatible response projection for subscribers and exporters. @@ -30,7 +30,7 @@ Follow this sequence to keep framework work attached to the expected runtime con 4. Pass a stable provider name and `model_name`. 5. Keep provider clients, streams, callbacks, and retry state outside emitted JSON payloads. -Use a request or response codec when provider payloads need normalization before middleware or events see them. Use [Advanced Guide: Provider Codecs](provider-codecs.md) for those cases. +Use a request or response codec when provider payloads need normalization before middleware or events see them. Use [Provider Codecs](provider-codecs.md) for those cases. ## Concrete LLM Example @@ -162,6 +162,6 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Add tool integration with [Basic Guide: Wrap Tool Calls](wrap-tool-calls.md). -- Normalize provider payloads with [Advanced Guide: Provider Codecs](provider-codecs.md). -- Use [Advanced Guide: Handle Non-Serializable Data](non-serializable-data.md) for provider clients, streams, and callback objects. +- Add tool integration with [Wrap Tool Calls](wrap-tool-calls.md). +- Normalize provider payloads with [Provider Codecs](provider-codecs.md). +- Use [Handle Non-Serializable Data](non-serializable-data.md) for provider clients, streams, and callback objects. diff --git a/docs/integrate-frameworks/wrap-tool-calls.md b/docs/integrate-frameworks/wrap-tool-calls.md index efce5faf..a1264077 100644 --- a/docs/integrate-frameworks/wrap-tool-calls.md +++ b/docs/integrate-frameworks/wrap-tool-calls.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Basic Guide: Wrap Tool Calls +# Wrap Tool Calls Use this guide when a framework, SDK, or orchestration layer owns tool invocation and you need NeMo Flow to observe and control those calls without changing the framework's public behavior. @@ -15,7 +15,7 @@ You will place a managed NeMo Flow tool execution wrapper at the framework's sta You need: -- A framework request or run scope. If the framework does not create one yet, start with [Basic Guide: Adding Scopes](adding-scopes.md). +- A framework request or run scope. If the framework does not create one yet, start with [Adding Scopes](adding-scopes.md). - A stable tool invocation boundary, such as a callback dispatcher, tool registry, or tool adapter. - A JSON-compatible projection of tool arguments and results. - A subscriber or exporter that can verify emitted tool events. @@ -158,6 +158,6 @@ Check these symptoms first when the workflow does not behave as expected. Use these links to continue from this workflow into the next related task. -- Add model-provider integration with [Basic Guide: Wrap LLM Calls](wrap-llm-calls.md). -- Add request ownership with [Basic Guide: Adding Scopes](adding-scopes.md). -- Use [Advanced Guide: Handle Non-Serializable Data](non-serializable-data.md) when framework objects need ID-based lookup. +- Add model-provider integration with [Wrap LLM Calls](wrap-llm-calls.md). +- Add request ownership with [Adding Scopes](adding-scopes.md). +- Use [Handle Non-Serializable Data](non-serializable-data.md) when framework objects need ID-based lookup. diff --git a/docs/integrations/about.md b/docs/integrations/about.md new file mode 100644 index 00000000..dc615cd6 --- /dev/null +++ b/docs/integrations/about.md @@ -0,0 +1,42 @@ + + +# About + +Use this section when your application already uses a supported framework or +agent harness and you want the maintained NeMo Flow integration path for that +surface. + +Supported integrations are end-user entry points. They use public framework or +plugin APIs where available and document the support level for observability, +security middleware, and adaptive optimization. + +## Support Matrix + +| Agent / Library | Observability | Security | Optimization | Notes | +|:--|:--:|:--:|:--:|:--| +| LangChain | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped tool and LLM calling. | +| LangGraph | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped tool and LLM calling. | +| Deep Agents | βœ… Yes | βœ… Yes | βœ… Yes | Wrapped tool and LLM calling. | +| OpenClaw | βœ… Yes | ❌ No | ❌ No | Observability support; missing middleware for wrapped execution. | + +## Guides + +Use these guide links to move from the support matrix into setup and usage +instructions. + +- [OpenClaw Plugin Guide](openclaw-plugin.md) covers configuring the OpenClaw + plugin, mapping OpenClaw hooks to NeMo Flow telemetry, and understanding + current LLM replay fidelity boundaries. +- [LangChain Integration Guide](langchain.md) covers installing the LangChain + extra and adding NeMo Flow middleware and callbacks to LangChain agents. +- [LangGraph Integration Guide](langgraph.md) covers installing the LangGraph + extra and adding NeMo Flow callbacks to LangGraph workflows. +- [Deep Agents Integration Guide](deepagents.md) covers installing the Deep + Agents extra and capturing Deep Agents-specific marks, skills, subagents, and + human-in-the-loop lifecycle events. + +If you are building a new framework integration or patching framework internals, +use [Integrate into Frameworks](../integrate-frameworks/about.md) instead. diff --git a/docs/getting-started/python/deepagents.md b/docs/integrations/deepagents.md similarity index 97% rename from docs/getting-started/python/deepagents.md rename to docs/integrations/deepagents.md index 77b7eaa2..9fd34f75 100644 --- a/docs/getting-started/python/deepagents.md +++ b/docs/integrations/deepagents.md @@ -116,5 +116,5 @@ It captures: Remote graphs or processes still need NeMo Flow instrumentation in that graph or process to capture their internal model and tool calls. -Refer to [Export Observability Data](../../export-observability-data/about.md) +Refer to [Observability](../plugins/observability/about.md) for details on exporting NeMo Flow observability data to third-party systems. diff --git a/docs/getting-started/python/langchain.md b/docs/integrations/langchain.md similarity index 92% rename from docs/getting-started/python/langchain.md rename to docs/integrations/langchain.md index 89993417..b8437ed2 100644 --- a/docs/getting-started/python/langchain.md +++ b/docs/integrations/langchain.md @@ -103,4 +103,4 @@ print(f"Final response: {final_message.content}") ## Observability -Refer to [Export Observability Data](../../export-observability-data/about.md) for details on exporting NeMo Flow observability data to third-party systems. +Refer to [Observability](../plugins/observability/about.md) for details on exporting NeMo Flow observability data to third-party systems. diff --git a/docs/getting-started/python/langgraph.md b/docs/integrations/langgraph.md similarity index 93% rename from docs/getting-started/python/langgraph.md rename to docs/integrations/langgraph.md index 3bd1074f..f68834ec 100644 --- a/docs/getting-started/python/langgraph.md +++ b/docs/integrations/langgraph.md @@ -118,4 +118,4 @@ pip install "nemo-flow[langgraph,langchain-nvidia]" ## Observability -Refer to [Export Observability Data](../../export-observability-data/about.md) for details on exporting NeMo Flow observability data to third-party systems. +Refer to [Observability](../plugins/observability/about.md) for details on exporting NeMo Flow observability data to third-party systems. diff --git a/docs/integrate-frameworks/openclaw-plugin.md b/docs/integrations/openclaw-plugin.md similarity index 94% rename from docs/integrate-frameworks/openclaw-plugin.md rename to docs/integrations/openclaw-plugin.md index 19e45c77..458d6242 100644 --- a/docs/integrate-frameworks/openclaw-plugin.md +++ b/docs/integrations/openclaw-plugin.md @@ -9,7 +9,13 @@ Use the OpenClaw plugin when OpenClaw owns the agent, tool, and LLM lifecycle that needs NeMo Flow observability. The plugin observes supported OpenClaw plugin hooks and converts them into NeMo Flow sessions, LLM spans, tool spans, and marks that the generic NeMo Flow observability component can export as -ATIF JSON, OpenTelemetry spans, and OpenInference/Phoenix spans. +Agent Trajectory Interchange Format (ATIF) JSON, OpenTelemetry spans, and +OpenInference/Phoenix spans. + +This public OpenClaw plugin provides observability support only. It does not +add NeMo Flow security middleware or adaptive optimization behavior to OpenClaw +execution. For middleware-backed behavior, use the patch-based OpenClaw +integration from the NeMo Flow repository. Use this guide to install the plugin, enable it in OpenClaw, configure telemetry outputs, verify exported traces, and understand current LLM replay fidelity. @@ -32,7 +38,7 @@ Optional: Install the plugin with OpenClaw so OpenClaw can register and manage it: ```bash -openclaw plugins install npm:nemo-flow-openclaw +openclaw plugins install npm:nemo-flow-openclaw@0.2.0 openclaw gateway restart ``` @@ -45,7 +51,7 @@ If you manage OpenClaw plugin dependencies directly in a Node.js project, install the package with npm: ```bash -npm install nemo-flow-openclaw +npm install nemo-flow-openclaw@0.2.0 ``` Installing with npm makes the package available to that project. Use @@ -170,7 +176,7 @@ Missing observability sections are disabled. Plugin-host validation or initialization errors degrade the NeMo Flow runtime as a whole, and the status method reports configured output health from the generic observability component. See -[Configure the Observability Plugin](../export-observability-data/observability-plugin.md) +[Observability Configuration](../plugins/observability/configuration.md) for the complete `observability` component schema and exporter-specific fields. ## Verify the Integration @@ -220,6 +226,9 @@ reason when present. The plugin maps supported OpenClaw hook events into NeMo Flow telemetry without changing OpenClaw execution behavior. +It does not change OpenClaw tool execution, provider routing, policy decisions, +or adaptive behavior. + | OpenClaw hook | NeMo Flow behavior | | --- | --- | | `gateway_start` | Touches the replay backend early; session roots still open lazily from session-scoped hooks. | diff --git a/docs/nemo-flow-cli/about.md b/docs/nemo-flow-cli/about.md new file mode 100644 index 00000000..c2e4d30f --- /dev/null +++ b/docs/nemo-flow-cli/about.md @@ -0,0 +1,63 @@ + + +# About + +Use this section when you want the `nemo-flow` binary to observe local coding +agent sessions through hooks, a passthrough LLM gateway, and NeMo Flow +observability exporters. + +The NeMo Flow CLI is installed by the `nemo-flow-cli` Cargo package. It can run +supported coding agents through a managed local gateway, forward agent hook +payloads into NeMo Flow lifecycle events, route OpenAI-compatible or +Anthropic-compatible model traffic through the gateway, and diagnose local +configuration. + +## Start Here When + +Use these guides when you need to: + +- Observe Claude Code, Codex, Cursor, or Hermes Agent sessions locally. +- Configure coding-agent hooks for NeMo Flow lifecycle events. +- Route model-provider traffic through the local NeMo Flow gateway. +- Export local sessions to Agent Trajectory Interchange Format (ATIF), Agent + Trajectory Observability Format (ATOF) JSONL, OpenTelemetry, or + OpenInference. +- Diagnose hook loading, gateway routing, and exporter output. + +If you are instrumenting an application or framework directly, use +[Instrument Applications](../instrument-applications/about.md) or +[Integrate into Frameworks](../integrate-frameworks/about.md) instead. + +## Agent Harness Support + +NeMo Flow CLI support is experimental and observability-focused. + +| Agent | Observability | Security | Optimization | Notes | +| --- | --- | --- | --- | --- | +| Claude Code | βœ… Yes | ❌ No | ❌ No | Observability only; no known issues. | +| Codex | βœ… Yes | ❌ No | ❌ No | Observability only; some hooks needed for full feature coverage are missing. | +| Hermes Agent | βœ… Yes | ❌ No | ❌ No | Observability only; no known issues. | +| Cursor | βœ… Yes | ❌ No | ❌ No | Observability only; missing hooks under `cursor-agent` limit feature coverage. | + +## Guides + +Use these guide links to move from CLI setup into agent-specific instructions. + +- [Basic Usage](basic-usage.md) explains gateway routes, transparent runs, + shared configuration, hook forwarding, and runtime mapping. +- [Claude Code](claude-code.md) covers transparent Claude Code + runs, Anthropic gateway routing, ATIF verification, and unsupported Claude + application modes. +- [Codex](codex.md) covers transparent Codex CLI runs, local + GUI/app caveats, model provider routing, and remote-task limits. +- [Cursor](cursor.md) covers transparent Cursor runs, temporary + hook patching, GUI and CLI smoke tests, and gateway routing limits. +- [Hermes Agent](hermes.md) covers Hermes shell hook installation, + dynamic gateway URL handling, session-finalize behavior, and hook consent + caveats. + +Start with [Basic Usage](basic-usage.md), then use the guide for the coding +agent that you want to observe. diff --git a/docs/integrate-frameworks/coding-agent-gateway.md b/docs/nemo-flow-cli/basic-usage.md similarity index 96% rename from docs/integrate-frameworks/coding-agent-gateway.md rename to docs/nemo-flow-cli/basic-usage.md index 56910b6e..da95f8e4 100644 --- a/docs/integrate-frameworks/coding-agent-gateway.md +++ b/docs/nemo-flow-cli/basic-usage.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Advanced Guide: Coding-Agent Gateway +# Basic Usage The `nemo-flow` binary observes coding agents that do not expose every LLM call site directly. It combines agent-specific hook endpoints with a @@ -257,11 +257,11 @@ Optional flags map to gateway headers: Use the per-agent guide for end-to-end setup, smoke tests, and GUI or application-mode caveats. -- [Claude Code Gateway Guide](coding-agent-claude-code.md) -- [Codex Gateway Guide](coding-agent-codex.md) -- [Cursor Gateway Guide](coding-agent-cursor.md) -- [Hermes Gateway Guide](coding-agent-hermes.md) +- [Claude Code](claude-code.md) +- [Codex](codex.md) +- [Cursor](cursor.md) +- [Hermes Agent](hermes.md) Each guide covers transparent run setup, gateway routing, hook smoke tests, -ATIF export verification on session end, and troubleshooting missing LLM -lifecycle data. +Agent Trajectory Interchange Format (ATIF) export verification on session end, +and troubleshooting missing LLM lifecycle data. diff --git a/docs/integrate-frameworks/coding-agent-claude-code.md b/docs/nemo-flow-cli/claude-code.md similarity index 98% rename from docs/integrate-frameworks/coding-agent-claude-code.md rename to docs/nemo-flow-cli/claude-code.md index 12b99925..8789e6f2 100644 --- a/docs/integrate-frameworks/coding-agent-claude-code.md +++ b/docs/nemo-flow-cli/claude-code.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Claude Code Gateway Guide +# Claude Code Use this guide to observe Claude Code sessions with NeMo Flow. Claude Code is the supported integration target. The Claude application, Claude web, and Claude @@ -122,7 +122,7 @@ is an allow/continue response. ## Verify Export End the Claude Code session and confirm that session-end closed the NeMo Flow -agent scope and wrote ATIF: +agent scope and wrote Agent Trajectory Interchange Format (ATIF): ```bash ls .nemo-flow/atif diff --git a/docs/integrate-frameworks/coding-agent-codex.md b/docs/nemo-flow-cli/codex.md similarity index 95% rename from docs/integrate-frameworks/coding-agent-codex.md rename to docs/nemo-flow-cli/codex.md index 3d6e6542..462d9f8e 100644 --- a/docs/integrate-frameworks/coding-agent-codex.md +++ b/docs/nemo-flow-cli/codex.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Codex Gateway Guide +# Codex Use this guide to observe local Codex CLI sessions and local Codex GUI or app sessions that honor the same local config and gateway routing. Cloud or remote @@ -125,9 +125,7 @@ NeMo Flow events. The transparent wrapper passes hook entries as Codex CLI config overrides and sets `features.hooks=true` for that launched process. Persistent install writes `.codex/config.toml` with `features.hooks = true` and merges generated hook -entries into `.codex/hooks.json`. (`features.codex_hooks` is the legacy alias -of `features.hooks`; new docs and configurations should prefer the canonical -name.) +entries into `.codex/hooks.json`. ## Smoke Test @@ -145,7 +143,8 @@ an empty JSON object. ## Verify Export -End the Codex session and confirm ATIF exists: +End the Codex session and confirm Agent Trajectory Interchange Format (ATIF) +exists: ```bash ls .nemo-flow/atif diff --git a/docs/integrate-frameworks/coding-agent-cursor.md b/docs/nemo-flow-cli/cursor.md similarity index 98% rename from docs/integrate-frameworks/coding-agent-cursor.md rename to docs/nemo-flow-cli/cursor.md index 56b20b2d..d5fe3d5f 100644 --- a/docs/integrate-frameworks/coding-agent-cursor.md +++ b/docs/nemo-flow-cli/cursor.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Cursor Gateway Guide +# Cursor Use this guide to observe Cursor hook lifecycle events with NeMo Flow. The repository ships a Cursor hook bundle under `integrations/coding-agents/cursor/` @@ -138,7 +138,8 @@ provider routing is available. ## Verify Export -End the Cursor session and confirm ATIF exists: +End the Cursor session and confirm Agent Trajectory Interchange Format (ATIF) +exists: ```bash ls .nemo-flow/atif diff --git a/docs/integrate-frameworks/coding-agent-hermes.md b/docs/nemo-flow-cli/hermes.md similarity index 97% rename from docs/integrate-frameworks/coding-agent-hermes.md rename to docs/nemo-flow-cli/hermes.md index d9d2a570..b0edf331 100644 --- a/docs/integrate-frameworks/coding-agent-hermes.md +++ b/docs/nemo-flow-cli/hermes.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Hermes Gateway Guide +# Hermes Agent Use this guide to observe local Hermes Agent sessions with NeMo Flow through Hermes shell hooks and the `nemo-flow` gateway. This gateway path is @@ -120,7 +120,8 @@ relying on unattended capture. ## Verify Export -End a Hermes turn or finalize the session and confirm ATIF exists: +End a Hermes turn or finalize the session and confirm Agent Trajectory +Interchange Format (ATIF) exists: ```bash ls .nemo-flow/atif diff --git a/docs/plugins/adaptive/about.md b/docs/plugins/adaptive/about.md new file mode 100644 index 00000000..3aec4a3c --- /dev/null +++ b/docs/plugins/adaptive/about.md @@ -0,0 +1,55 @@ + + +# Adaptive + +Use the Adaptive plugin when you want NeMo Flow to collect runtime signals and +activate measured adaptive behavior through the shared plugin system. + +Adaptive is a first-party plugin component with kind `adaptive`. It uses the +same runtime model as the rest of NeMo Flow: scopes and managed calls emit +lifecycle events, subscribers and learners observe those events, intercepts can +add guidance, and plugin configuration controls what is active. + +The plugin can coordinate: + +- Adaptive state for learned runtime signals. +- Telemetry subscribers for adaptive learners. +- Adaptive hints injected into outgoing model requests. +- Tool-parallelism observation or scheduling behavior. +- Adaptive Cache Governor (ACG) prompt-cache planning. +- Component-local validation policy. + +## Use Adaptive When + +Adaptive is useful when an agent workflow repeats similar work and you want to +observe or tune behavior without hard-coding optimization logic into every +application. + +Start here when you need to: + +- Collect runtime signals before changing behavior. +- Add model-request hints in a controlled way. +- Plan prompt-cache breakpoints for supported providers. +- Evaluate tool parallelism opportunities. +- Share adaptive state across workers when needed. +- Roll out optimization as a configuration change. + +If instrumentation is not in place yet, start with +[Instrument Applications](../../instrument-applications/about.md) or +[Integrate into Frameworks](../../integrate-frameworks/about.md). + +## Pages + +- [Adaptive Configuration](configuration.md) documents the full plugin + component shape, validation, activation, teardown, and whole-plugin settings. +- [ACG](acg.md) explains Adaptive Cache Governor configuration and what prompt + cache planning accomplishes. +- [Adaptive Hints](adaptive-hints.md) explains request hint injection and how + downstream model paths can consume the hints. + +State, telemetry, tool parallelism, and policy are whole-plugin configuration +areas. They are documented on [Adaptive Configuration](configuration.md) rather +than as separate area pages. diff --git a/docs/plugins/adaptive/acg.md b/docs/plugins/adaptive/acg.md new file mode 100644 index 00000000..558ebd0f --- /dev/null +++ b/docs/plugins/adaptive/acg.md @@ -0,0 +1,262 @@ + + +# Adaptive Cache Governor (ACG) + +Use the Adaptive Cache Governor (ACG) when repeated LLM requests contain stable +prompt sections that can benefit from provider prompt caching. + +ACG decomposes LLM requests into Prompt IR, scores block stability across +observed runs, and plans provider-specific prompt-cache breakpoints. The `acg` +section is optional. Omit it to keep cache planning disabled. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "adaptive" +enabled = true + +[components.config] +version = 1 +agent_id = "planner" + +[components.config.state.backend] +kind = "in_memory" + +[components.config.telemetry] +subscriber_name = "adaptive.telemetry" +learners = ["acg"] + +[components.config.acg] +provider = "anthropic" +observation_window = 100 +priority = 50 + +[components.config.acg.stability_thresholds] +stable_threshold = 0.95 +semi_stable_threshold = 0.50 +min_observations_for_full_confidence = 20 +``` + +This configuration enables adaptive telemetry and configures ACG to plan cache +breakpoints for Anthropic-style request surfaces after it has enough observed +prompt samples. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the +Adaptive Cache Governor (ACG) runtime lifecycle. + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig(learners=["acg"]), + acg=nemo_flow.adaptive.AcgConfig(provider="anthropic"), +) + +plugin_config = nemo_flow.plugin.PluginConfig( + components=[nemo_flow.adaptive.ComponentSpec(adaptive_config)] +) + +report = nemo_flow.plugin.validate(plugin_config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await nemo_flow.plugin.initialize(plugin_config) +try: + # Run instrumented application work here. + pass +finally: + nemo_flow.plugin.clear() +``` +::: + +:::{tab-item} Node.js +:sync: node + +```js +const adaptive = require("nemo-flow-node/adaptive"); +const plugin = require("nemo-flow-node/plugin"); + +const adaptiveConfig = adaptive.defaultConfig(); +adaptiveConfig.agent_id = "planner"; +adaptiveConfig.state = { backend: adaptive.inMemoryBackend() }; +adaptiveConfig.telemetry = adaptive.telemetryConfig({ learners: ["acg"] }); +adaptiveConfig.acg = adaptive.acgConfig({ provider: "anthropic" }); + +const pluginConfig = plugin.defaultConfig(); +pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)]; + +const report = plugin.validate(pluginConfig); +if (report.diagnostics.some((diagnostic) => diagnostic.level === "error")) { + throw new Error(JSON.stringify(report.diagnostics)); +} + +await plugin.initialize(pluginConfig); +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; +use nemo_flow_adaptive::plugin_component::ComponentSpec; +use nemo_flow_adaptive::{ + AcgComponentConfig, AdaptiveConfig, BackendSpec, StateConfig, TelemetryComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + learners: vec!["acg".into()], + ..TelemetryComponentConfig::default() +}); +adaptive.acg = Some(AcgComponentConfig { + provider: "anthropic".into(), + ..AcgComponentConfig::default() +}); + +let mut plugin_config = PluginConfig::default(); +plugin_config.components.push(ComponentSpec::new(adaptive).into()); + +let report = validate_plugin_config(&plugin_config); +assert!(!report.has_errors()); + +let active = initialize_plugins(plugin_config).await?; +``` +::: + +:::: + +## Manual API + +Use the manual runtime API when an integration needs to own adaptive lifecycle +directly instead of activating the top-level plugin component. + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig(learners=["acg"]), + acg=nemo_flow.adaptive.AcgConfig(provider="anthropic"), +) + +runtime = nemo_flow.adaptive.AdaptiveRuntime(adaptive_config.to_dict()) +await runtime.register() +try: + # Run instrumented application work here. + runtime.wait_for_idle() +finally: + await runtime.shutdown() +``` +::: + +:::{tab-item} Node.js +:sync: node + +The Node.js binding exposes ACG through the adaptive plugin component helpers. +Use the Plugin Configuration example above when activating ACG from Node.js. +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow_adaptive::{ + AcgComponentConfig, AdaptiveConfig, AdaptiveRuntime, BackendSpec, StateConfig, + TelemetryComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + learners: vec!["acg".into()], + ..TelemetryComponentConfig::default() +}); +adaptive.acg = Some(AcgComponentConfig { + provider: "anthropic".into(), + ..AcgComponentConfig::default() +}); + +let mut runtime = AdaptiveRuntime::new(adaptive).await?; +runtime.register().await?; + +// Run instrumented application work here. + +runtime.wait_for_idle(); +runtime.shutdown().await?; +``` +::: + +:::: + +## Fields + +| Field | Default | Notes | +|---|---|---| +| `provider` | `passthrough` | `passthrough`, `anthropic`, or `openai`. | +| `observation_window` | `100` | Rolling Prompt IR sample window for stability analysis. | +| `priority` | `50` | LLM execution intercept priority. Lower values run earlier. | +| `stability_thresholds.stable_threshold` | `0.95` | Minimum effective score classified as stable. | +| `stability_thresholds.semi_stable_threshold` | `0.50` | Minimum effective score classified as semi-stable. | +| `stability_thresholds.min_observations_for_full_confidence` | `20` | Observation count required for full confidence. | + +Use `passthrough` when you want ACG observations without provider-specific hint +translation. Set `provider` to the backend API surface the agent actually calls +when you are ready to apply cache planning. + +## Expected Output + +When ACG is active, instrumented LLM calls still return the same application +result. ACG records observations and, when enough stable prompt structure is +available, emits adaptive diagnostics and cache-planning decisions through the +adaptive runtime. + +Provider-specific cache hints are useful only when the request surface supports +them. Validate against representative LLM traffic before enabling ACG in +production. + +## Common Validation Failures + +- `provider` is not one of `passthrough`, `anthropic`, or `openai`. +- Stability thresholds are outside the supported numeric range. +- ACG is enabled before the application emits managed LLM events. +- The configured provider does not match the real model API surface. diff --git a/docs/plugins/adaptive/adaptive-hints.md b/docs/plugins/adaptive/adaptive-hints.md new file mode 100644 index 00000000..64fe9044 --- /dev/null +++ b/docs/plugins/adaptive/adaptive-hints.md @@ -0,0 +1,259 @@ + + +# Adaptive Hints + +Use Adaptive Hints when downstream model calls or provider adapters can safely +receive guidance metadata from the adaptive runtime. + +Adaptive hints register as LLM request intercepts. Lower numeric priority values +run earlier in the intercept chain. The default priority is chosen relative to +other middleware rather than as a standalone importance score. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "adaptive" +enabled = true + +[components.config] +version = 1 +agent_id = "planner" + +[components.config.state.backend] +kind = "in_memory" + +[components.config.telemetry] +subscriber_name = "adaptive.telemetry" +learners = ["tool_parallelism"] + +[components.config.adaptive_hints] +priority = 100 +break_chain = false +inject_header = true +inject_body_path = "nvext.agent_hints" +``` + +This configuration injects adaptive guidance into outgoing model requests while +allowing later request intercepts to continue running. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the +Adaptive Hints request-intercept lifecycle. + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig(learners=["tool_parallelism"]), + adaptive_hints=nemo_flow.adaptive.AdaptiveHintsConfig( + inject_body_path="nvext.agent_hints", + ), +) + +plugin_config = nemo_flow.plugin.PluginConfig( + components=[nemo_flow.adaptive.ComponentSpec(adaptive_config)] +) + +report = nemo_flow.plugin.validate(plugin_config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await nemo_flow.plugin.initialize(plugin_config) +try: + # Run instrumented application work here. + pass +finally: + nemo_flow.plugin.clear() +``` +::: + +:::{tab-item} Node.js +:sync: node + +```js +const adaptive = require("nemo-flow-node/adaptive"); +const plugin = require("nemo-flow-node/plugin"); + +const adaptiveConfig = adaptive.defaultConfig(); +adaptiveConfig.agent_id = "planner"; +adaptiveConfig.state = { backend: adaptive.inMemoryBackend() }; +adaptiveConfig.telemetry = adaptive.telemetryConfig({ learners: ["tool_parallelism"] }); +adaptiveConfig.adaptive_hints = adaptive.adaptiveHintsConfig({ + inject_body_path: "nvext.agent_hints", +}); + +const pluginConfig = plugin.defaultConfig(); +pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)]; + +const report = plugin.validate(pluginConfig); +if (report.diagnostics.some((diagnostic) => diagnostic.level === "error")) { + throw new Error(JSON.stringify(report.diagnostics)); +} + +await plugin.initialize(pluginConfig); +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; +use nemo_flow_adaptive::plugin_component::ComponentSpec; +use nemo_flow_adaptive::{ + AdaptiveConfig, AdaptiveHintsComponentConfig, BackendSpec, StateConfig, TelemetryComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + learners: vec!["tool_parallelism".into()], + ..TelemetryComponentConfig::default() +}); +adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig { + inject_body_path: "nvext.agent_hints".into(), + ..AdaptiveHintsComponentConfig::default() +}); + +let mut plugin_config = PluginConfig::default(); +plugin_config.components.push(ComponentSpec::new(adaptive).into()); + +let report = validate_plugin_config(&plugin_config); +assert!(!report.has_errors()); + +let active = initialize_plugins(plugin_config).await?; +``` +::: + +:::: + +## Manual API + +Use the manual runtime API when an integration needs to own adaptive lifecycle +directly instead of activating the top-level plugin component. + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig(learners=["tool_parallelism"]), + adaptive_hints=nemo_flow.adaptive.AdaptiveHintsConfig( + inject_body_path="nvext.agent_hints", + ), +) + +runtime = nemo_flow.adaptive.AdaptiveRuntime(adaptive_config.to_dict()) +await runtime.register() +try: + # Run instrumented application work here. + nemo_flow.adaptive.set_latency_sensitivity(8) +finally: + await runtime.shutdown() +``` +::: + +:::{tab-item} Node.js +:sync: node + +The Node.js binding exposes Adaptive Hints through the adaptive plugin component +helpers. Use the Plugin Configuration example above when activating Adaptive +Hints from Node.js. +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow_adaptive::{ + set_latency_sensitivity, AdaptiveConfig, AdaptiveHintsComponentConfig, AdaptiveRuntime, + BackendSpec, StateConfig, TelemetryComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + learners: vec!["tool_parallelism".into()], + ..TelemetryComponentConfig::default() +}); +adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig { + inject_body_path: "nvext.agent_hints".into(), + ..AdaptiveHintsComponentConfig::default() +}); + +let mut runtime = AdaptiveRuntime::new(adaptive).await?; +runtime.register().await?; + +// Run instrumented application work here. +set_latency_sensitivity(8).ok(); + +runtime.shutdown().await?; +``` +::: + +:::: + +## Fields + +| Field | Default | Notes | +|---|---|---| +| `priority` | `100` | Request intercept priority. Lower values run earlier. | +| `break_chain` | `false` | Whether this intercept stops later request intercepts. | +| `inject_header` | `true` | Whether to add adaptive hints as request header metadata. | +| `inject_body_path` | `nvext.agent_hints` | JSON body path for request-body hint injection. | + +Disable `break_chain` unless the adaptive hint should be the final request +transform. Adjust `priority` only when adaptive hints need to run before or +after known application middleware. + +## Expected Output + +Outgoing managed LLM requests receive adaptive hint metadata in the configured +header and body location. The hints do not replace the application callback or +change the returned value by themselves. Downstream code must explicitly +interpret the metadata before behavior changes. + +## Common Validation Failures + +- Unknown adaptive hint fields when unknown fields are treated as errors. +- `inject_body_path` does not match the request shape expected by downstream + provider adapters. +- Hint injection is enabled before downstream model paths can consume or ignore + the metadata safely. diff --git a/docs/plugins/adaptive/configuration.md b/docs/plugins/adaptive/configuration.md new file mode 100644 index 00000000..69491a51 --- /dev/null +++ b/docs/plugins/adaptive/configuration.md @@ -0,0 +1,324 @@ + + +# Adaptive Configuration + +Use this page when you want to configure the built-in Adaptive plugin component +as a whole. The component kind is `adaptive`. + +Adaptive plugin configuration uses the generic NeMo Flow plugin document shape. +Field names stay `snake_case` in every binding and in `plugins.toml`, even when +language helper functions use language-native naming conventions. + +For plugin file discovery, precedence, merge behavior, editor controls, and +gateway conflict rules, see +[Plugin Configuration Files](../../build-plugins/plugin-configuration-files.md). + +## Component Shape + +The top-level adaptive object contains: + +| Field | Purpose | +|---|---| +| `version` | Adaptive config schema version. Defaults to `1`. | +| `agent_id` | Stable logical agent or workflow identifier for learned state. | +| `state` | Adaptive state backend. | +| `telemetry` | Adaptive subscriber and learner settings. | +| `adaptive_hints` | Request hint-injection behavior. | +| `tool_parallelism` | Tool scheduling observation or scheduling behavior. | +| `acg` | Adaptive Cache Governor prompt-cache planning. | +| `policy` | Adaptive-local handling for unknown fields and unsupported values. | + +The requested area pages cover [Adaptive Cache Governor (ACG)](acg.md) and +[Adaptive Hints](adaptive-hints.md). State, telemetry, tool parallelism, and +policy remain whole-plugin settings: + +- Use `state.backend.kind = "in_memory"` for local experiments. +- Use Redis state when learned state must survive restarts or be shared across + workers. +- Enable `telemetry` when adaptive learners should consume runtime events. +- Keep `tool_parallelism.mode = "observe_only"` until scheduling behavior has + been validated. +- Keep `policy.unsupported_value = "error"` for rollout safety. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "adaptive" +enabled = true + +[components.config] +version = 1 +agent_id = "planner" + +[components.config.state.backend] +kind = "in_memory" + +[components.config.telemetry] +subscriber_name = "adaptive.telemetry" +learners = ["tool_parallelism"] + +[components.config.tool_parallelism] +mode = "observe_only" +priority = 100 + +[components.config.adaptive_hints] +priority = 100 +break_chain = false +inject_header = true +inject_body_path = "nvext.agent_hints" + +[components.config.acg] +provider = "passthrough" +observation_window = 100 +priority = 50 + +[components.config.acg.stability_thresholds] +stable_threshold = 0.95 +semi_stable_threshold = 0.50 +min_observations_for_full_confidence = 20 + +[components.config.policy] +unknown_component = "warn" +unknown_field = "warn" +unsupported_value = "error" +``` + +This configuration activates adaptive telemetry, keeps tool parallelism +observational, injects adaptive hints, and leaves ACG in `passthrough` mode so +requests can be observed without provider-specific cache translation. + +## Per-Language Plugin Configuration + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig( + subscriber_name="adaptive.telemetry", + learners=["tool_parallelism"], + ), + tool_parallelism=nemo_flow.adaptive.ToolParallelismConfig(mode="observe_only"), + adaptive_hints=nemo_flow.adaptive.AdaptiveHintsConfig( + inject_body_path="nvext.agent_hints", + ), + acg=nemo_flow.adaptive.AcgConfig(provider="passthrough"), +) + +plugin_config = nemo_flow.plugin.PluginConfig( + components=[nemo_flow.adaptive.ComponentSpec(adaptive_config)] +) + +report = nemo_flow.plugin.validate(plugin_config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +active = await nemo_flow.plugin.initialize(plugin_config) +``` +::: + +:::{tab-item} Node.js +:sync: node + +```js +const adaptive = require("nemo-flow-node/adaptive"); +const plugin = require("nemo-flow-node/plugin"); + +const adaptiveConfig = adaptive.defaultConfig(); +adaptiveConfig.agent_id = "planner"; +adaptiveConfig.state = { backend: adaptive.inMemoryBackend() }; +adaptiveConfig.telemetry = adaptive.telemetryConfig({ + subscriber_name: "adaptive.telemetry", + learners: ["tool_parallelism"], +}); +adaptiveConfig.tool_parallelism = adaptive.toolParallelismConfig({ mode: "observe_only" }); +adaptiveConfig.adaptive_hints = adaptive.adaptiveHintsConfig({ + inject_body_path: "nvext.agent_hints", +}); +adaptiveConfig.acg = adaptive.acgConfig({ provider: "passthrough" }); + +const pluginConfig = plugin.defaultConfig(); +pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)]; + +const report = plugin.validate(pluginConfig); +if (report.diagnostics.some((diagnostic) => diagnostic.level === "error")) { + throw new Error(JSON.stringify(report.diagnostics)); +} + +const active = await plugin.initialize(pluginConfig); +``` +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; +use nemo_flow_adaptive::plugin_component::ComponentSpec; +use nemo_flow_adaptive::{ + AdaptiveConfig, + BackendSpec, + StateConfig, + TelemetryComponentConfig, + ToolParallelismComponentConfig, + AdaptiveHintsComponentConfig, + AcgComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + subscriber_name: Some("adaptive.telemetry".into()), + learners: vec!["tool_parallelism".into()], +}); +adaptive.tool_parallelism = Some(ToolParallelismComponentConfig::default()); +adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig { + inject_body_path: "nvext.agent_hints".into(), + ..AdaptiveHintsComponentConfig::default() +}); +adaptive.acg = Some(AcgComponentConfig { + provider: "passthrough".into(), + ..AcgComponentConfig::default() +}); + +let mut plugin_config = PluginConfig::default(); +plugin_config.components.push(ComponentSpec::new(adaptive).into()); + +let report = validate_plugin_config(&plugin_config); +assert!(!report.has_errors()); + +let active = initialize_plugins(plugin_config).await?; +``` +::: + +:::: + +## Manual API + +Use the manual runtime API when an integration needs to own adaptive lifecycle +directly instead of activating the top-level plugin component. + +::::{tab-set} +:sync-group: language + +:::{tab-item} Python +:sync: python + +```python +import nemo_flow + +adaptive_config = nemo_flow.adaptive.AdaptiveConfig( + agent_id="planner", + state=nemo_flow.adaptive.StateConfig( + backend=nemo_flow.adaptive.BackendSpec.in_memory(), + ), + telemetry=nemo_flow.adaptive.TelemetryConfig( + subscriber_name="adaptive.telemetry", + learners=["tool_parallelism"], + ), + tool_parallelism=nemo_flow.adaptive.ToolParallelismConfig(mode="observe_only"), + adaptive_hints=nemo_flow.adaptive.AdaptiveHintsConfig( + inject_body_path="nvext.agent_hints", + ), + acg=nemo_flow.adaptive.AcgConfig(provider="passthrough"), +) + +runtime = nemo_flow.adaptive.AdaptiveRuntime(adaptive_config.to_dict()) +await runtime.register() +try: + # Run instrumented application work here. + runtime.wait_for_idle() +finally: + await runtime.shutdown() +``` +::: + +:::{tab-item} Node.js +:sync: node + +The Node.js binding exposes the built-in adaptive runtime through the adaptive +plugin component helpers. Use the Plugin Configuration example above when +activating adaptive behavior from Node.js. +::: + +:::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow_adaptive::{ + AcgComponentConfig, AdaptiveConfig, AdaptiveHintsComponentConfig, AdaptiveRuntime, + BackendSpec, StateConfig, TelemetryComponentConfig, ToolParallelismComponentConfig, +}; + +let mut adaptive = AdaptiveConfig::default(); +adaptive.agent_id = Some("planner".into()); +adaptive.state = Some(StateConfig { + backend: BackendSpec::in_memory(), +}); +adaptive.telemetry = Some(TelemetryComponentConfig { + subscriber_name: Some("adaptive.telemetry".into()), + learners: vec!["tool_parallelism".into()], +}); +adaptive.tool_parallelism = Some(ToolParallelismComponentConfig::default()); +adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig { + inject_body_path: "nvext.agent_hints".into(), + ..AdaptiveHintsComponentConfig::default() +}); +adaptive.acg = Some(AcgComponentConfig { + provider: "passthrough".into(), + ..AcgComponentConfig::default() +}); + +let mut runtime = AdaptiveRuntime::new(adaptive).await?; +runtime.register().await?; + +// Run instrumented application work here. + +runtime.wait_for_idle(); +runtime.shutdown().await?; +``` +::: + +:::: + +## Validation And Teardown + +Validate plugin configuration before initialization. Disabled adaptive +components are still validated, which lets operators prepare a rollout before +setting `enabled = true`. + +Common validation failures include: + +- Unknown adaptive fields when policy treats unknown fields as errors. +- Unsupported backend kinds, tool-parallelism modes, or ACG providers. +- Unsupported schema versions. +- Backend-specific fields that do not match the selected backend. + +Clear plugin configuration during shutdown or test cleanup. Clearing the plugin +configuration deregisters adaptive subscribers and intercepts owned by the +plugin runtime. + +## Rollout Guidance + +Start by enabling state and telemetry in a development environment. Run +representative instrumented workflows, inspect emitted events and adaptive +reports, and then enable active behavior one area at a time. Keep rollback as a +configuration change. diff --git a/docs/plugins/observability/about.md b/docs/plugins/observability/about.md new file mode 100644 index 00000000..2bbbd659 --- /dev/null +++ b/docs/plugins/observability/about.md @@ -0,0 +1,71 @@ + + +# Observability + +Use the Observability plugin when you need to inspect NeMo Flow lifecycle events +in process or export agent activity to tracing, trajectory, or analysis +systems from one plugin configuration document. + +Observability in NeMo Flow starts with events. Scopes, marks, managed tool +calls, managed LLM calls, middleware, and manual lifecycle APIs emit the +canonical Agent Trajectory Observability Format (ATOF) event stream. +Subscribers consume that stream in process, and exporter-oriented subscribers +write raw ATOF JSONL or translate events into Agent Trajectory Interchange +Format (ATIF), OpenTelemetry, or OpenInference. + +The first-party plugin component has kind `observability`. It can install: + +- Agent Trajectory Observability Format (ATOF) JSONL export for raw lifecycle events. +- Agent Trajectory Interchange Format (ATIF) trajectory export for each top-level agent scope. +- OpenTelemetry OTLP trace export. +- OpenInference-oriented OTLP trace export. + +## Plugin-Managed Versus Manual Export + +Use the Observability plugin for process-level exporter setup that should be +activated from config, `plugins.toml`, or a shared plugin document. + +Use manual subscriber or exporter APIs when a test, script, or application +needs direct control over registration names, collection windows, explicit +flush timing, or per-run exporter objects. The plugin owns subscriber names and +teardown for the sections it enables. + +## Use Observability When + +Start here when you need to: + +- Verify that instrumentation is attached to the right scope. +- Inspect tool and LLM inputs and outputs after sanitization. +- Correlate concurrent agent runs by root scope. +- Export traces to OTLP-compatible infrastructure. +- Produce trajectory data for analysis, replay, or evaluation workflows. + +If you have not instrumented scopes, tools, or LLM calls yet, start with +[Instrument Applications](../../instrument-applications/about.md). + +## Exporter Selection + +Choose the exporter based on the downstream system: + +| Need | Use | +|---|---| +| Raw canonical event stream | [Agent Trajectory Observability Format (ATOF)](atof.md) | +| Offline analysis, replay, or evaluation trajectories | [Agent Trajectory Interchange Format (ATIF)](atif.md) | +| Generic OTLP traces | [OpenTelemetry](opentelemetry.md) | +| OpenInference-oriented agent and LLM spans | [OpenInference](openinference.md) | + +Start with local event inspection before production export. Add sanitize +guardrails before exporters receive sensitive payloads. + +## Pages + +- [Observability Configuration](configuration.md) documents the whole plugin + component shape, activation, validation, and teardown. +- [Agent Trajectory Observability Format (ATOF)](atof.md) covers raw JSONL event stream export. +- [Agent Trajectory Interchange Format (ATIF)](atif.md) covers per-agent trajectory export. +- [OpenTelemetry](opentelemetry.md) covers generic OTLP trace export. +- [OpenInference](openinference.md) covers OpenInference-oriented OTLP trace + export. diff --git a/docs/plugins/observability/atif.md b/docs/plugins/observability/atif.md new file mode 100644 index 00000000..c9bad9f0 --- /dev/null +++ b/docs/plugins/observability/atif.md @@ -0,0 +1,267 @@ + + +# Agent Trajectory Interchange Format (ATIF) + +Use the `atif` section when you want one Agent Trajectory Interchange Format +(ATIF) trajectory artifact per top-level agent run. + +The plugin-managed ATIF dispatcher watches for direct child scopes with category +`agent`, creates a scope-local exporter for each one, and writes the trajectory +when that agent scope ends. Nested agent scopes remain in the parent +trajectory. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "observability" +enabled = true + +[components.config] +version = 1 + +[components.config.atif] +enabled = true +agent_name = "Planner" +agent_version = "1.0.0" +model_name = "unknown" +output_directory = "logs" +filename_template = "trajectory-{session_id}.json" +``` + +This configuration writes a trajectory file such as +`logs/trajectory-.json` for each top-level agent scope. + +## Fields + +| Field | Default | Notes | +|---|---|---| +| `enabled` | `false` | Must be `true` to write trajectories. | +| `agent_name` | `NeMo Flow` | Agent metadata written into the trajectory. | +| `agent_version` | NeMo Flow crate version | Agent version metadata. | +| `model_name` | `unknown` | Default model metadata when no call-level model is present. | +| `tool_definitions` | Omitted | Optional ATIF tool metadata. | +| `extra` | Omitted | Optional ATIF agent metadata. | +| `output_directory` | Current working directory | Directory containing trajectory files. | +| `filename_template` | `nemo-flow-atif-{session_id}.json` | Must contain `{session_id}`. | + +## Expected Output + +The exporter translates NeMo Flow lifecycle events into ATIF v1.6 trajectory +data. LLM start and end events become model steps, tool events become tool +calls and observations, and scope nesting contributes lineage metadata. + +The plugin writes each trajectory when the top-level agent scope closes. If the +plugin is cleared while an agent is still open, teardown flushes the partial +trajectory. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the ATIF +dispatcher lifecycle. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import plugin +from nemo_flow.observability import AtifConfig, ComponentSpec, ObservabilityConfig + +config = plugin.PluginConfig( + components=[ + ComponentSpec( + ObservabilityConfig( + atif=AtifConfig( + enabled=True, + agent_name="Planner", + agent_version="1.0.0", + model_name="unknown", + output_directory="logs", + filename_template="trajectory-{session_id}.json", + ) + ) + ) + ] +) + +report = plugin.validate(config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await plugin.initialize(config) +try: + # Run instrumented application work here. + pass +finally: + plugin.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const plugin = require("nemo-flow-node/plugin"); +const observability = require("nemo-flow-node/observability"); + +await plugin.initialize({ + version: 1, + components: [ + observability.ComponentSpec({ + version: 1, + atif: observability.atifConfig({ + enabled: true, + agent_name: "Planner", + agent_version: "1.0.0", + model_name: "unknown", + output_directory: "logs", + filename_template: "trajectory-{session_id}.json", + }), + }), + ], +}); + +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::plugin_component::{ + AtifSectionConfig, ComponentSpec, ObservabilityConfig, +}; +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; + +let component = ComponentSpec::new(ObservabilityConfig { + atif: Some(AtifSectionConfig { + enabled: true, + agent_name: "Planner".into(), + agent_version: "1.0.0".into(), + model_name: "unknown".into(), + output_directory: Some("logs".into()), + filename_template: "trajectory-{session_id}.json".into(), + ..AtifSectionConfig::default() + }), + ..ObservabilityConfig::default() +}); + +let config = PluginConfig { + version: 1, + components: vec![component.into()], + policy: Default::default(), +}; + +let report = validate_plugin_config(&config); +assert!(!report.has_errors()); + +let active = initialize_plugins(config).await?; +``` + +:::: + +::::: + +## Manual API + +Use the manual `AtifExporter` API when you need explicit collection boundaries +or one exporter object per run. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import AtifExporter + +exporter = AtifExporter("session-1", "agent", "1.0.0", model_name="demo-model") +exporter.register("atif-exporter") + +# Run instrumented application work here. + +trajectory = exporter.export() +exporter.deregister("atif-exporter") +exporter.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const { AtifExporter } = require("nemo-flow-node"); + +const exporter = new AtifExporter("session-1", "agent", "1.0.0", "demo-model"); +exporter.register("atif-exporter"); + +try { + // Run instrumented application work here. + + const trajectoryJson = exporter.exportJson(); + console.log(trajectoryJson); +} finally { + exporter.deregister("atif-exporter"); + exporter.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::api::subscriber::{deregister_subscriber, register_subscriber}; +use nemo_flow::observability::atif::{AtifAgentInfo, AtifExporter}; + +let exporter = AtifExporter::new( + "session-1".to_string(), + AtifAgentInfo { + name: "agent".to_string(), + version: "1.0.0".to_string(), + model_name: Some("demo-model".to_string()), + tool_definitions: None, + extra: None, + }, +); +register_subscriber("atif-exporter", exporter.subscriber())?; + +// Run instrumented application work here. + +let trajectory = exporter.export(); +let trajectory_json = serde_json::to_string_pretty(&trajectory)?; +println!("{trajectory_json}"); + +let _ = deregister_subscriber("atif-exporter")?; +exporter.clear(); +``` + +:::: + +::::: + +## Common Validation Failures + +- `filename_template` does not contain `{session_id}`. +- The output directory is not writable at runtime. +- Tool definitions or `extra` metadata are not JSON-compatible. +- The application never opens a top-level `agent` scope, so no trajectory file + is created. diff --git a/docs/plugins/observability/atof.md b/docs/plugins/observability/atof.md new file mode 100644 index 00000000..9e9f8c18 --- /dev/null +++ b/docs/plugins/observability/atof.md @@ -0,0 +1,251 @@ + + +# Agent Trajectory Observability Format (ATOF) + +Use the `atof` section when you want the raw Agent Trajectory Observability +Format (ATOF) `0.1` event stream written as JSONL. + +ATOF JSONL export is useful for local debugging, offline inspection, and +preserving the canonical event stream before it is translated into another +format. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "observability" +enabled = true + +[components.config] +version = 1 + +[components.config.atof] +enabled = true +output_directory = "logs" +filename = "events.jsonl" +mode = "overwrite" +``` + +This configuration registers the plugin-managed ATOF exporter and writes one +JSON object per lifecycle event to `logs/events.jsonl`. + +## Fields + +| Field | Default | Notes | +|---|---|---| +| `enabled` | `false` | Must be `true` to write events. | +| `output_directory` | Current working directory | Directory containing the JSONL file. | +| `filename` | Timestamped `nemo-flow-events-*.jsonl` | Explicit output filename. | +| `mode` | `append` | `append` or `overwrite`. | + +## Expected Output + +Each emitted scope, tool, LLM, middleware, or mark event is written as one ATOF +JSON object per line. For event field semantics, see +[Events](../../about/concepts/events.md). + +Register the plugin before instrumented work starts and clear it during +shutdown so file handles flush. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the ATOF +exporter lifecycle. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import plugin +from nemo_flow.observability import AtofConfig, ComponentSpec, ObservabilityConfig + +config = plugin.PluginConfig( + components=[ + ComponentSpec( + ObservabilityConfig( + atof=AtofConfig( + enabled=True, + output_directory="logs", + filename="events.jsonl", + mode="overwrite", + ) + ) + ) + ] +) + +report = plugin.validate(config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await plugin.initialize(config) +try: + # Run instrumented application work here. + pass +finally: + plugin.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const plugin = require("nemo-flow-node/plugin"); +const observability = require("nemo-flow-node/observability"); + +await plugin.initialize({ + version: 1, + components: [ + observability.ComponentSpec({ + version: 1, + atof: observability.atofConfig({ + enabled: true, + output_directory: "logs", + filename: "events.jsonl", + mode: "overwrite", + }), + }), + ], +}); + +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::plugin_component::{ + AtofSectionConfig, ComponentSpec, ObservabilityConfig, +}; +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; + +let component = ComponentSpec::new(ObservabilityConfig { + atof: Some(AtofSectionConfig { + enabled: true, + output_directory: Some("logs".into()), + filename: Some("events.jsonl".into()), + mode: "overwrite".into(), + }), + ..ObservabilityConfig::default() +}); + +let config = PluginConfig { + version: 1, + components: vec![component.into()], + policy: Default::default(), +}; + +let report = validate_plugin_config(&config); +assert!(!report.has_errors()); + +let active = initialize_plugins(config).await?; +``` + +:::: + +::::: + +## Manual API + +Use the manual `AtofExporter` API when a test or script needs a custom +subscriber name or explicit registration window. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import AtofExporter, AtofExporterConfig, AtofExporterMode + +config = AtofExporterConfig() +config.output_directory = "logs" +config.filename = "events.jsonl" +config.mode = AtofExporterMode.Overwrite + +exporter = AtofExporter(config) +exporter.register("atof-exporter") + +# Run instrumented application work here. + +exporter.force_flush() +exporter.deregister("atof-exporter") +exporter.shutdown() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const { AtofExporter } = require("nemo-flow-node"); + +const exporter = new AtofExporter({ + outputDirectory: "logs", + filename: "events.jsonl", + mode: "overwrite", +}); +exporter.register("atof-exporter"); + +try { + // Run instrumented application work here. + + exporter.forceFlush(); +} finally { + exporter.deregister("atof-exporter"); + exporter.shutdown(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::atof::{ + AtofExporter, AtofExporterConfig, AtofExporterMode, +}; + +let config = AtofExporterConfig::new() + .with_output_directory("logs") + .with_filename("events.jsonl") + .with_mode(AtofExporterMode::Overwrite); +let exporter = AtofExporter::new(config)?; +exporter.register("atof-exporter")?; + +// Run instrumented application work here. + +exporter.force_flush()?; +let _ = exporter.deregister("atof-exporter")?; +exporter.shutdown()?; +``` + +:::: + +::::: + +## Common Validation Failures + +- `mode` is not `append` or `overwrite`. +- The output directory is not writable at runtime. +- ATOF is enabled in a target that cannot access the native filesystem. diff --git a/docs/plugins/observability/configuration.md b/docs/plugins/observability/configuration.md new file mode 100644 index 00000000..b8ec96ed --- /dev/null +++ b/docs/plugins/observability/configuration.md @@ -0,0 +1,319 @@ + + +# Observability Configuration + +Use this page when an application should install standard observability +exporters from one plugin configuration document instead of manually registering +each subscriber. + +The plugin kind is `observability`. It is registered by the core runtime, so +applications do not need to register a plugin implementation before validation +or initialization. + +For plugin file discovery, precedence, merge behavior, editor controls, and +gateway conflict rules, see +[Plugin Configuration Files](../../build-plugins/plugin-configuration-files.md). + +:::{note} +Observability plugin configuration uses the generic NeMo Flow plugin document +shape, so field names are `snake_case` in every binding. This differs from +Node.js runtime classes such as `OpenTelemetrySubscriber`, which use +Node-native `camelCase` option names outside the plugin system. +::: + +## What It Installs + +Every exporter section is optional and defaults to disabled. A section is active +only when it includes `enabled: true`. + +| Section | Runtime behavior | +|---|---| +| `atof` | Registers a global Agent Trajectory Observability Format (ATOF) JSONL exporter for raw lifecycle events. | +| `atif` | Registers one Agent Trajectory Interchange Format (ATIF) dispatcher that writes one trajectory file for each top-level agent scope. | +| `opentelemetry` | Registers a global OpenTelemetry OTLP subscriber. | +| `openinference` | Registers a global OpenInference OTLP subscriber. | + +`subscriber_name` is not part of this config. The runtime infers +component-local subscriber names and registers them under the observability +plugin namespace: + +- Agent Trajectory Observability Format (ATOF): `__nemo_flow_plugin__observability__atof` +- Agent Trajectory Interchange Format (ATIF) dispatcher: `__nemo_flow_plugin__observability__atif` +- Per-agent ATIF scope subscriber: `__nemo_flow_plugin__observability__atif-{agent_scope_uuid}` +- OpenTelemetry: `__nemo_flow_plugin__observability__opentelemetry` +- OpenInference: `__nemo_flow_plugin__observability__openinference` + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "observability" +enabled = true + +[components.config] +version = 1 + +[components.config.atof] +enabled = true +output_directory = "logs" +filename = "events.jsonl" +mode = "overwrite" + +[components.config.atif] +enabled = true +output_directory = "logs" +filename_template = "trajectory-{session_id}.json" + +[components.config.opentelemetry] +enabled = true +transport = "http_binary" +endpoint = "http://localhost:4318/v1/traces" +service_name = "nemo-flow" +service_namespace = "agent" +service_version = "0.2.0" +instrumentation_scope = "nemo-flow-observability" +timeout_millis = 3000 + +[components.config.opentelemetry.headers] +authorization = "Bearer " + +[components.config.opentelemetry.resource_attributes] +"deployment.environment" = "dev" +"service.instance.id" = "local" + +[components.config.openinference] +enabled = true +transport = "http_binary" +endpoint = "http://localhost:6006/v1/traces" +service_name = "nemo-flow" +service_namespace = "agent" +service_version = "0.2.0" +instrumentation_scope = "nemo-flow-openinference" +timeout_millis = 3000 + +[components.config.openinference.headers] +authorization = "Bearer " + +[components.config.openinference.resource_attributes] +"deployment.environment" = "dev" +"service.instance.id" = "local" + +[components.config.policy] +unknown_component = "warn" +unknown_field = "warn" +unsupported_value = "error" +``` + +Include only the sections you want to configure. In layered `plugins.toml` +files, omission inherits lower-precedence values; write `enabled = false` to +disable an inherited section. + +## Per-Language Plugin Configuration + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import plugin, scope, ScopeType +from nemo_flow.observability import ( + AtifConfig, + AtofConfig, + ComponentSpec, + ObservabilityConfig, + OtlpConfig, +) + +config = plugin.PluginConfig( + components=[ + ComponentSpec( + ObservabilityConfig( + atof=AtofConfig( + enabled=True, + output_directory="logs", + filename="events.jsonl", + mode="overwrite", + ), + atif=AtifConfig( + enabled=True, + output_directory="logs", + filename_template="trajectory-{session_id}.json", + ), + opentelemetry=OtlpConfig( + enabled=True, + endpoint="http://localhost:4318/v1/traces", + service_name="nemo-flow", + service_namespace="agent", + service_version="0.2.0", + instrumentation_scope="nemo-flow-observability", + resource_attributes={"deployment.environment": "dev"}, + ), + openinference=OtlpConfig( + enabled=True, + endpoint="http://localhost:6006/v1/traces", + service_name="nemo-flow", + service_namespace="agent", + service_version="0.2.0", + instrumentation_scope="nemo-flow-openinference", + resource_attributes={"deployment.environment": "dev"}, + ), + ) + ) + ] +) + +report = plugin.validate(config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await plugin.initialize(config) +try: + with scope.scope("agent", ScopeType.Agent): + pass +finally: + plugin.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const plugin = require("nemo-flow-node/plugin"); +const observability = require("nemo-flow-node/observability"); + +await plugin.initialize({ + version: 1, + components: [ + observability.ComponentSpec({ + version: 1, + atof: observability.atofConfig({ + enabled: true, + output_directory: "logs", + filename: "events.jsonl", + mode: "overwrite", + }), + atif: observability.atifConfig({ + enabled: true, + output_directory: "logs", + filename_template: "trajectory-{session_id}.json", + }), + opentelemetry: observability.otlpConfig({ + enabled: true, + endpoint: "http://localhost:4318/v1/traces", + service_name: "nemo-flow", + service_namespace: "agent", + service_version: "0.2.0", + instrumentation_scope: "nemo-flow-observability", + resource_attributes: { + "deployment.environment": "dev", + }, + }), + openinference: observability.otlpConfig({ + enabled: true, + endpoint: "http://localhost:6006/v1/traces", + service_name: "nemo-flow", + service_namespace: "agent", + service_version: "0.2.0", + instrumentation_scope: "nemo-flow-openinference", + resource_attributes: { + "deployment.environment": "dev", + }, + }), + }), + ], +}); + +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::plugin_component::{ + AtifSectionConfig, AtofSectionConfig, ComponentSpec, ObservabilityConfig, + OtlpSectionConfig, +}; +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; + +let component = ComponentSpec::new(ObservabilityConfig { + atof: Some(AtofSectionConfig { + enabled: true, + output_directory: Some("logs".into()), + filename: Some("events.jsonl".into()), + mode: "overwrite".into(), + }), + atif: Some(AtifSectionConfig { + enabled: true, + output_directory: Some("logs".into()), + filename_template: "trajectory-{session_id}.json".into(), + ..AtifSectionConfig::default() + }), + opentelemetry: Some(OtlpSectionConfig { + enabled: true, + endpoint: Some("http://localhost:4318/v1/traces".into()), + service_name: "nemo-flow".into(), + service_namespace: Some("agent".into()), + service_version: Some("0.2.0".into()), + instrumentation_scope: Some("nemo-flow-observability".into()), + resource_attributes: [("deployment.environment".into(), "dev".into())].into(), + ..OtlpSectionConfig::default() + }), + openinference: Some(OtlpSectionConfig { + enabled: true, + endpoint: Some("http://localhost:6006/v1/traces".into()), + service_name: "nemo-flow".into(), + service_namespace: Some("agent".into()), + service_version: Some("0.2.0".into()), + instrumentation_scope: Some("nemo-flow-openinference".into()), + resource_attributes: [("deployment.environment".into(), "dev".into())].into(), + ..OtlpSectionConfig::default() + }), + ..ObservabilityConfig::default() +}); + +let config = PluginConfig { + version: 1, + components: vec![component.into()], + policy: Default::default(), +}; + +let report = validate_plugin_config(&config); +assert!(!report.has_errors()); + +let active = initialize_plugins(config).await?; +``` + +:::: + +::::: + +## Validation And Teardown + +Validate plugin configuration before activating it. The plugin reports +unsupported transports, unsupported ATOF modes, unsafe ATIF filename templates, +unknown fields according to policy, and enabled exporters that are unavailable +in the current build or target. + +Call `plugin.clear()` or `clear_plugin_configuration()` during teardown. +Clearing the plugin config deregisters inferred subscribers, flushes file +exporters, and shuts down owned OTLP subscribers. + +Use manual subscriber/exporter APIs instead of the plugin when you need custom +subscriber names, explicit per-run exporter objects, or direct control over the +collection window. diff --git a/docs/plugins/observability/openinference.md b/docs/plugins/observability/openinference.md new file mode 100644 index 00000000..0d574aaa --- /dev/null +++ b/docs/plugins/observability/openinference.md @@ -0,0 +1,297 @@ + + +# OpenInference + +Use the `openinference` section when you want NeMo Flow lifecycle events +exported as OTLP trace spans with OpenInference-oriented semantics. + +OpenInference export maps model-centric payloads directly into trace +attributes. Scope, tool, and LLM start inputs become `input.value`; end outputs +become `output.value`; LLM usage metadata maps to token-count attributes when +the provider response includes usage information. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "observability" +enabled = true + +[components.config] +version = 1 + +[components.config.openinference] +enabled = true +transport = "http_binary" +endpoint = "http://localhost:6006/v1/traces" +service_name = "agent-service" +service_namespace = "nemo" +service_version = "1.0.0" +instrumentation_scope = "nemo-flow-openinference" +timeout_millis = 3000 + +[components.config.openinference.headers] +authorization = "Bearer " + +[components.config.openinference.resource_attributes] +"deployment.environment" = "dev" +``` + +This configuration registers a plugin-owned OpenInference subscriber and sends +OpenInference-style OTLP spans to Phoenix or another compatible backend. + +## Fields + +OpenInference uses the same OTLP section shape as +[OpenTelemetry](opentelemetry.md): + +| Field | Default | Notes | +|---|---|---| +| `enabled` | `false` | Must be `true` to construct and register the subscriber. | +| `transport` | `http_binary` | `http_binary` or `grpc`. | +| `endpoint` | Exporter default | OTLP endpoint. | +| `headers` | `{}` | String-to-string exporter headers. | +| `resource_attributes` | `{}` | String-to-string OTLP resource attributes. | +| `service_name` | `nemo-flow` | `service.name` resource attribute. | +| `service_namespace` | Omitted | Optional `service.namespace`. | +| `service_version` | Omitted | Optional `service.version`. | +| `instrumentation_scope` | Omitted | Optional instrumentation scope name. | +| `timeout_millis` | `3000` | Export timeout. | + +## Expected Output + +The backend should show OpenInference-oriented spans for scopes, tools, and LLM +calls from the same `root_uuid`. LLM usage metadata appears as token counters +when provider responses include usage information. + +Redact sensitive event payloads with sanitize guardrails before production +export. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the +OpenInference subscriber lifecycle. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import plugin +from nemo_flow.observability import ComponentSpec, ObservabilityConfig, OtlpConfig + +config = plugin.PluginConfig( + components=[ + ComponentSpec( + ObservabilityConfig( + openinference=OtlpConfig( + enabled=True, + transport="http_binary", + endpoint="http://localhost:6006/v1/traces", + service_name="agent-service", + service_namespace="nemo", + service_version="1.0.0", + instrumentation_scope="nemo-flow-openinference", + resource_attributes={"deployment.environment": "dev"}, + headers={"authorization": "Bearer "}, + ) + ) + ) + ] +) + +report = plugin.validate(config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await plugin.initialize(config) +try: + # Run instrumented application work here. + pass +finally: + plugin.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const plugin = require("nemo-flow-node/plugin"); +const observability = require("nemo-flow-node/observability"); + +await plugin.initialize({ + version: 1, + components: [ + observability.ComponentSpec({ + version: 1, + openinference: observability.otlpConfig({ + enabled: true, + transport: "http_binary", + endpoint: "http://localhost:6006/v1/traces", + service_name: "agent-service", + service_namespace: "nemo", + service_version: "1.0.0", + instrumentation_scope: "nemo-flow-openinference", + resource_attributes: { + "deployment.environment": "dev", + }, + headers: { + authorization: "Bearer ", + }, + }), + }), + ], +}); + +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::plugin_component::{ + ComponentSpec, ObservabilityConfig, OtlpSectionConfig, +}; +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; + +let component = ComponentSpec::new(ObservabilityConfig { + openinference: Some(OtlpSectionConfig { + enabled: true, + transport: "http_binary".into(), + endpoint: Some("http://localhost:6006/v1/traces".into()), + service_name: "agent-service".into(), + service_namespace: Some("nemo".into()), + service_version: Some("1.0.0".into()), + instrumentation_scope: Some("nemo-flow-openinference".into()), + resource_attributes: [("deployment.environment".into(), "dev".into())].into(), + headers: [("authorization".into(), "Bearer ".into())].into(), + ..OtlpSectionConfig::default() + }), + ..ObservabilityConfig::default() +}); + +let config = PluginConfig { + version: 1, + components: vec![component.into()], + policy: Default::default(), +}; + +let report = validate_plugin_config(&config); +assert!(!report.has_errors()); + +let active = initialize_plugins(config).await?; +``` + +:::: + +::::: + +## Manual API + +Use the manual subscriber API when you need an explicit subscriber name or +direct `force_flush` control. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import OpenInferenceConfig, OpenInferenceSubscriber + +config = OpenInferenceConfig() +config.transport = "http_binary" +config.endpoint = "http://localhost:6006/v1/traces" +config.service_name = "agent-service" +config.set_resource_attribute("deployment.environment", "dev") + +subscriber = OpenInferenceSubscriber(config) +subscriber.register("openinference-exporter") + +# Run instrumented application work here. + +subscriber.force_flush() +subscriber.deregister("openinference-exporter") +subscriber.shutdown() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const { OpenInferenceSubscriber } = require("nemo-flow-node"); + +const subscriber = new OpenInferenceSubscriber({ + transport: "http_binary", + endpoint: "http://localhost:6006/v1/traces", + serviceName: "agent-service", + resourceAttributes: { + "deployment.environment": "dev", + }, +}); +subscriber.register("openinference-exporter"); + +try { + // Run instrumented application work here. + + subscriber.forceFlush(); +} finally { + subscriber.deregister("openinference-exporter"); + subscriber.shutdown(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::openinference::{ + OpenInferenceConfig, OpenInferenceSubscriber, +}; + +let config = OpenInferenceConfig::new() + .with_service_name("agent-service") + .with_endpoint("http://localhost:6006/v1/traces") + .with_resource_attribute("deployment.environment", "dev"); +let subscriber = OpenInferenceSubscriber::new(config)?; +subscriber.register("openinference-exporter")?; + +// Run instrumented application work here. + +subscriber.force_flush()?; +let _ = subscriber.deregister("openinference-exporter")?; +subscriber.shutdown()?; +``` + +:::: + +::::: + +## Common Validation Failures + +- `transport` is not `http_binary` or `grpc`. +- Headers or resource attributes are not string-to-string maps. +- The OpenInference feature is unavailable in the current build or target. +- Tool and LLM calls do not use managed helpers, so spans contain only scope + lifecycle data. diff --git a/docs/plugins/observability/opentelemetry.md b/docs/plugins/observability/opentelemetry.md new file mode 100644 index 00000000..97f948fc --- /dev/null +++ b/docs/plugins/observability/opentelemetry.md @@ -0,0 +1,290 @@ + + +# OpenTelemetry + +Use the `opentelemetry` section when you want NeMo Flow lifecycle events +exported as generic OpenTelemetry Protocol (OTLP) trace spans. + +OpenTelemetry export is a good fit when your tracing backend already expects +OTLP spans and you want NeMo Flow scopes, tool calls, LLM calls, and marks to +appear in the same tracing pipeline as the rest of the application. + +## `plugins.toml` Example + +```toml +version = 1 + +[[components]] +kind = "observability" +enabled = true + +[components.config] +version = 1 + +[components.config.opentelemetry] +enabled = true +transport = "http_binary" +endpoint = "http://localhost:4318/v1/traces" +service_name = "agent-service" +service_namespace = "nemo" +service_version = "1.0.0" +instrumentation_scope = "nemo-flow-otel" +timeout_millis = 3000 + +[components.config.opentelemetry.headers] +authorization = "Bearer " + +[components.config.opentelemetry.resource_attributes] +"deployment.environment" = "dev" +``` + +This configuration registers a plugin-owned OpenTelemetry subscriber and sends +NeMo Flow trace spans to the configured OTLP endpoint. + +## Fields + +| Field | Default | Notes | +|---|---|---| +| `enabled` | `false` | Must be `true` to construct and register the subscriber. | +| `transport` | `http_binary` | `http_binary` or `grpc`. | +| `endpoint` | Exporter default | OTLP endpoint. | +| `headers` | `{}` | String-to-string exporter headers. | +| `resource_attributes` | `{}` | String-to-string OTLP resource attributes. | +| `service_name` | `nemo-flow` | `service.name` resource attribute. | +| `service_namespace` | Omitted | Optional `service.namespace`. | +| `service_version` | Omitted | Optional `service.version`. | +| `instrumentation_scope` | Omitted | Optional instrumentation scope name. | +| `timeout_millis` | `3000` | Export timeout. | + +## Expected Output + +The collector should receive OTLP trace export requests. The tracing backend +should show spans for NeMo Flow scopes, tools, LLM calls, and marks grouped by +root scope. + +Register the plugin before the first instrumented request, use stable service +identity fields, keep credentials outside source code, and flush during +graceful shutdown. + +## Plugin Configuration + +Use plugin configuration when the application should let NeMo Flow own the +OpenTelemetry subscriber lifecycle. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import plugin +from nemo_flow.observability import ComponentSpec, ObservabilityConfig, OtlpConfig + +config = plugin.PluginConfig( + components=[ + ComponentSpec( + ObservabilityConfig( + opentelemetry=OtlpConfig( + enabled=True, + transport="http_binary", + endpoint="http://localhost:4318/v1/traces", + service_name="agent-service", + service_namespace="nemo", + service_version="1.0.0", + instrumentation_scope="nemo-flow-otel", + resource_attributes={"deployment.environment": "dev"}, + headers={"authorization": "Bearer "}, + ) + ) + ) + ] +) + +report = plugin.validate(config) +if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]): + raise RuntimeError(report["diagnostics"]) + +await plugin.initialize(config) +try: + # Run instrumented application work here. + pass +finally: + plugin.clear() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const plugin = require("nemo-flow-node/plugin"); +const observability = require("nemo-flow-node/observability"); + +await plugin.initialize({ + version: 1, + components: [ + observability.ComponentSpec({ + version: 1, + opentelemetry: observability.otlpConfig({ + enabled: true, + transport: "http_binary", + endpoint: "http://localhost:4318/v1/traces", + service_name: "agent-service", + service_namespace: "nemo", + service_version: "1.0.0", + instrumentation_scope: "nemo-flow-otel", + resource_attributes: { + "deployment.environment": "dev", + }, + headers: { + authorization: "Bearer ", + }, + }), + }), + ], +}); + +try { + // Run instrumented application work here. +} finally { + plugin.clear(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::plugin_component::{ + ComponentSpec, ObservabilityConfig, OtlpSectionConfig, +}; +use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; + +let component = ComponentSpec::new(ObservabilityConfig { + opentelemetry: Some(OtlpSectionConfig { + enabled: true, + transport: "http_binary".into(), + endpoint: Some("http://localhost:4318/v1/traces".into()), + service_name: "agent-service".into(), + service_namespace: Some("nemo".into()), + service_version: Some("1.0.0".into()), + instrumentation_scope: Some("nemo-flow-otel".into()), + resource_attributes: [("deployment.environment".into(), "dev".into())].into(), + headers: [("authorization".into(), "Bearer ".into())].into(), + ..OtlpSectionConfig::default() + }), + ..ObservabilityConfig::default() +}); + +let config = PluginConfig { + version: 1, + components: vec![component.into()], + policy: Default::default(), +}; + +let report = validate_plugin_config(&config); +assert!(!report.has_errors()); + +let active = initialize_plugins(config).await?; +``` + +:::: + +::::: + +## Manual API + +Use the manual subscriber API when you need an explicit subscriber name or +direct `force_flush` control. + +:::::{tab-set} +:sync-group: language + +::::{tab-item} Python +:sync: python + +```python +from nemo_flow import OpenTelemetryConfig, OpenTelemetrySubscriber + +config = OpenTelemetryConfig() +config.transport = "http_binary" +config.endpoint = "http://localhost:4318/v1/traces" +config.service_name = "agent-service" +config.set_resource_attribute("deployment.environment", "dev") + +subscriber = OpenTelemetrySubscriber(config) +subscriber.register("otel-exporter") + +# Run instrumented application work here. + +subscriber.force_flush() +subscriber.deregister("otel-exporter") +subscriber.shutdown() +``` + +:::: + +::::{tab-item} Node.js +:sync: node + +```js +const { OpenTelemetrySubscriber } = require("nemo-flow-node"); + +const subscriber = new OpenTelemetrySubscriber({ + transport: "http_binary", + endpoint: "http://localhost:4318/v1/traces", + serviceName: "agent-service", + resourceAttributes: { + "deployment.environment": "dev", + }, +}); +subscriber.register("otel-exporter"); + +try { + // Run instrumented application work here. + + subscriber.forceFlush(); +} finally { + subscriber.deregister("otel-exporter"); + subscriber.shutdown(); +} +``` + +:::: + +::::{tab-item} Rust +:sync: rust + +```rust +use nemo_flow::observability::otel::{OpenTelemetryConfig, OpenTelemetrySubscriber}; + +let config = OpenTelemetryConfig::http_binary("agent-service") + .with_endpoint("http://localhost:4318/v1/traces") + .with_resource_attribute("deployment.environment", "dev"); +let subscriber = OpenTelemetrySubscriber::new(config)?; +subscriber.register("otel-exporter")?; + +// Run instrumented application work here. + +subscriber.force_flush()?; +let _ = subscriber.deregister("otel-exporter")?; +subscriber.shutdown()?; +``` + +:::: + +::::: + +## Common Validation Failures + +- `transport` is not `http_binary` or `grpc`. +- Headers or resource attributes are not string-to-string maps. +- The exporter feature is unavailable in the current build or target. +- The endpoint is unreachable at runtime. diff --git a/docs/reference/api/index.md b/docs/reference/api/index.md index bfbdc2e7..f36356a0 100644 --- a/docs/reference/api/index.md +++ b/docs/reference/api/index.md @@ -3,9 +3,10 @@ SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# API +# APIs -Use these pages for generated symbol-level API documentation across the supported public language surfaces. +Use these pages for generated symbol-level API documentation across the +supported public language surfaces. ```{toctree} :maxdepth: 2 diff --git a/docs/reference/api/nodejs/index.md b/docs/reference/api/nodejs/index.md index 970219aa..0fa1d33e 100644 --- a/docs/reference/api/nodejs/index.md +++ b/docs/reference/api/nodejs/index.md @@ -67,8 +67,8 @@ Use these links to continue from the API reference into task-focused guides. - [Middleware](../../../about/concepts/middleware.md) - [Subscribers](../../../about/concepts/subscribers.md) - [Plugins](../../../about/concepts/plugins.md) -- [Adaptive Optimization](../../../use-adaptive-optimization/about.md) -- [Configure the Observability Plugin](../../../export-observability-data/observability-plugin.md) +- [Adaptive Optimization](../../../plugins/adaptive/about.md) +- [Observability Configuration](../../../plugins/observability/configuration.md) - [Instrument a Tool Call](../../../instrument-applications/instrument-tool-call.md) - [Typed Wrappers and Codecs](../../../integrate-frameworks/using-codecs.md) - [Framework Integration Surfaces](../../../integrate-frameworks/about.md) diff --git a/docs/reference/api/python/index.md b/docs/reference/api/python/index.md index 72b68aa9..674eb495 100644 --- a/docs/reference/api/python/index.md +++ b/docs/reference/api/python/index.md @@ -69,7 +69,7 @@ Use these links to continue from the API reference into task-focused guides. - [Middleware](../../../about/concepts/middleware.md) - [Subscribers](../../../about/concepts/subscribers.md) - [Plugins](../../../about/concepts/plugins.md) -- [Adaptive Optimization](../../../use-adaptive-optimization/about.md) -- [Configure the Observability Plugin](../../../export-observability-data/observability-plugin.md) +- [Adaptive Optimization](../../../plugins/adaptive/about.md) +- [Observability Configuration](../../../plugins/observability/configuration.md) - [Typed Wrappers and Codecs](../../../integrate-frameworks/using-codecs.md) - [Framework Integration Surfaces](../../../integrate-frameworks/about.md) diff --git a/docs/reference/api/rust/index.md b/docs/reference/api/rust/index.md index 18078dd0..9d83cafc 100644 --- a/docs/reference/api/rust/index.md +++ b/docs/reference/api/rust/index.md @@ -40,7 +40,7 @@ Within `nemo-flow-adaptive`, the main surfaces include adaptive configuration, plugin components, storage abstractions, learners, trie-backed data structures, and optional Redis-backed helpers when the feature is enabled. `nemo-flow-cli` is a binary crate, so its end-user surface is documented in -the coding-agent gateway guides rather than generated Rust API pages. +the NeMo Flow CLI guides rather than generated Rust API pages. ## How To Read The Generated Pages @@ -75,8 +75,8 @@ Use these links to continue from the API reference into task-focused guides. - [Middleware](../../../about/concepts/middleware.md) - [Subscribers](../../../about/concepts/subscribers.md) - [Plugins](../../../about/concepts/plugins.md) -- [Adaptive Optimization](../../../use-adaptive-optimization/about.md) -- [Configure the Observability Plugin](../../../export-observability-data/observability-plugin.md) +- [Adaptive Optimization](../../../plugins/adaptive/about.md) +- [Observability Configuration](../../../plugins/observability/configuration.md) - [Typed Wrappers and Codecs](../../../integrate-frameworks/using-codecs.md) - [Framework Integration Surfaces](../../../integrate-frameworks/about.md) -- [Coding-Agent Gateway](../../../integrate-frameworks/coding-agent-gateway.md) +- [NeMo Flow CLI Basic Usage](../../../nemo-flow-cli/basic-usage.md) diff --git a/docs/resources/support-and-faqs.md b/docs/resources/support-and-faqs.md index 6d9f7916..7c535cb2 100644 --- a/docs/resources/support-and-faqs.md +++ b/docs/resources/support-and-faqs.md @@ -100,9 +100,9 @@ shared runtime boundary. The primary documented bindings are Rust, Python, and Node.js. Python uses a native PyO3 extension, Node.js uses a NAPI binding, Go uses the raw C FFI, and WebAssembly uses a `wasm-bindgen` binding. -The documentation and examples also cover integration with ATIF trajectory -export, OpenTelemetry traces, OpenInference-compatible data, and third-party -agent framework patch sets. +The documentation and examples also cover integration with Agent Trajectory +Interchange Format (ATIF) trajectory export, OpenTelemetry traces, +OpenInference-compatible data, and third-party agent framework patch sets. ### Why Is NeMo Flow's Core Written In Rust? @@ -151,19 +151,9 @@ Use these questions to find the right first guide for your task. ### What Should I Read First? -Use the reading path that matches your task: - -| Task | Start With | -|---|---| -| Run a minimal example | [Quick Start](../getting-started/quick-start.md) | -| Install packages or local source | [Installation](../getting-started/installation.md) | -| Understand the runtime model | [Concepts](../about/concepts/index.md) | -| Instrument an application | [Instrument Applications](../instrument-applications/about.md) | -| Integrate a framework | [Integrate into Frameworks](../integrate-frameworks/about.md) | -| Package reusable behavior | [Build Plugins](../build-plugins/about.md) | -| Export traces or trajectories | [Export Observability Data](../export-observability-data/about.md) | -| Configure adaptive behavior | [Use Adaptive Optimization](../use-adaptive-optimization/about.md) | -| Look up symbols | [API](../reference/api/index.md) | +Use the [What Should I Read First?](../index.md#what-should-i-read-first) +table on the documentation overview page to pick the right starting point for +your task. ## Runtime Model And Execution @@ -250,7 +240,7 @@ Choose the surface based on what must change: | Wrap streaming chunk delivery and finalization | Stream execution intercept | Refer to [Middleware](../about/concepts/middleware.md) and -[Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md). +[Add Middleware](../instrument-applications/advanced-guide.md). ### What Is The Middleware Pipeline? @@ -317,7 +307,7 @@ configuration and source code, and emit summarized metadata when full request or response bodies are not needed. Refer to [Middleware](../about/concepts/middleware.md#guardrails) and -[Export Observability Data](../export-observability-data/about.md). +[Observability](../plugins/observability/about.md). ## Observability And Export @@ -331,20 +321,20 @@ operational tracing, trajectory export, or analytics. Refer to [Subscribers](../about/concepts/subscribers.md), [Events](../about/concepts/events.md), and -[Register a Subscriber](../export-observability-data/basic-guide.md). +[Observability](../plugins/observability/about.md). ### Which Exporter Should I Use? Use a local subscriber for debugging and development. Use OpenTelemetry when you want OTLP-compatible traces in existing observability infrastructure. Use OpenInference when your tracing stack expects OpenInference-style agent and LLM -semantics. Use ATIF when you need trajectory artifacts for analysis, replay, or -evaluation workflows. +semantics. Use Agent Trajectory Interchange Format (ATIF) when you need +trajectory artifacts for analysis, replay, or evaluation workflows. -Refer to [Exporter Selection](../export-observability-data/code-examples.md#exporter-selection), -[Export OpenTelemetry Data](../export-observability-data/opentelemetry.md), -[Export OpenInference Data](../export-observability-data/advanced-guide.md), and -[Export ATIF](../export-observability-data/atif.md). +Refer to [Exporter Selection](../plugins/observability/about.md#exporter-selection), +[OpenTelemetry](../plugins/observability/opentelemetry.md), +[OpenInference](../plugins/observability/openinference.md), and +[Agent Trajectory Interchange Format (ATIF)](../plugins/observability/atif.md). ### Can I Use NeMo Flow Just For Observability Without Adaptive Optimization Or Middleware? @@ -390,9 +380,10 @@ telemetry and in-memory state so the runtime can observe representative workflows before changing behavior. Enable active behavior one area at a time, such as adaptive hints, tool parallelism, or cache-governor behavior. -Refer to [Use Adaptive Optimization](../use-adaptive-optimization/about.md), -[Configure Adaptive Optimization](../use-adaptive-optimization/configure.md), -and [Adaptive Components](../use-adaptive-optimization/adaptive-components.md). +Refer to [Adaptive](../plugins/adaptive/about.md), +[Adaptive Configuration](../plugins/adaptive/configuration.md), +[Adaptive Cache Governor (ACG)](../plugins/adaptive/acg.md), and +[Adaptive Hints](../plugins/adaptive/adaptive-hints.md). ## Framework Integration And APIs diff --git a/docs/troubleshooting/troubleshooting-guide.md b/docs/troubleshooting/troubleshooting-guide.md index 8b929222..bfe95405 100644 --- a/docs/troubleshooting/troubleshooting-guide.md +++ b/docs/troubleshooting/troubleshooting-guide.md @@ -84,7 +84,7 @@ Use [Adding Scopes and Marks](../instrument-applications/adding-scopes-and-marks Check whether the middleware was registered globally or scope-locally. Scope-local middleware is visible only to the owning scope and descendant scopes, and it is cleaned up when the owning scope closes. -Use [Middleware](../about/concepts/middleware.md), [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md), and [Instrument Applications Code Examples](../instrument-applications/code-examples.md#middleware-registration-families) to verify the expected registration family. +Use [Middleware](../about/concepts/middleware.md), [Add Middleware](../instrument-applications/advanced-guide.md), and [Instrument Applications Code Examples](../instrument-applications/code-examples.md#middleware-registration-families) to verify the expected registration family. ## Middleware Runs In The Wrong Order @@ -96,7 +96,7 @@ Use [Middleware](../about/concepts/middleware.md) and [Instrument Applications C A guardrail rejection is expected to stop the protected tool or LLM call. Inspect the guardrail result and confirm whether the guardrail was intended to sanitize input, sanitize output, or reject the request completely. -Use [Advanced Guide: Add Middleware](../instrument-applications/advanced-guide.md) to verify the guardrail family and expected behavior. +Use [Add Middleware](../instrument-applications/advanced-guide.md) to verify the guardrail family and expected behavior. ## Request Intercept Changes Are Not Visible @@ -114,19 +114,24 @@ Use [Middleware](../about/concepts/middleware.md) to confirm when an execution i Confirm that the subscriber is registered before the runtime emits the events you expect. For scope-local subscribers, confirm that the active scope is the owning scope or a descendant scope. -Use [Subscribers](../about/concepts/subscribers.md), [Events](../about/concepts/events.md), and [Export Observability Data](../export-observability-data/about.md) to verify lifecycle timing and event names. +Use [Subscribers](../about/concepts/subscribers.md), [Events](../about/concepts/events.md), and [Observability](../plugins/observability/about.md) to verify lifecycle timing and event names. ## Events Are Missing Expected Fields Managed tool and LLM helpers populate semantic fields such as inputs, outputs, model names, and tool call IDs. Manual lifecycle calls require you to provide the relevant params explicitly. -Use [Export Observability Data Code Examples](../export-observability-data/code-examples.md#event-shape) and [Instrument Applications Code Examples](../instrument-applications/code-examples.md) to verify the emitted payload shape. +Use [Events](../about/concepts/events.md) and [Instrument Applications Code Examples](../instrument-applications/code-examples.md) to verify the emitted payload shape. -## ATIF Export Is Empty Or Mixed Across Agents +## Agent Trajectory Interchange Format (ATIF) Export Is Empty Or Mixed Across Agents -An empty ATIF export usually means the exporter subscribed after the relevant events were emitted, or the export filter does not match the active `root_uuid`. Mixed trajectories usually mean multiple agents share a root scope or the export did not filter by root scope. +An empty Agent Trajectory Interchange Format (ATIF) export usually means the +exporter subscribed after the relevant events were emitted, or the export +filter does not match the active `root_uuid`. Mixed trajectories usually mean +multiple agents share a root scope or the export did not filter by root scope. -Use [ATIF Export](../export-observability-data/atif.md) and [Export Observability Data Code Examples](../export-observability-data/code-examples.md) to confirm exporter setup, event collection timing, and root-scope filtering. +Use [Agent Trajectory Interchange Format (ATIF)](../plugins/observability/atif.md) +and [Observability](../plugins/observability/about.md) to confirm exporter +setup, event collection timing, and root-scope filtering. ## LLM Stream Output Is Missing The Final Chunk @@ -150,7 +155,10 @@ Use [Validate Configuration](../build-plugins/validate-configuration.md), [Advan Confirm that adaptive optimization is configured for the component you expect and that the runtime path actually reaches that component. If behavior does not change, check whether the configured policy is disabled, scoped too narrowly, or not connected to the call path under test. -Use [Configure Adaptive Optimization](../use-adaptive-optimization/configure.md) and [Adaptive Components](../use-adaptive-optimization/adaptive-components.md) to verify component names and configuration scope. +Use [Adaptive Configuration](../plugins/adaptive/configuration.md), +[Adaptive Cache Governor (ACG)](../plugins/adaptive/acg.md), and +[Adaptive Hints](../plugins/adaptive/adaptive-hints.md) to verify component +names and configuration scope. ## Third-Party Patch Does Not Apply diff --git a/docs/use-adaptive-optimization/about.md b/docs/use-adaptive-optimization/about.md deleted file mode 100644 index 5774ff7e..00000000 --- a/docs/use-adaptive-optimization/about.md +++ /dev/null @@ -1,55 +0,0 @@ - - -# About - -Use this section when you want NeMo Flow to collect runtime signals and activate adaptive behavior through the plugin system. - -Adaptive optimization uses the same runtime model as the rest of NeMo Flow: -instrumented scopes and calls emit events, subscribers and learners observe -those events, request intercepts can add hints, and plugin configuration -controls activation. The adaptive component coordinates state, telemetry, -adaptive hints, tool parallelism, cache-governor behavior, and rollout policy. - -Agent workflows often repeat similar work, call tools with different dependency -patterns, and send prompts with stable and variable sections. Adaptive -optimization gives the runtime a place to observe those patterns and expose -controlled behavior changes without hard-coding optimization logic into every -application. - -## Start Here When - -Use these signals to decide whether this documentation path matches your current task. - -- Collect runtime signals before changing behavior -- Evaluate tool parallelism opportunities -- Add model-request hints in a controlled way -- Plan prompt-cache breakpoints for supported providers -- Share adaptive state across workers when needed -- Roll out optimization through config instead of code changes - -If instrumentation is not in place yet, start with -[Instrument Applications](../instrument-applications/about.md) or -[Integrate into Frameworks](../integrate-frameworks/about.md). - -## Guides - -Use these guide links to move from the overview into task-specific instructions. - -- [Basic Guide: Configure Adaptive Optimization](configure.md) shows the conservative first configuration and validation workflow. -- [Advanced Guide: Configure Adaptive Components](adaptive-components.md) explains the adaptive plugin component and its config fields in more detail. -- [Advanced Guide: Tune Adaptive Behavior](advanced-guide.md) explains state tuning, telemetry tuning, adaptive hints, tool parallelism, cache-governor tuning, and diagnostics. -- [Code Examples](code-examples.md) provides binding-level adaptive helper names, defaults, ACG threshold overrides, and runtime-adjacent variables. - -Validate the basic workflow before tuning lower-level adaptive settings. First -confirm that instrumented work emits lifecycle events with the configuration in -[Basic Guide: Configure Adaptive Optimization](configure.md). After that -baseline is visible, keep tool parallelism in `observe_only`, leave cache -planning disabled until you have stable prompt samples, and enable active -behavior one area at a time. - -Treat every adaptive change as a measured rollout. Record a baseline, change one -setting, compare events and reports, and keep rollback as a configuration -change. diff --git a/docs/use-adaptive-optimization/adaptive-components.md b/docs/use-adaptive-optimization/adaptive-components.md deleted file mode 100644 index 2c49ce40..00000000 --- a/docs/use-adaptive-optimization/adaptive-components.md +++ /dev/null @@ -1,275 +0,0 @@ - - -# Advanced Guide: Configure Adaptive Components - -Use this guide when you are configuring the built-in adaptive component that enables NeMo Flow adaptive behavior across frameworks and bindings. - -## What You Build - -You will configure the built-in adaptive component, validate it through the plugin system, initialize it, and understand which runtime surfaces adaptive behavior can register. - -## Before You Start - -You need: - -- Instrumented tool or LLM calls that emit lifecycle events. -- A plugin configuration path for the target binding. -- A stable logical agent ID. -- A decision about whether adaptive state is process-local or shared. - -## What Adaptive Covers - -The adaptive component is a plugin that enables built-in optimization and learning behaviors across frameworks and bindings. - -Its top-level config is a single component with kind `adaptive`. - -## What It Registers - -Adaptive is not one middleware hook. It is a bundle of runtime behavior that can register: - -- Subscribers for adaptive telemetry -- LLM request intercepts for adaptive hint injection -- Tool-related behaviors for parallelism guidance or scheduling -- LLM execution intercepts that plan prompt-cache breakpoints via the adaptive cache governor (ACG) -- State backends used by those features - -## Main Config Areas - -These areas organize the adaptive configuration fields by runtime responsibility. - -### State - -Select where adaptive state lives: - -- In-memory backend -- Redis backend - -### Telemetry - -Configure the built-in adaptive subscriber and the enabled learners. - -### Adaptive Hints - -Configure the request-intercept behavior that injects adaptive hints: - -- `priority` -- `break_chain` -- `inject_header` -- `inject_body_path` - -Adaptive hints register as request intercepts. Lower numeric priority values run -earlier in the intercept chain, so the default priority is chosen relative to -other middleware rather than as a standalone importance score. - -### Tool Parallelism - -Configure the tool scheduling behavior: - -- `priority` -- `mode` - -Tool-parallelism priority follows the same middleware ordering rule: lower -numbers run earlier, higher numbers leave earlier intercepts and guardrails in -front of adaptive scheduling behavior. - -Supported modes include: - -- `observe_only` -- `inject_hints` -- `schedule` - -### Cache Governor (ACG) - -Configure the adaptive cache governor, the internal subsystem that decomposes -LLM requests into an addressable Prompt IR, scores block stability across -observed runs, and plans provider-specific prompt-cache breakpoints: - -- `provider` -- `observation_window` -- `priority` -- `stability_thresholds` - -Supported providers: - -- `passthrough` -- `anthropic` -- `openai` - -ACG is an optional section on the adaptive config; omit it to keep cache -planning disabled. - -## End-To-End Example - -The examples below show a complete adaptive component configuration in each binding -style. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -import nemo_flow - -adaptive_config = nemo_flow.adaptive.AdaptiveConfig( - agent_id="planner", - state=nemo_flow.adaptive.StateConfig( - backend=nemo_flow.adaptive.BackendSpec.in_memory(), - ), - telemetry=nemo_flow.adaptive.TelemetryConfig( - subscriber_name="adaptive.telemetry", - learners=["tool_parallelism"], - ), - adaptive_hints=nemo_flow.adaptive.AdaptiveHintsConfig(), - tool_parallelism=nemo_flow.adaptive.ToolParallelismConfig(mode="observe_only"), - acg=nemo_flow.adaptive.AcgConfig(provider="anthropic"), -) - -plugin_config = nemo_flow.plugin.PluginConfig( - components=[nemo_flow.adaptive.ComponentSpec(adaptive_config)] -) - -report = nemo_flow.plugin.validate(plugin_config) -active = await nemo_flow.plugin.initialize(plugin_config) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import * as adaptive from 'nemo-flow-node/adaptive'; -import * as plugin from 'nemo-flow-node/plugin'; - -const adaptiveConfig = adaptive.defaultConfig(); -adaptiveConfig.agent_id = 'planner'; -adaptiveConfig.state = { backend: adaptive.inMemoryBackend() }; -adaptiveConfig.telemetry = adaptive.telemetryConfig({ - subscriber_name: 'adaptive.telemetry', - learners: ['tool_parallelism'], -}); -adaptiveConfig.adaptive_hints = adaptive.adaptiveHintsConfig(); -adaptiveConfig.tool_parallelism = adaptive.toolParallelismConfig({ mode: 'observe_only' }); -adaptiveConfig.acg = adaptive.acgConfig({ provider: 'anthropic' }); - -const pluginConfig = plugin.defaultConfig(); -pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)]; - -const report = plugin.validate(pluginConfig); -const active = await plugin.initialize(pluginConfig); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::plugin::{PluginConfig, initialize_plugins, validate_plugin_config}; -use nemo_flow_adaptive::{ - AcgComponentConfig, - AdaptiveConfig, - AdaptiveHintsComponentConfig, - BackendSpec, - ComponentSpec, - StateConfig, - TelemetryComponentConfig, - ToolParallelismComponentConfig, -}; - -let mut adaptive = AdaptiveConfig::default(); -adaptive.agent_id = Some("planner".into()); -adaptive.state = Some(StateConfig { - backend: BackendSpec::in_memory(), -}); -adaptive.telemetry = Some(TelemetryComponentConfig { - subscriber_name: Some("adaptive.telemetry".into()), - learners: vec!["tool_parallelism".into()], -}); -adaptive.adaptive_hints = Some(AdaptiveHintsComponentConfig::default()); -adaptive.tool_parallelism = Some(ToolParallelismComponentConfig::default()); -adaptive.acg = Some(AcgComponentConfig { - provider: "anthropic".into(), - ..AcgComponentConfig::default() -}); - -let mut plugin_config = PluginConfig::default(); -plugin_config.components.push(ComponentSpec::new(adaptive).into()); - -let report = validate_plugin_config(&plugin_config); -let active = initialize_plugins(plugin_config).await?; -``` -::: -:::: - -## Field-Level Guidance - -These notes describe how to set individual adaptive configuration fields. - -### `agent_id` - -Use `agent_id` when one adaptive configuration should consistently identify a logical agent across requests. - -### `state` - -Choose `in_memory` for local development and process-local experiments. Choose `redis` when adaptive state must survive restarts or be shared across workers. - -### `telemetry` - -Set `subscriber_name` when you want the adaptive subscriber to appear under a predictable runtime name. Use `learners` to control which internal learners consume the observed event stream. - -### `adaptive_hints` - -Use this when you want the runtime to add hint metadata into outgoing model -requests. The most important controls are priority, whether the request-intercept -chain should stop, and where the injected data should live. The default -`adaptive_hints.priority`, `100`, means the hint injector runs after intercepts -with lower numeric values and before intercepts with higher numeric values; -adjust it only when you need adaptive hints to run before or after known -application middleware. - -### `tool_parallelism` - -Start with `observe_only`. Move to `inject_hints` when downstream code can interpret guidance safely. Use `schedule` only when you explicitly want adaptive behavior to influence execution strategy. - -### `acg` - -Enable the cache governor when you want adaptive to plan prompt-cache -breakpoints for outgoing LLM requests. Set `provider` to match the backend -API surface the agent actually calls (`anthropic`, `openai`, or -`passthrough` to disable the planner while keeping observations). Tune -`observation_window` to control how many recent PromptIR samples feed -stability analysis, and use `stability_thresholds` to adjust when a block -is classified as stable enough to cache. - -## Why It Matters for Plugin Authors - -Adaptive shows how to build a cross-language component that: - -- Validates config -- Registers multiple runtime surfaces -- Uses shared state -- Stays visible through the generic plugin system - -## Validate the Component - -Before enabling adaptive behavior beyond observation: - -1. Validate the plugin config and inspect diagnostics. -2. Initialize the component in a development environment. -3. Run representative instrumented workflows. -4. Confirm telemetry receives scope, tool, and LLM events. -5. Keep tool parallelism in `observe_only` until scheduling behavior is verified. -6. Enable ACG only after repeated LLM requests show stable prompt sections. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **Adaptive appears inactive**: Confirm the app emits NeMo Flow lifecycle events. -- **State disappears after restart**: Use Redis-backed state instead of in-memory state. -- **Provider cache hints are wrong**: Set `provider` to match the actual model API surface or use `passthrough` while observing. -- **Scheduling changes behavior**: Return tool parallelism to `observe_only` and validate tool idempotency. diff --git a/docs/use-adaptive-optimization/advanced-guide.md b/docs/use-adaptive-optimization/advanced-guide.md deleted file mode 100644 index 5b8747cb..00000000 --- a/docs/use-adaptive-optimization/advanced-guide.md +++ /dev/null @@ -1,144 +0,0 @@ - - -# Advanced Guide: Tune Adaptive Behavior - -Use this guide after the adaptive component is initialized and observing representative NeMo Flow traffic. - -## What You Tune - -Adaptive optimization has several independent control surfaces: - -- State backend and retention. -- Telemetry subscribers and learners. -- Adaptive hint injection. -- Tool parallelism mode. -- Adaptive cache governor behavior. -- Policy and rollout controls. - -Tune one area at a time. Keep a stable baseline so you can tell whether a change improves latency, cost, cache behavior, or reliability. - -## Before You Start - -Complete [Basic Guide: Configure Adaptive Optimization](configure.md). You should have: - -- Instrumented tool or LLM calls. -- Adaptive telemetry enabled. -- A stable `agent_id`. -- At least one representative workflow that you can run repeatedly. -- A way to inspect lifecycle events, adaptive reports, or exported traces. - -## Tuning Workflow - -Follow this workflow to tune adaptive behavior from measured baselines rather than -guesses. - -1. Record baseline latency, tool count, model usage, cache behavior, and error rate. -2. Enable one adaptive area in observe-only or low-impact mode. -3. Run the same representative workflow several times. -4. Compare events, reports, and application behavior against the baseline. -5. Promote the setting only when the measured behavior is stable. -6. Document the setting and the workflow it was tuned for. - -## State Tuning - -Use in-memory state for local development, tests, and short-lived experiments. Use Redis-backed state when multiple workers need shared observations or when adaptive behavior should survive process restarts. - -Keep these decisions explicit: - -- Which logical agent owns the state. -- How long observations remain useful. -- Whether state can be shared across tenants. -- How state is reset during tests or rollbacks. - -## Telemetry Tuning - -Telemetry controls what adaptive learners observe. Start with only the learners needed for the feature you are testing. - -Use telemetry to answer concrete questions: - -- Which tools are frequently independent and safe to parallelize? -- Which prompt sections are stable across repeated runs? -- Which requests carry enough metadata for hints or cache planning? -- Which failures correlate with specific runtime paths? - -If telemetry volume is high, reduce the enabled learners before reducing instrumentation. Instrumentation is also used by subscribers and exporters outside adaptive optimization. - -## Adaptive Hint Tuning - -Adaptive hints attach guidance to model requests. Use them when downstream code or provider adapters can safely consume hint metadata. - -Tune hints conservatively: - -1. Start with a low priority so existing request intercepts run first. -2. Inject hints into a predictable header or body path. -3. Validate the transformed request before it reaches the provider. -4. Disable `break_chain` unless the adaptive hint should be the final request transform. - -## Tool Parallelism Tuning - -Tool parallelism should move through three phases: - -| Mode | Use When | Expected Behavior | -|---|---|---| -| `observe_only` | You are gathering data | Execution does not change | -| `inject_hints` | Downstream code can interpret guidance | Runtime adds guidance but does not own scheduling | -| `schedule` | You want adaptive behavior to influence execution strategy | Runtime can affect scheduling decisions | - -Only use `schedule` when tool callbacks are idempotent or safe to run under the planned concurrency model. - -## Adaptive Cache Governor Tuning - -Enable the adaptive cache governor when repeated LLM requests contain stable prompt sections that can benefit from provider prompt caching. - -Tune these fields together: - -- `provider`: Set to `anthropic`, `openai`, or `passthrough` based on the provider surface. -- `observation_window`: Increase when prompts vary across runs and stability needs more samples. -- `stability_thresholds`: Raise thresholds when cache breakpoints are too aggressive. -- Policy controls: Keep reporting enabled while tuning so decisions are auditable. - -Use `passthrough` when you want to keep observing prompt structure without applying provider-specific cache planning. - -## Diagnostics and Runtime Variables - -NeMo Flow does not require application-level environment variables for normal adaptive runtime use. Prefer explicit adaptive config objects for application behavior. - -Use these variables only for adjacent diagnostics and tests: - -| Variable | Scope | Purpose | -|---|---|---| -| `NEMO_FLOW_ACG_DEBUG` | Adaptive cache-governor diagnostics | Enables cache-governor debug diagnostics in adaptive internals. | -| `NEMO_FLOW_RUN_REDIS_TESTS` | Test workflows | Enables Redis-backed adaptive tests. | - -Internal variables such as `NEMO_FLOW_BINDING_KIND` and `NEMO_FLOW_RUNTIME_OWNER` are binding and test controls. Do not set them in application code unless a maintainer asks you to debug runtime ownership behavior. - -## Validation Checklist - -For each change, verify: - -- Application results are unchanged unless the change intentionally affects scheduling. -- Emitted events still include the expected scope, tool, and LLM spans. -- Adaptive reports explain the decision that changed behavior. -- Latency, token usage, or cache behavior improves on representative traffic. -- Rollback is a configuration change, not a code change. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **Observed behavior is noisy**: Increase the observation window or reduce the workflow variance during tuning. -- **Hints conflict with application intercepts**: Adjust priority or disable `break_chain`. -- **Parallelism creates race conditions**: Return to `observe_only` and audit tool idempotency. -- **Cache planning is unstable**: Use more samples, raise stability thresholds, or set provider to `passthrough`. -- **State leaks across tenants**: Scope state by `agent_id` and deployment boundaries. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Review [Code Examples](code-examples.md) for binding APIs, defaults, and ACG threshold overrides. -- Review [Advanced Guide: Configure Adaptive Components](adaptive-components.md) for plugin-level adaptive configuration. -- Export traces with [Advanced Guide: Export OpenInference Data](../export-observability-data/advanced-guide.md) to compare behavior across runs. diff --git a/docs/use-adaptive-optimization/code-examples.md b/docs/use-adaptive-optimization/code-examples.md deleted file mode 100644 index 931fd529..00000000 --- a/docs/use-adaptive-optimization/code-examples.md +++ /dev/null @@ -1,237 +0,0 @@ - - -# Code Examples - -Use these examples when you need the binding-level adaptive helper names or a concrete configuration shape. - -- [Advanced Guide: Configure Adaptive Components](adaptive-components.md) - -## Complete Adaptive Component - -Adaptive is configured as one plugin component with fixed kind `adaptive`. - -At a high level: - -1. Identify the component version and optional `agent_id`. -2. Choose a state backend. -3. Optionally enable telemetry. -4. Optionally configure adaptive hints. -5. Optionally configure tool parallelism. -6. Optionally configure the adaptive cache governor. -7. Apply config validation policy. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import adaptive - -config = adaptive.AdaptiveConfig() -config.agent_id = "planner" -config.state = adaptive.StateConfig( - backend=adaptive.BackendSpec.in_memory(), -) -config.telemetry = adaptive.TelemetryConfig( - subscriber_name="adaptive.telemetry", - learners=["tool_parallelism"], -) -config.adaptive_hints = adaptive.AdaptiveHintsConfig() -config.tool_parallelism = adaptive.ToolParallelismConfig(mode="observe_only") -config.acg = adaptive.AcgConfig(provider="openai") - -component = adaptive.ComponentSpec(config) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import * as adaptive from 'nemo-flow-node/adaptive'; - -const config = adaptive.defaultConfig(); -config.agent_id = 'planner'; -config.state = { - backend: adaptive.inMemoryBackend(), -}; -config.telemetry = adaptive.telemetryConfig({ - subscriber_name: 'adaptive.telemetry', - learners: ['tool_parallelism'], -}); -config.adaptive_hints = adaptive.adaptiveHintsConfig(); -config.tool_parallelism = adaptive.toolParallelismConfig({ mode: 'observe_only' }); -config.acg = adaptive.acgConfig({ provider: 'openai' }); - -const component = adaptive.ComponentSpec(config); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow_adaptive::{ - AcgComponentConfig, - AdaptiveConfig, - AdaptiveHintsComponentConfig, - BackendSpec, - ComponentSpec, - StateConfig, - TelemetryComponentConfig, - ToolParallelismComponentConfig, -}; - -let mut config = AdaptiveConfig::default(); -config.agent_id = Some("planner".into()); -config.state = Some(StateConfig { - backend: BackendSpec::in_memory(), -}); -config.telemetry = Some(TelemetryComponentConfig { - subscriber_name: Some("adaptive.telemetry".into()), - learners: vec!["tool_parallelism".into()], -}); -config.adaptive_hints = Some(AdaptiveHintsComponentConfig::default()); -config.tool_parallelism = Some(ToolParallelismComponentConfig::default()); -config.acg = Some(AcgComponentConfig { - provider: "openai".into(), - ..AcgComponentConfig::default() -}); - -let component = ComponentSpec::new(config); -``` -::: - -:::: - -## Adaptive Cache Governor Thresholds - -Override stability thresholds when cache breakpoint planning is too conservative or too aggressive for representative prompts. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -from nemo_flow import adaptive - -config = adaptive.AdaptiveConfig() -config.acg = adaptive.AcgConfig( - provider="openai", - stability_thresholds=adaptive.AcgStabilityThresholds( - stable_threshold=0.99, - semi_stable_threshold=0.75, - min_observations_for_full_confidence=12, - ), -) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```ts -import * as adaptive from 'nemo-flow-node/adaptive'; - -const config = adaptive.defaultConfig(); -config.acg = adaptive.acgConfig({ - provider: 'openai', - stability_thresholds: { - stable_threshold: 0.99, - semi_stable_threshold: 0.75, - min_observations_for_full_confidence: 12, - }, -}); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow_adaptive::acg::stability::StabilityThresholds; -use nemo_flow_adaptive::{AcgComponentConfig, AdaptiveConfig}; - -let mut config = AdaptiveConfig::default(); -config.acg = Some(AcgComponentConfig { - provider: "openai".into(), - stability_thresholds: StabilityThresholds { - stable_threshold: 0.99, - semi_stable_threshold: 0.75, - min_observations_for_full_confidence: 12, - }, - ..AcgComponentConfig::default() -}); -``` -::: - -:::: - -## Configuration Fields - -The top-level `AdaptiveConfig` contains: - -- `version` -- `agent_id` -- `state` -- `telemetry` -- `adaptive_hints` -- `tool_parallelism` -- `acg` -- `policy` - -Important nested fields: - -| Section | Fields | -|---|---| -| `state` | `backend.kind`, `backend.config` | -| `telemetry` | `subscriber_name`, `learners` | -| `adaptive_hints` | `priority`, `break_chain`, `inject_header`, `inject_body_path` | -| `tool_parallelism` | `priority`, `mode` | -| `acg` | `provider`, `observation_window`, `priority`, `stability_thresholds` | -| `policy` | `unknown_component`, `unknown_field`, `unsupported_value` | - -Supported state backends are `in_memory` and `redis`. Supported tool-parallelism modes are `observe_only`, `inject_hints`, and `schedule`. Supported ACG providers are `passthrough`, `anthropic`, and `openai`. - -## Defaults To Know - -These defaults are important when reading or overriding adaptive configuration examples. - -Middleware priorities are ordered by ascending number. For example, -`adaptive_hints.priority = 100` is not a percentage or severity level; it places -adaptive hint injection after middleware with lower numeric priorities and before -middleware with higher numeric priorities. - -- `version = 1` -- `agent_id = None` -- `state = None` -- `telemetry = None` -- `adaptive_hints.priority = 100` -- `adaptive_hints.break_chain = false` -- `adaptive_hints.inject_header = true` -- `adaptive_hints.inject_body_path = "nvext.agent_hints"` -- `tool_parallelism.mode = "observe_only"` -- `acg.provider = "passthrough"` -- `acg.observation_window = 100` -- `acg.priority = 50` -- `acg.stability_thresholds.stable_threshold = 0.95` -- `acg.stability_thresholds.semi_stable_threshold = 0.50` -- `acg.stability_thresholds.min_observations_for_full_confidence = 20` - -## Runtime-Adjacent Variables - -NeMo Flow does not require application-level environment variables for normal adaptive runtime use. These variables are available for adjacent workflows: - -| Variable | Scope | Purpose | -|---|---|---| -| `NEMO_FLOW_ACG_DEBUG` | Adaptive cache-governor diagnostics | Enables cache-governor debug diagnostics in adaptive internals. | -| `NEMO_FLOW_RUN_REDIS_TESTS` | Test workflows | Enables Redis-backed adaptive tests. | - -Internal variables such as `NEMO_FLOW_BINDING_KIND` and `NEMO_FLOW_RUNTIME_OWNER` are binding and test controls. Do not set them in application code unless a maintainer asks you to debug runtime ownership behavior. diff --git a/docs/use-adaptive-optimization/configure.md b/docs/use-adaptive-optimization/configure.md deleted file mode 100644 index e50d3a8c..00000000 --- a/docs/use-adaptive-optimization/configure.md +++ /dev/null @@ -1,185 +0,0 @@ - - -# Basic Guide: Configure Adaptive Optimization - -Use this guide when you want to enable NeMo Flow adaptive behavior through the adaptive plugin component. - -## What You Build - -You will create an adaptive component configuration, validate it, initialize it through the plugin system, and run instrumented work while adaptive telemetry observes lifecycle events. - -Start with the smallest configuration that gives you observability before you let adaptive behavior influence execution. A conservative first configuration uses: - -- In-memory state. -- Telemetry enabled. -- Tool parallelism in `observe_only` mode. -- Optional adaptive hints disabled or enabled only for a controlled model path. -- Cache governor disabled until you have representative LLM traffic. - -## Before You Start - -Complete these steps: - -1. Instrument tool or LLM calls with NeMo Flow managed helpers. -2. Register observability during development so you can inspect lifecycle events. -3. Choose a stable `agent_id` for the logical agent or workflow. -4. Decide whether adaptive state can be process-local or must be shared across workers. - -Use [Advanced Guide: Tune Adaptive Behavior](advanced-guide.md) after the basic configuration is running. - -## Configuration Areas - -The table below summarizes the adaptive configuration areas and when to use them. - -| Area | Start With | Use When | -|---|---|---| -| State | In-memory backend | You want local development or process-local experiments | -| Telemetry | Enabled | You want adaptive components to observe runtime events | -| Adaptive hints | Disabled or low-priority | Downstream model calls can safely receive hint metadata | -| Tool parallelism | `observe_only` | You want measurements before adaptive scheduling | -| Cache governor | Disabled | Enable later for stable prompt-cache planning | -| Policy | Conservative defaults | You need explicit rollout or reporting controls | - -The top-level adaptive object contains `version`, `agent_id`, `state`, `telemetry`, `adaptive_hints`, `tool_parallelism`, `acg`, and `policy`. You wrap that object with `ComponentSpec(...)`, insert it into a plugin configuration, validate, and initialize through the plugin system. - -:::{note} -Adaptive plugin configuration uses the generic NeMo Flow plugin document shape, -so configuration field names stay `snake_case` in every binding. Node.js helper -function names such as `toolParallelismConfig(...)` are `camelCase`, but the -config object they create uses canonical `snake_case` keys. -::: - -## Create a Minimal Adaptive Component - -The examples below create a minimal adaptive component configuration for each binding. - -::::{tab-set} -:sync-group: language - -:::{tab-item} Python -:sync: python - -```python -import nemo_flow - -adaptive_config = nemo_flow.adaptive.AdaptiveConfig( - agent_id="planner", - state=nemo_flow.adaptive.StateConfig( - backend=nemo_flow.adaptive.BackendSpec.in_memory(), - ), - telemetry=nemo_flow.adaptive.TelemetryConfig( - subscriber_name="adaptive.telemetry", - learners=["tool_parallelism"], - ), - tool_parallelism=nemo_flow.adaptive.ToolParallelismConfig(mode="observe_only"), -) - -plugin_config = nemo_flow.plugin.PluginConfig( - components=[nemo_flow.adaptive.ComponentSpec(adaptive_config)] -) - -report = nemo_flow.plugin.validate(plugin_config) -active = await nemo_flow.plugin.initialize(plugin_config) -``` -::: - -:::{tab-item} Node.js -:sync: node - -```js -const adaptive = require("nemo-flow-node/adaptive"); -const plugin = require("nemo-flow-node/plugin"); - -const adaptiveConfig = adaptive.defaultConfig(); -adaptiveConfig.agent_id = "planner"; -adaptiveConfig.state = { backend: adaptive.inMemoryBackend() }; -adaptiveConfig.telemetry = adaptive.telemetryConfig({ - subscriber_name: "adaptive.telemetry", - learners: ["tool_parallelism"], -}); -adaptiveConfig.tool_parallelism = adaptive.toolParallelismConfig({ mode: "observe_only" }); - -const pluginConfig = plugin.defaultConfig(); -pluginConfig.components = [adaptive.ComponentSpec(adaptiveConfig)]; - -const report = plugin.validate(pluginConfig); -const active = await plugin.initialize(pluginConfig); -``` -::: - -:::{tab-item} Rust -:sync: rust - -```rust -use nemo_flow::plugin::{initialize_plugins, validate_plugin_config, PluginConfig}; -use nemo_flow_adaptive::{ - AdaptiveConfig, - BackendSpec, - ComponentSpec, - StateConfig, - TelemetryComponentConfig, - ToolParallelismComponentConfig, -}; - -let mut adaptive = AdaptiveConfig::default(); -adaptive.agent_id = Some("planner".into()); -adaptive.state = Some(StateConfig { - backend: BackendSpec::in_memory(), -}); -adaptive.telemetry = Some(TelemetryComponentConfig { - subscriber_name: Some("adaptive.telemetry".into()), - learners: vec!["tool_parallelism".into()], -}); -adaptive.tool_parallelism = Some(ToolParallelismComponentConfig::default()); - -let mut plugin_config = PluginConfig::default(); -plugin_config.components.push(ComponentSpec::new(adaptive).into()); - -let report = validate_plugin_config(&plugin_config); -let active = initialize_plugins(plugin_config).await?; -``` -::: - -:::: - -## Validate the Configuration - -Before you run production traffic, confirm these points: - -- Validation diagnostics do not include errors. -- Initialization returns an active plugin handle or active configuration object. -- Instrumented tool or LLM calls still return the same application result. -- The adaptive telemetry subscriber appears in emitted events or logs. -- Tool parallelism stays observational when mode is `observe_only`. - -If validation reports warnings, decide whether the configuration is acceptable for the environment before initialization. If validation reports errors, fix the component config before initialization. - -## Roll Out Safely - -Use this sequence for first deployment: - -1. Enable in-memory state and telemetry in a development environment. -2. Run representative agent workflows. -3. Review emitted events and adaptive reports. -4. Enable persistent state only after the observed signals are useful. -5. Enable adaptive hints or scheduling only after downstream components can interpret them safely. - -## Common Issues - -Check these symptoms first when the workflow does not behave as expected. - -- **No adaptive activity appears**: Confirm that application work emits NeMo Flow lifecycle events. -- **State resets unexpectedly**: In-memory state is process-local. Use Redis-backed state when state must survive restarts or be shared across workers. -- **Tool execution changes too early**: Keep tool parallelism in `observe_only` until you are ready for adaptive scheduling. -- **Model requests receive unexpected metadata**: Disable adaptive hints or lower their priority while debugging. - -## Next Steps - -Use these links to continue from this workflow into the next related task. - -- Tune advanced behavior with [Advanced Guide: Tune Adaptive Behavior](advanced-guide.md). -- Review the plugin-facing shape in [Advanced Guide: Configure Adaptive Components](adaptive-components.md). -- Use [Code Examples](code-examples.md) for binding-level helper names, ACG threshold overrides, defaults, and runtime-adjacent variables. diff --git a/go/nemo_flow/README.md b/go/nemo_flow/README.md index 33e97803..3805247e 100644 --- a/go/nemo_flow/README.md +++ b/go/nemo_flow/README.md @@ -12,6 +12,7 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) # NeMo Flow Go Binding diff --git a/integrations/openclaw/README.md b/integrations/openclaw/README.md index fdaa1c36..7ca9166c 100644 --- a/integrations/openclaw/README.md +++ b/integrations/openclaw/README.md @@ -11,6 +11,11 @@ LLM spans, tool spans, and lifecycle marks that the generic NeMo Flow observability component can export as ATIF JSON, OpenTelemetry spans, and OpenInference/Phoenix spans. +This public OpenClaw plugin package provides observability support only. It +does not add NeMo Flow security middleware or adaptive optimization behavior to +OpenClaw execution. For middleware-backed behavior, use the patch-based +OpenClaw integration from the NeMo Flow repository. + ## Why Use It? - Observe OpenClaw sessions without patching OpenClaw. @@ -178,6 +183,9 @@ openclaw gateway call nemoFlow.status --json The plugin maps supported OpenClaw hook events into NeMo Flow telemetry without changing OpenClaw execution behavior. +It does not change OpenClaw tool execution, provider routing, policy decisions, +or adaptive behavior. + Current OpenClaw public hooks expose request, response, message-write, and provider timing details through separate event streams. The plugin correlates those events within the same session, provider, model, and run. When timing diff --git a/python/nemo_flow/README.md b/python/nemo_flow/README.md index 5b6b4368..0ddd7e61 100644 --- a/python/nemo_flow/README.md +++ b/python/nemo_flow/README.md @@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0 [![npm wasm](https://img.shields.io/npm/v/nemo-flow-wasm?label=nemo-flow-wasm&color=CC3534&logo=npm)](https://www.npmjs.com/package/nemo-flow-wasm) [![Crates.io](https://img.shields.io/crates/v/nemo-flow?label=nemo-flow&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow) [![Crates.io](https://img.shields.io/crates/v/nemo-flow-adaptive?label=nemo-flow-adaptive&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-adaptive) +[![Crates.io](https://img.shields.io/crates/v/nemo-flow-cli?label=nemo-flow-cli&color=B7410E&logo=rust)](https://crates.io/crates/nemo-flow-cli) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/NeMo-Flow) -# NeMo Flow Python Package +# NeMo Flow `nemo-flow` is the NeMo Flow package for Python applications. It gives Python code access to a portable agent runtime for execution scopes, middleware, @@ -31,7 +32,8 @@ runtime semantics as the Rust and Node.js surfaces. - πŸ›‘οΈ **Package policy around callbacks**: Use guardrails and intercepts to block work, sanitize observability payloads, rewrite requests, or wrap execution. - πŸ“‘ **Emit one lifecycle stream**: Send runtime events to in-process - subscribers, ATIF, OpenTelemetry, or OpenInference workflows. + subscribers, Agent Trajectory Interchange Format (ATIF), OpenTelemetry, or + OpenInference workflows. - 🧩 **Integrate without a framework migration**: Wrap framework or provider callbacks while preserving the application’s orchestration model. @@ -88,6 +90,20 @@ uv add "nemo-flow[langgraph]" pip install "nemo-flow[langgraph]" ``` +#### Deep Agents Integration + +[Deep Agents](https://www.langchain.com/deep-agents) integration is available +with the `deepagents` extra. This extra builds upon and includes the +`langgraph` and `langchain` extras. + +```bash +# With uv +uv add "nemo-flow[deepagents]" + +# With pip +pip install "nemo-flow[deepagents]" +``` + #### LangChain NVIDIA Integration The [LangChain NVIDIA](https://github.com/langchain-ai/langchain-nvidia) extra builds upon the `langchain` extra adding a compatible version of the `langchain-nvidia-ai-endpoints` package. @@ -104,9 +120,9 @@ To install this along with the `langgraph` extra, use: ```bash # With uv -uv add nemo-flow[langgraph,langchain-nvidia] +uv add "nemo-flow[langgraph,langchain-nvidia]" # With pip -pip install nemo-flow[langgraph,langchain-nvidia] +pip install "nemo-flow[langgraph,langchain-nvidia]" ``` ## Getting Started @@ -149,6 +165,7 @@ The public package modules are: - `nemo_flow.integrations.langchain` - `nemo_flow.integrations.langgraph` +- `nemo_flow.integrations.deepagents` The compiled extension is exposed as `nemo_flow._native`. diff --git a/third_party/README-openclaw.md b/third_party/README-openclaw.md index d22b4e44..bf446597 100644 --- a/third_party/README-openclaw.md +++ b/third_party/README-openclaw.md @@ -44,7 +44,7 @@ npm run build ## Usage Example Install or enable the local extension from the patched OpenClaw checkout, then -enable wrapping and ATIF export in the OpenClaw plugin config: +configure the NeMo Flow plugin host directly under the OpenClaw plugin config: ```json { @@ -53,21 +53,25 @@ enable wrapping and ATIF export in the OpenClaw plugin config: "nemo-flow": { "enabled": true, "config": { - "wrapping": { - "tools": true, - "llm": true - }, - "atif": { - "enabled": true - }, - "codecs": { - "default": "auto", - "providers": { - "anthropic": "anthropic_messages" - }, - "models": { - "gpt-5*": "openai_responses" + "version": 1, + "components": [ + { + "kind": "observability", + "enabled": true, + "config": { + "version": 1, + "atif": { + "enabled": true, + "agent_name": "openclaw", + "output_directory": "./nemo-flow-atif" + } + } } + ], + "policy": { + "unknown_component": "warn", + "unknown_field": "warn", + "unsupported_value": "error" } } } @@ -76,9 +80,16 @@ enable wrapping and ATIF export in the OpenClaw plugin config: } ``` -With that config, OpenClaw tool calls and LLM streams are routed through the -registered NeMo Flow middleware when the plugin is active. ATIF output is -written to the configured plugin state directory when sessions end. +With that config, the patched plugin initializes the NeMo Flow plugin host and +activates the `observability` component. Wrapping is implicit when the plugin is +enabled and initialized: the extension registers PI runtime streaming LLM and +tool-call middleware with OpenClaw. + +The patched plugin config is the canonical NeMo Flow plugin-host document. Old +wrapper keys are rejected, including `enabled`, `backend`, `capture`, +`correlation`, `plugins`, `nemoFlow`, `atif`, and `telemetry`. Configure +observability through component-local keys such as `atof`, `atif`, +`opentelemetry`, and `openinference`. ## Validation