Skip to content

fix: improve OpenClaw observability consistency#203

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
mnajafian-nv:fix/openclaw-observability-consistency
Jun 3, 2026
Merged

fix: improve OpenClaw observability consistency#203
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
mnajafian-nv:fix/openclaw-observability-consistency

Conversation

@mnajafian-nv
Copy link
Copy Markdown
Contributor

@mnajafian-nv mnajafian-nv commented Jun 2, 2026

Overview

This PR covers the OpenClaw-focused ATIF, ATOF, and OpenInference consistency work against the shared observability contract.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • Normalizes Anthropic /v1/messages content blocks so ATIF promotes tool_use blocks into tool calls while keeping readable assistant text and cache usage metrics.
  • Improves OpenInference display and usage extraction for Anthropic messages, OpenAI Responses, and Chat Completions cache-token shapes.
  • Preserves raw ATOF JSONL lifecycle payloads for /v1/messages, /v1/responses, and /v1/chat/completions.
  • Extends OpenClaw hook replay usage normalization for provider-specific token and cache fields.

Where should the reviewer start?

Start with crates/core/src/observability/atif.rs and crates/core/src/observability/openinference.rs, then review the new fixture coverage in the matching Rust tests and integrations/openclaw/test/llm-replay.test.ts.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Relates to OpenClaw observability consistency work.

Validation

  • cargo test -p nemo-relay
  • cargo test -p nemo-relay atif
  • cargo test -p nemo-relay openinference
  • cargo test -p nemo-relay atof
  • cargo fmt --all --check
  • env PYO3_PYTHON=/Users/mnajafian/projects/NeMo-Relay/.venv/bin/python cargo clippy --workspace --all-targets -- -D warnings
  • just test-rust
  • npm test --workspace=nemo-relay-openclaw
  • npm run typecheck --workspace=nemo-relay-openclaw
  • npm run test:live --workspace=nemo-relay-openclaw
  • npm run pack:check --workspace=nemo-relay-openclaw
  • uv run pre-commit run --files crates/core/src/observability/atif.rs crates/core/src/observability/openinference.rs crates/core/tests/unit/atif_tests.rs crates/core/tests/unit/observability/atof_tests.rs crates/core/tests/unit/observability/openinference_tests.rs integrations/openclaw/src/hook-replay/llm.ts integrations/openclaw/test/llm-replay.test.ts

Additional local validation

  • Ran OpenClaw + NeMo Relay examples locally with NVIDIA Inference and Tavily-backed tool use across diverse prompts.
  • Inspected generated ATIF, ATOF, OpenInference, and Phoenix traces for multi-step agent runs.

Follow-up Work

I’ll keep the remaining items as follow-ups rather than expanding this PR: flattened OpenInference message/tool attrs, OpenClaw ATOF provenance/fidelity audit, and nested-subagent fixture coverage if we have a real OpenClaw scenario.

Summary by CodeRabbit

  • New Features

    • Added support for Anthropic Claude message workflows including tool-use and function calling.
    • Extended OpenAI integration with response format support and cached token tracking.
    • Improved multi-provider LLM observability and message parsing.
  • Tests

    • Added test coverage for Anthropic and OpenAI LLM interaction scenarios.

Signed-off-by: mnajafian-nv <mnajafian@nvidia.com>
@mnajafian-nv mnajafian-nv added this to the 0.4 milestone Jun 2, 2026
@mnajafian-nv mnajafian-nv self-assigned this Jun 2, 2026
@mnajafian-nv mnajafian-nv requested a review from a team as a code owner June 2, 2026 23:03
@mnajafian-nv mnajafian-nv added the Bug issue describes bug; PR fixes bug label Jun 2, 2026
@github-actions github-actions Bot added size:L PR is large lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code labels Jun 2, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 2, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: f086e50a-fdff-4aa5-aba3-4a698b375fea

📥 Commits

Reviewing files that changed from the base of the PR and between b9390a0 and bec8787.

📒 Files selected for processing (1)
  • crates/core/src/observability/openinference.rs
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Check / Run
  • GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (12)
**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Rust identifiers (e.g., nemo_relay_tool_call)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: Run cargo fmt --all to format all Rust code
Run cargo clippy --workspace --all-targets -- -D warnings to enforce all clippy lints as errors

**/*.rs: Run cargo fmt --all when Rust files changed as part of WebAssembly work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files changed as part of WebAssembly work

**/*.rs: If any Rust code changed, always run just test-rust
If any Rust code changed, also run cargo fmt --all
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting with cargo fmt --all
Run Rust linting with cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Use cargo fmt for Rust code formatting
Run cargo clippy -- -D warnings to lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code with uv run pre-commit run --all-files to enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...

Files:

  • crates/core/src/observability/openinference.rs
crates/core/src/observability/{atif,otel,openinference}.rs

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in crates/core/src/observability/atif.rs, crates/core/src/observability/otel.rs, and crates/core/src/observability/openinference.rs in sync

Files:

  • crates/core/src/observability/openinference.rs
**/{Cargo.toml,**/*.rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Rust package names in Cargo.toml and their actual usage across the codebase

Files:

  • crates/core/src/observability/openinference.rs
**/*.{h,hpp,c,cpp,rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Ensure FFI header and library naming follows consistent conventions across platform-specific builds

Files:

  • crates/core/src/observability/openinference.rs
{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

  • crates/core/src/observability/openinference.rs
**/*.{rs,toml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Rust crate names and module prefixes during coordinated rename operations

Files:

  • crates/core/src/observability/openinference.rs
crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

crates/core/**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

Files:

  • crates/core/src/observability/openinference.rs
crates/{core,adaptive}/**

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly

Files:

  • crates/core/src/observability/openinference.rs
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.

Files:

  • crates/core/src/observability/openinference.rs
**/*.{rs,py,go,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python use snake_case, C FFI exports prefixed nemo_relay_, Go uses PascalCase for public APIs, Node.js uses camelCase.

Files:

  • crates/core/src/observability/openinference.rs
crates/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Use Json = serde_json::Value in Rust-facing runtime APIs for JSON payload handling.

Files:

  • crates/core/src/observability/openinference.rs
crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

  • crates/core/src/observability/openinference.rs
🔇 Additional comments (1)
crates/core/src/observability/openinference.rs (1)

1089-1089: LGTM!


Walkthrough

Adds Anthropic content/tool-use parsing into ATIF extraction, extends OpenInference to read nested cached-token fields and OpenAI output arrays for display text and tool detection, preserves wire-format LLM payloads in JSONL export, and normalizes diverse usage fields in replay logic with accompanying tests.

Changes

LLM Event Extraction and Replay

Layer / File(s) Summary
Anthropic message and tool-use extraction
crates/core/src/observability/atif.rs, crates/core/tests/unit/atif_tests.rs
extract_llm_response_message detects Anthropic content payloads and builds step messages from text blocks via anthropic_messages_content_message (empty for tool-only responses). extract_tool_calls adds Anthropic tool_use detection (anthropic_messages_tool_use_items) and expands argument extraction to args/input; tool_call_extra excludes input. Test asserts tool promotion and correlation.
OpenAI responses and cached token extraction
crates/core/src/observability/openinference.rs, crates/core/tests/unit/observability/openinference_tests.rs
display_text_from_json now prefers output arrays and aggregates message text and requested tools via new helpers. usage_from_manual_llm_output gains fallbacks into nested *_tokens_details.cached_tokens using new nested-u64 helpers. Tests validate display text, requested tools, and prompt cache read/write attribution for OpenAI and Anthropic payloads.
Wire-format LLM payload preservation in JSONL export
crates/core/tests/unit/observability/atof_tests.rs
New wire_format_llm_event helper and test ensure JSONL export preserves original wire-format LLM lifecycle provider payloads (Anthropic messages, OpenAI responses, chat completions) and metadata exactly.
Replay format normalization for diverse token fields
integrations/openclaw/src/hook-replay/llm.ts, integrations/openclaw/test/llm-replay.test.ts
mapUsage extended to consume additional token/cost field aliases (including nested cached-token sources) and uses nestedNumberField helper. Tests cover Anthropic message-write replay normalization and OpenAI usage normalization including cached and cache-read breakdowns.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed Title follows Conventional Commits format with 'fix' type and concise imperative summary under 72 characters.
Description check ✅ Passed Description includes all required sections: Overview with checklists, Details with specific changes, reviewer guidance, and related issues.
Docstring Coverage ✅ Passed Docstring coverage is 89.19% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@integrations/openclaw/test/llm-replay.test.ts`:
- Around line 107-234: The build fails due to missing Node type definitions in
the integrations/openclaw tests; add `@types/node` as a devDependency in
integrations/openclaw/package.json and/or install it in the workspace, then fix
the local tsconfig used for tests (update compilerOptions.types to include
"node" or remove the explicit types array so defaults apply) so TypeScript can
find Node types; after updating, reinstall deps and rerun npm test
--workspace=integrations/openclaw -- llm-replay.test.ts to validate the replay
normalization assertions exercised by tests using helpers like
createNemoRelayRuntime and createBackend in llm-replay.test.ts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b434f48b-2ad6-4661-a4b9-1c3aa4f250b1

📥 Commits

Reviewing files that changed from the base of the PR and between 603d337 and b9390a0.

📒 Files selected for processing (7)
  • crates/core/src/observability/atif.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • integrations/openclaw/src/hook-replay/llm.ts
  • integrations/openclaw/test/llm-replay.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Check / Run
  • GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (17)
**/*{test,spec,smoke}.{js,ts,py}

📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)

Relevant integration tests or smoke path must pass

Files:

  • integrations/openclaw/test/llm-replay.test.ts
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Maintain documented and tested validation and report behavior for adaptive surfaces

Files:

  • integrations/openclaw/test/llm-replay.test.ts
  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
**/*.{wasm,js,ts}{,x}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Ensure WebAssembly package naming conventions are consistent with generated package expectations and downstream consumption

Files:

  • integrations/openclaw/test/llm-replay.test.ts
  • integrations/openclaw/src/hook-replay/llm.ts
**/*.{js,ts,jsx,tsx}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Run Node.js formatting with npm run format --workspace=nemo-relay-node

Include SPDX license header in all JavaScript and TypeScript source files using double-slash comment syntax

Files:

  • integrations/openclaw/test/llm-replay.test.ts
  • integrations/openclaw/src/hook-replay/llm.ts
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.

Files:

  • integrations/openclaw/test/llm-replay.test.ts
  • integrations/openclaw/src/hook-replay/llm.ts
  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
**/*.{rs,py,go,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python use snake_case, C FFI exports prefixed nemo_relay_, Go uses PascalCase for public APIs, Node.js uses camelCase.

Files:

  • integrations/openclaw/test/llm-replay.test.ts
  • integrations/openclaw/src/hook-replay/llm.ts
  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Rust identifiers (e.g., nemo_relay_tool_call)

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: Run cargo fmt --all to format all Rust code
Run cargo clippy --workspace --all-targets -- -D warnings to enforce all clippy lints as errors

**/*.rs: Run cargo fmt --all when Rust files changed as part of WebAssembly work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files changed as part of WebAssembly work

**/*.rs: If any Rust code changed, always run just test-rust
If any Rust code changed, also run cargo fmt --all
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting with cargo fmt --all
Run Rust linting with cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Use cargo fmt for Rust code formatting
Run cargo clippy -- -D warnings to lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code with uv run pre-commit run --all-files to enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
**/{Cargo.toml,**/*.rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Rust package names in Cargo.toml and their actual usage across the codebase

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
**/*.{h,hpp,c,cpp,rs}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Ensure FFI header and library naming follows consistent conventions across platform-specific builds

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
{crates/core,crates/adaptive}/**/*

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
**/*.{rs,toml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Rust crate names and module prefixes during coordinated rename operations

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)

If the change touched crates/core or shared runtime semantics, also use validate-change for broader validation

crates/core/**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
crates/{core,adaptive}/**

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If crates/core or crates/adaptive changed, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
crates/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Use Json = serde_json::Value in Rust-facing runtime APIs for JSON payload handling.

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
  • crates/core/src/observability/atif.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/tests/unit/observability/openinference_tests.rs
crates/core/src/observability/{atif,otel,openinference}.rs

📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)

When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in crates/core/src/observability/atif.rs, crates/core/src/observability/otel.rs, and crates/core/src/observability/openinference.rs in sync

Files:

  • crates/core/src/observability/openinference.rs
  • crates/core/src/observability/atif.rs
🧠 Learnings (12)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-observability/SKILL.md:0-0
Timestamp: 2026-05-21T22:48:57.484Z
Learning: Applies to crates/core/src/observability/{atif,otel,openinference}.rs : When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in `crates/core/src/observability/atif.rs`, `crates/core/src/observability/otel.rs`, and `crates/core/src/observability/openinference.rs` in sync
📚 Learning: 2026-05-07T18:04:44.387Z
Learnt from: mnajafian-nv
Repo: NVIDIA/NeMo-Flow PR: 67
File: integrations/openclaw/src/modules.ts:1-2
Timestamp: 2026-05-07T18:04:44.387Z
Learning: In NVIDIA/NeMo-Flow, TypeScript source files should use `//` line comments for SPDX headers (e.g., `// SPDX-FileCopyrightText: ...` and `// SPDX-License-Identifier: ...`) rather than C-style block comments (`/* ... */`). The repo’s copyright checker enforces this mapping, so `//` SPDX headers in `.ts` files should not be flagged as a style violation.

Applied to files:

  • integrations/openclaw/test/llm-replay.test.ts
  • integrations/openclaw/src/hook-replay/llm.ts
📚 Learning: 2026-05-21T22:48:57.484Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-observability/SKILL.md:0-0
Timestamp: 2026-05-21T22:48:57.484Z
Learning: Applies to crates/core/src/observability/{atif,otel,openinference}.rs : When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in `crates/core/src/observability/atif.rs`, `crates/core/src/observability/otel.rs`, and `crates/core/src/observability/openinference.rs` in sync

Applied to files:

  • crates/core/tests/unit/atif_tests.rs
  • crates/core/tests/unit/observability/atof_tests.rs
  • crates/core/src/observability/openinference.rs
  • crates/core/src/observability/atif.rs
📚 Learning: 2026-05-21T22:51:04.073Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-go-binding/SKILL.md:0-0
Timestamp: 2026-05-21T22:51:04.073Z
Learning: Applies to crates/ffi/**/*.rs : If the change touched `crates/ffi`, also use `test-ffi-surface` for validation

Applied to files:

  • crates/core/tests/unit/atif_tests.rs
📚 Learning: 2026-05-21T22:48:57.484Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-observability/SKILL.md:0-0
Timestamp: 2026-05-21T22:48:57.484Z
Learning: When event fields change, run Rust crate tests and execute `just test-rust` to validate the affected observability modules

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-29T21:25:49.977Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-05-29T21:25:49.977Z
Learning: Run tests for every language affected by a change. If you touch the Rust core runtime, middleware semantics, event shape, scope behavior, typed codecs, plugins, or observability, expect to validate every affected binding because the bindings share the same runtime contract.

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:51:27.841Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-05-21T22:51:27.841Z
Learning: Use narrower crate tests as a local debug loop, not as the final validation story for Rust changes

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:47:58.898Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/add-middleware/SKILL.md:0-0
Timestamp: 2026-05-21T22:47:58.898Z
Learning: Applies to crates/core/src/api/runtime/state.rs : Add chain execution helpers to `NemoRelayContextState` following the pattern of existing methods like `tool_sanitize_request_chain` or `tool_request_intercepts_chain`

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:47:58.898Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/add-middleware/SKILL.md:0-0
Timestamp: 2026-05-21T22:47:58.898Z
Learning: Applies to crates/core/src/api/runtime/state.rs : Add registry fields as `SortedRegistry<GuardrailEntry<T>>` or `SortedRegistry<Intercept<T>>` to `NemoRelayContextState` in `crates/core/src/api/runtime/state.rs`

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:47:33.109Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/add-binding-feature/SKILL.md:0-0
Timestamp: 2026-05-21T22:47:33.109Z
Learning: Applies to crates/ffi/src/api/**/*.rs : Use `nemo_relay_` prefix for C FFI function names (e.g., `nemo_relay_tool_call`)

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:47:58.898Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/add-middleware/SKILL.md:0-0
Timestamp: 2026-05-21T22:47:58.898Z
Learning: Applies to crates/core/src/api/registry.rs : Use existing `global_*_registry_api!` and `scope_*_registry_api!` macro patterns in `crates/core/src/api/registry.rs` for both global and scope-local registration APIs unless design explicitly rules one out

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
📚 Learning: 2026-05-21T22:48:57.484Z
Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-observability/SKILL.md:0-0
Timestamp: 2026-05-21T22:48:57.484Z
Learning: Applies to docs/{about/concepts/subscribers,export-observability-data/about}.md : Update documentation under `docs/about/concepts/subscribers.md` and `docs/export-observability-data/about.md` to reflect lifecycle changes: create, register, run, deregister, flush, shutdown

Applied to files:

  • crates/core/tests/unit/observability/atof_tests.rs
🔇 Additional comments (11)
crates/core/src/observability/atif.rs (1)

490-520: LGTM!

Also applies to: 904-905, 932-933, 977-988, 1022-1022

crates/core/tests/unit/atif_tests.rs (1)

758-863: LGTM!

crates/core/src/observability/openinference.rs (1)

782-798: LGTM!

Also applies to: 857-879, 1027-1033, 1045-1093

crates/core/tests/unit/observability/openinference_tests.rs (3)

691-696: LGTM!

Also applies to: 717-720


723-804: LGTM!


1501-1596: LGTM!

integrations/openclaw/src/hook-replay/llm.ts (2)

1187-1210: LGTM!


1252-1256: LGTM!

integrations/openclaw/test/llm-replay.test.ts (2)

107-187: LGTM!


189-234: LGTM!

crates/core/tests/unit/observability/atof_tests.rs (1)

256-432: Run Rust formatting, clippy, and tests for this crates/core change

  • Run:
    • cargo fmt --all --check
    • cargo clippy --workspace --all-targets -- -D warnings
    • just test-rust
  • The current sandbox run can’t reach those checks: cargo fmt isn’t available (missing rustfmt component), just isn’t installed, and native builds fail in ring due to missing C system headers (assert.h, stdint.h).

Comment thread integrations/openclaw/test/llm-replay.test.ts
willkill07
willkill07 previously approved these changes Jun 2, 2026
Copy link
Copy Markdown
Member

@willkill07 willkill07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with minor feedback.

I think this is good. Out-of-scope for this PR would be to generalize/refactor to reduce maintenance burden. This was a little tough to review just because of the the logic tracing, though the solution is sound based on existing functionality and design.

Comment thread crates/core/src/observability/openinference.rs Outdated
Co-authored-by: Will Killian <2007799+willkill07@users.noreply.github.com>
Signed-off-by: mnajafian-nv <mnajafian@nvidia.com>
@mnajafian-nv
Copy link
Copy Markdown
Contributor Author

Thanks, agreed. I’ll keep this PR focused on the OpenClaw observability consistency fixes. After it lands, I can look at a follow-up to simplify/generalize the provider-format extraction paths so this is easier to maintain and review.

@willkill07
Copy link
Copy Markdown
Member

/merge

@rapids-bot rapids-bot Bot merged commit bb18d99 into NVIDIA:main Jun 3, 2026
122 of 124 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug issue describes bug; PR fixes bug lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code size:L PR is large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants