feat: add native OpenAI streaming - #201
Conversation
WalkthroughThe PR adds native OpenAI Chat Completions streaming through authenticated loopback NDJSON transport. It updates the Rust core, adapter lifecycle, Python SDK, schemas, tests, API references, and integration guidance while keeping Relay streaming separate. ChangesNative OpenAI streaming
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Runtime
participant OpenAIInvokeStream
participant NativeBinding
participant AdapterLifecycle
participant AdapterRuntime
Runtime->>OpenAIInvokeStream: create native stream
OpenAIInvokeStream->>NativeBinding: invoke_openai_stream with transport
NativeBinding->>AdapterLifecycle: send lifecycle request
AdapterLifecycle->>AdapterRuntime: invoke_openai_stream(payload, emit)
AdapterRuntime->>AdapterLifecycle: emit correlated chunks
AdapterLifecycle-->>OpenAIInvokeStream: deliver NDJSON chunks and end record
OpenAIInvokeStream-->>Runtime: return terminal RunResult
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Fern docs preview: https://nvidia-preview-pull-request-201.docs.buildwithfern.com/nemo/fabric |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@adapters/common/src/nemo_fabric_adapters/common/lifecycle.py`:
- Around line 604-611: Update the lifecycle cleanup block around writer.finish()
so adapter_error always takes precedence over a finalization failure, including
asyncio.CancelledError. When both errors exist, attach the finish exception as
context to adapter_error and re-raise adapter_error; only raise the finish
exception when no adapter error exists, and remove the stderr-only handling.
- Around line 298-370: The OpenAI chunk schema is missing the explicit uint64
maximum bound. Update the Rust schema source for the chunk’s created field to
enforce both 0 and u64::MAX, regenerate the schema snapshot, and add or update
shared tests covering values at u64::MAX and just beyond it.
In `@crates/fabric-core/src/schema.rs`:
- Around line 341-374: The test
openai_stream_schemas_freeze_transport_and_record_invariants must assert the
generated OpenAiStreamSink token schema’s pattern in addition to minLength. Add
an assertion on sink["properties"]["token"]["pattern"] requiring the exact
header-safe pattern ^[^\r\n]*\S[^\r\n]*$, while preserving the existing
invariants.
In `@docs/reference/api/python-library-reference/nemo_fabric.openai_streaming.md`:
- Line 22: Update the OpenAIInvokeStream source docstring in the relevant class
or method under python/src/nemo_fabric/openai_streaming.py to wrap result and
aclose in Markdown backticks instead of reStructuredText roles, then regenerate
the API reference using just docs; do not edit the generated Markdown directly.
In
`@docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunk.mdx`:
- Line 2: Update the Rust API-reference title-generation logic to render the
identifier segment OpenAi as OpenAI, then regenerate the affected references.
Ensure the generated titles are corrected in
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunk.mdx:2,
struct-openaichatcompletionchunkchoice.mdx:2,
struct-openaichatcompletionchunkdelta.mdx:2, and
struct-openaistreaminvocation.mdx:2, producing “OpenAI” with the existing
spacing and title structure.
In
`@docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdx`:
- Around line 4-14: Update the OpenAiStreamSink documentation comment in
runtime.rs to say “generated by NVIDIA NeMo Fabric,” replacing the current
abbreviated product wording. Then run just docs and regenerate the schema
snapshots so the generated API pages and openai-stream-invocation.schema.json
reflect the updated description.
In `@python/src/nemo_fabric/runtime.py`:
- Line 76: Define a shared stream protocol near the runtime type declarations
with _finalized: bool, _task: asyncio.Task[Any], and an aclose() method, then
annotate _current_stream with that protocol instead of the InvokeStream |
OpenAIInvokeStream union. Ensure both stream implementations satisfy the
protocol so Runtime’s existing accesses remain type-checked.
In `@skills/nemo-fabric-integrate/references/sdk-api-inventory.md`:
- Around line 68-71: Update the stream-finalization guidance to document await
stream.result() as a valid way to finalize OpenAIInvokeStream before starting
another turn, including that it drains unread chunks. Preserve the requirement
to fully consume or await aclose() for other streams, and explicitly state that
Relay InvokeStream.result() does not consume unread ATOF records.
In `@tests/adapters/test_adapters_common_lifecycle.py`:
- Around line 282-334: Extend the parameter list in
test_common_host_rejects_chunks_outside_the_declared_openai_profile with invalid
chunks covering a wrong object discriminator, non-list choices, non-mapping
delta, and malformed tool_calls, logprobs, and usage. Keep each case consistent
with the existing minimal chunk structure and verify they all raise
LifecycleError with code lifecycle_invalid_openai_stream_event through
_validated_openai_chunk.
In `@tests/python/test_openai_streaming.py`:
- Line 19: Rename the conflicting parameter named openai_streaming in
_runtime_wrapper and
test_native_and_relay_streaming_capabilities_are_independent to
native_streaming, updating all references and preserving the existing behavior
while keeping the module alias unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 2b2ee309-c94e-4ac7-9b61-f86b57be0f1a
📒 Files selected for processing (74)
README.mdadapters/common/README.mdadapters/common/src/nemo_fabric_adapters/common/lifecycle.pycrates/fabric-core/src/error.rscrates/fabric-core/src/lib.rscrates/fabric-core/src/runtime.rscrates/fabric-core/src/schema.rscrates/fabric-python/src/lib.rsdocs/adapter-contract/conformance.mddocs/adapter-contract/execution.mddocs/index.ymldocs/reference/api/python-library-reference/index.mddocs/reference/api/python-library-reference/nemo_fabric.openai_streaming.mddocs/reference/api/python-library-reference/nemo_fabric.runtime.mddocs/reference/api/python-library-reference/nemo_fabric.streaming.mddocs/reference/api/rust-library-reference/nemo-fabric-core/adapter-contract/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/agent-config/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/agent-execution/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/doctor/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/fn-version.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/constant-openai-chat-completions-chunk-profile.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/constant-openai-stream-host.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/constant-openai-stream-protocol-version.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-errorstage.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaichatcompletionchunkobject.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamhost.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamprofile.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamprotocolversion.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-runstatus.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-invoke-openai-stream.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-invoke-runtime.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-prepare-environment.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-run-plan.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-start-runtime.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/fn-stop-runtime.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunk.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunkchoice.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunkdelta.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreaminvocation.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamtransport.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-runrequest.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-runresult.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-runtimecontext.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-runtimehandle.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-runtimetelemetrycontext.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-telemetryref.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/schema/enum-schemaname.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/schema/index.mdxdocs/sdk/python.mdxpython/src/nemo_fabric/__init__.pypython/src/nemo_fabric/_native.pyipython/src/nemo_fabric/openai_streaming.pypython/src/nemo_fabric/runtime.pyschemas/SCHEMA.mdschemas/adapter-contract/legacy/openai-stream-invocation.schema.jsonschemas/adapter-contract/legacy/openai-stream-record.schema.jsonscripts/docs/enhance_python_api_reference.pyscripts/generate_api_docs.shskills/nemo-fabric-build-adapter/SKILL.mdskills/nemo-fabric-integrate/SKILL.mdskills/nemo-fabric-integrate/references/sdk-api-inventory.mdtests/adapters/test_adapters_common_lifecycle.pytests/docs/test_python_api_docs.pytests/fixtures/hermes-shim-agent/adapters/hermes-shim/fabric-adapter.jsontests/fixtures/hermes-shim-agent/adapters/hermes-shim/src/nemo_fabric_test_adapters/hermes_shim/adapter.pytests/python/test_native_sdk.pytests/python/test_openai_streaming.py
| self._status = RuntimeStatus.ACTIVE | ||
| self._current_task: asyncio.Task[Any] | None = None | ||
| self._current_stream: InvokeStream | None = None | ||
| self._current_stream: InvokeStream | OpenAIInvokeStream | None = None |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Make the shared stream contract explicit.
Runtime now reaches into _finalized and _task on both InvokeStream and OpenAIInvokeStream at lines 151, 275, 317, 366, and 433. These are private attributes of two unrelated classes. No base class or protocol enforces that both keep them. A rename in either class breaks the other call sites, and a type checker cannot catch it because the union permits attribute access on both members.
Define a small Protocol with _finalized: bool, _task: asyncio.Task[Any], and aclose(), and annotate _current_stream with it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/src/nemo_fabric/runtime.py` at line 76, Define a shared stream
protocol near the runtime type declarations with _finalized: bool, _task:
asyncio.Task[Any], and an aclose() method, then annotate _current_stream with
that protocol instead of the InvokeStream | OpenAIInvokeStream union. Ensure
both stream implementations satisfy the protocol so Runtime’s existing accesses
remain type-checked.
4c4539c to
0368c79
Compare
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/fabric-core/src/runtime.rs`:
- Around line 1332-1363: Extract the two inline streaming redaction statements
in the LocalHostInvocation::OpenAiStream branch into a named
redact_openai_stream_invocation helper placed beside redact_adapter_invocation.
Have the helper redact the runtime context environment and replace stream.token
with the redacted value, then call it for persisted streaming invocations while
preserving the existing lifecycle payload behavior.
In
`@docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdx`:
- Around line 24-68: Update the Rust reference generator in the variant-field
rendering logic of scripts/docs/generate_rust_library_reference.py so fields
under each enum variant’s `#### Fields` label use `#####` headings instead of
`###`, preserving proper nesting and unique variant-scoped anchors. Regenerate
the documentation with `just docs` and verify the generated Chunk and End
sections reflect the corrected hierarchy.
In `@python/src/nemo_fabric/runtime.py`:
- Around line 116-121: Update Runtime.supports_openai_streaming to require both
_plan.capabilities.streaming and the resolved adapter descriptor’s streaming
capability, matching the core contract and the existing guard. In
tests/python/test_openai_streaming.py lines 40-52, add
adapter_descriptor.descriptor.capabilities to _plan, parameterize it
independently from the plan capability, and verify a plan-only capability raises
FabricCapabilityError with code openai_streaming_unavailable before any native
call.
In `@scripts/docs/generate_rust_library_reference.py`:
- Line 512: Add an inline comment next to the title normalization in the
relevant generation flow documenting that the regex targets “Open Ai” only when
followed by an uppercase letter, inserts a single trailing space, and leaves
terminal “Open Ai” unchanged.
In `@tests/adapters/test_adapters_common_lifecycle.py`:
- Around line 132-136: Update the three listener.records.get() awaits in the
lifecycle test to use asyncio.wait_for with the file’s established timeout
pattern, ensuring each read fails promptly if no record is emitted while
preserving the existing record collection behavior.
In `@tests/python/test_openai_streaming.py`:
- Around line 40-52: Update the plan fixture’s adapter_descriptor.descriptor to
include an explicit capabilities block with streaming enabled, while retaining
the existing top-level capabilities.streaming parameter. Add or adjust coverage
so a plan advertising streaming is also tested against a descriptor without
streaming, verifying the descriptor capability rule is enforced.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: a470239a-0912-451e-8132-5b34a5667402
📒 Files selected for processing (24)
adapters/common/src/nemo_fabric_adapters/common/lifecycle.pycrates/fabric-core/src/runtime.rscrates/fabric-core/src/schema.rsdocs/reference/api/python-library-reference/nemo_fabric.openai_streaming.mddocs/reference/api/python-library-reference/nemo_fabric.runtime.mddocs/reference/api/python-library-reference/nemo_fabric.streaming.mddocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaichatcompletionchunkobject.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamhost.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamprofile.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamprotocolversion.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunk.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunkchoice.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaichatcompletionchunkdelta.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreaminvocation.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamtransport.mdxpython/src/nemo_fabric/openai_streaming.pypython/src/nemo_fabric/runtime.pypython/src/nemo_fabric/streaming.pyschemas/adapter-contract/legacy/openai-stream-record.schema.jsonscripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pytests/python/test_openai_streaming.py
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
0368c79 to
8c3c5a3
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/fabric-core/src/runtime.rs`:
- Around line 1230-1243: Update run_local_host_openai_stream_adapter to avoid
the fixed local_host_invoke_timeout deadline for active streams: use a
streaming-specific timeout and ensure each received stream chunk resets or
extends the terminal-response deadline in exchange_lifecycle_message. Preserve
the existing timeout behavior for non-streaming invocations while allowing
healthy streams to continue as chunks arrive.
In `@python/src/nemo_fabric/runtime.py`:
- Around line 330-334: Extract the repeated active-stream validation into a
private _ensure_no_active_stream method on the containing class, preserving the
existing _current_stream/_finalized condition and FabricStateError message.
Replace the guards in invoke, invoke_stream, and the current method with calls
to this helper.
In `@tests/python/test_openai_streaming.py`:
- Around line 497-502: Update the local wait_for_end function to use a small
non-zero asyncio.sleep delay while polling stream._end_observed, replacing the
zero-delay yield; keep the existing wait_for_end polling behavior and one-second
asyncio.wait_for timeout unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: f407ed28-62a8-4da2-a9ad-a88098095427
📒 Files selected for processing (12)
crates/fabric-core/src/runtime.rsdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxpython/src/nemo_fabric/runtime.pyschemas/adapter-contract/legacy/openai-stream-invocation.schema.jsonscripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pytests/python/test_openai_streaming.py
📜 Review details
⏰ Context from checks skipped due to timeout. (17)
- GitHub Check: Test (Python 3.13, linux-amd64)
- GitHub Check: Test (Python 3.11, linux-amd64)
- GitHub Check: Test (Python 3.13, windows-amd64)
- GitHub Check: Test (Python 3.11, macos-arm64)
- GitHub Check: Test (Python 3.11, linux-arm64)
- GitHub Check: Test (Python 3.14, macos-arm64)
- GitHub Check: Test (Python 3.13, macos-arm64)
- GitHub Check: Test (Python 3.11, windows-amd64)
- GitHub Check: Test (Python 3.12, windows-amd64)
- GitHub Check: Test (Python 3.12, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-amd64)
- GitHub Check: Test (Python 3.14, linux-arm64)
- GitHub Check: Test (Python 3.13, linux-arm64)
- GitHub Check: Test (Python 3.12, linux-arm64)
- GitHub Check: Test (Python 3.14, windows-amd64)
- GitHub Check: Pre-commit
- GitHub Check: Test (arm64)
🧰 Additional context used
📓 Path-based instructions (36)
**/*.{rs,py,pyi,json,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Determine and update every affected public surface, including the CLI, PyO3 bindings, Python SDK, type stubs, schemas, and adapter contract, so they remain in parity.
Files:
scripts/docs/generate_rust_library_reference.pyschemas/adapter-contract/legacy/openai-stream-invocation.schema.jsontests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
scripts/docs/generate_rust_library_reference.pyschemas/adapter-contract/legacy/openai-stream-invocation.schema.jsondocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxtests/adapters/test_adapters_common_lifecycle.pydocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxpython/src/nemo_fabric/runtime.pydocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxtests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rsdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
**/*.{rs,py}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
For native binding changes, run
cargo check -p fabric-python --locked.Use
snake_casefor functions and variables; usePascalCasefor Rust types and Python classes.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
**/*.{py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If Python code or a Python-facing adapter changes, run
just test-python.In Python SDK, adapters, examples, and tests, follow the existing style, use type annotations for public APIs, and keep native binding declarations synchronized with their Rust implementations.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.py
**/*.{rs,py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,pyi}: If public configuration types change, confirm schema snapshot tests injust test-rustpass and review generated schema diffs.
For schema or public contract changes, run both language suites and review changes underschemas/and generated API references.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
**/*.{py,pyi,rs}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For Python SDK or PyO3 binding changes, use
python-tests, run focused pytest tests first, thenjust test-python; rebuild withjust build-pythonwhen native code or packaging changes.Public contract changes must keep native Python binding declarations synchronized with their Rust implementations.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
**/*.{rs,py,toml}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
When editing version helpers, verify every
nemo-fabric-*workspace package through Cargo metadata and reject a static version inpython/pyproject.toml.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
**/*.{toml,rs,py}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
Avoid blind repository-wide replacement of version-like strings; distinguish package-version references from examples and unrelated dependency versions.
Files:
scripts/docs/generate_rust_library_reference.pytests/adapters/test_adapters_common_lifecycle.pypython/src/nemo_fabric/runtime.pytests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rs
scripts/docs/generate_rust_library_reference.py
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
For Rust API reference changes, update
scripts/docs/generate_rust_library_reference.pywhen the generator itself must change.
Files:
scripts/docs/generate_rust_library_reference.py
**/*.{md,mdx,yml,py,rs,sh}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Keep documentation aligned with current NeMo Fabric behavior, repository layout, entry points, commands, package names, APIs, bindings, and support claims.
Files:
scripts/docs/generate_rust_library_reference.pydocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxtests/adapters/test_adapters_common_lifecycle.pydocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxpython/src/nemo_fabric/runtime.pydocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxtests/python/test_openai_streaming.pycrates/fabric-core/src/runtime.rsdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
**/*.{json,jsonschema}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Public contract changes must keep checked-in JSON Schema snapshots synchronized.
Files:
schemas/adapter-contract/legacy/openai-stream-invocation.schema.json
schemas/**/*
⚙️ CodeRabbit configuration file
schemas/**/*: Schemas are generated public contract snapshots. Check that schema diffs correspond to intentional Rust type changes and are covered by core tests.
Files:
schemas/adapter-contract/legacy/openai-stream-invocation.schema.json
{README.md,docs/**/*.{md,mdx,yml},examples/**/*.{md,mdx,yml}}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repository references, and build commands current in documentation and examples.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
{docs/**/*.{md,mdx,yml},examples/**/*.{md,mdx,yml}}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Update relevant getting-started, reference, adapter, and example documentation when the corresponding examples or adapters change.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
In MDX files, use JSX comment delimiters (
{/*and*/}) for top-of-file comments, including SPDX headers; do not use HTML comments.
**/*.mdx: For documentation site changes, runjust docsto regenerate Python and Rust API references and validate Fern configuration.
MDX files must use the specified JSX-comment SPDX header.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
docs/**/*.{md,mdx,yml}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Run
just docswhen the documentation site changes.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
docs/reference/api/**/*
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Treat all files under
docs/reference/api/as generated output and do not modify them directly.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
docs/**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
docs/**/*.mdx: Use source-relative links with the target.mdxextension for links between files underdocs/; do not use Fern site-root paths.
Use{/* ... */}delimiters for top-of-file MDX SPDX comments, not HTML comment delimiters.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon first use, typically in the title and H1; useNeMo Fabricthereafter. Usefabricalone only for the CLI tool and surround it with backticks.
Treat incorrect or stale commands, package names, paths, APIs, support claims, procedures, examples, terminology, or public behavior documentation as blocking issues.
CapitalizeNVIDIAcorrectly and format code, commands, paths, and filenames as inline code where needed.
Use title case for technical-documentation headings.
Introduce code blocks, tables, and lists with complete lead-in sentences; ensure examples match current APIs and build commands.
Use descriptive anchor text, avoid raw URLs and generic labels such ashere, and use repository-relative.mdxpaths for links withindocs/.
Prefer active voice, present tense, short sentences, plain English, consistent terminology, and imperative, parallel, scannable procedures.
Useafterinstead ofoncewhen expressing temporal sequence, and usecanrather thanmaywhen describing possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
When reporting documentation-review findings, lead withMust fix,Should fix, andNice to havecategories; include file path, line reference, current problem, rationale, and a concrete rewrite or direction.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
docs/**
📄 CodeRabbit inference engine (AGENTS.md)
Run just docs after changing the documentation site.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
{docs/**,README.md,AGENTS.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,AGENTS.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency with generated schemas.
For links between files under docs/, require paths relative to the source file with the target file's .mdx extension so they work in both Fern builds and repository browsers. Flag Fern site-root links such as NeMo Fabric overview; use the repository-relative equivalent, such as NeMo Fabric overview.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
{*.md,**/*.md,**/*.mdx,**/*.ipynb}
⚙️ CodeRabbit configuration file
{*.md,**/*.md,**/*.mdx,**/*.ipynb}: Enforce the product name in user-facing prose: use "NVIDIA NeMo Fabric" on first use and "NeMo Fabric" thereafter. Flag standalone capitalized "Fabric" when it refers to the product. Do not flag the lowercasefabricCLI command, package/import/crate names, code identifiers, API symbols, configuration keys, file paths, or unrelated generic uses of the word.
Files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
tests/adapters/**/*.py
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
tests/adapters/**/*.py: If an adapter or integration changes, run its focused tests.
For adapter behavior changes, run focused adapter tests undertests/adapters, then runjust test-python.
Files:
tests/adapters/test_adapters_common_lifecycle.py
tests/**/*.{rs,py}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When adding functionality, include tests in the corresponding Rust crate or the relevant area under
tests/.
Files:
tests/adapters/test_adapters_common_lifecycle.pytests/python/test_openai_streaming.py
tests/**/*.py
📄 CodeRabbit inference engine (.agents/skills/python-tests/SKILL.md)
tests/**/*.py: Use pytest to run Python tests.
Do not add@pytest.mark.asyncioto tests; async tests are automatically detected by the async runner.
Do not add-> Nonereturn type annotations to test functions.
When mocking a class, useunittest.mock.MagicMockorAsyncMock, using thespecargument when necessary, rather than defining a new class.
Prefix mocked class names withmock, notfake.
Prefer pytest fixtures over helper methods.
If a fixture is needed in multiple test files, define it once inconftest.pyrather than repeating it.
Define fixtures using@pytest.fixture(name="<fixture_name>"[, scope="<scope>"])and a<fixture_name>_fixturefunction; specifyscopeonly when it is notfunction.
Preferpytest.mark.parametrizeover separate tests for different input types.
Use@pytest.mark.usefixtureswhen a fixture is needed but its returned value is unused or it returns no value.
Avoid defensive programming in tests; access expected values directly so missing data raises a clear failure, such as usingresults["data"]instead ofresults.get("data").
When adapter installation metadata changes, packaging metadata tests must directly assert that the root project depends unconditionally on the exact-versionnemo-fabric-runtimedistribution.
Packaging metadata tests must verify that each root harness extra delegates to the matching version of the leaf adapter'sharnessextra.
Packaging metadata tests must verify that bare leaf dependencies remain adapter-owned and that the rootadapter-testsdependency group installs each leaf through itsharnessextra.
Packaging metadata tests must verify that every leaf providesfull; only adapters importing NeMo Relay Python APIs providerelay, while adapters using an external Relay executable havefullequal toharness.
Files:
tests/adapters/test_adapters_common_lifecycle.pytests/python/test_openai_streaming.py
{tests/**,python/tests/**}
⚙️ CodeRabbit configuration file
{tests/**,python/tests/**}: Tests should cover the behavior promised by the changed API surface, including error paths, lifecycle cleanup, and SDK/native parity where relevant.
Files:
tests/adapters/test_adapters_common_lifecycle.pytests/python/test_openai_streaming.py
python/src/nemo_fabric/**/*.py
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
For Python API reference changes, update source docstrings under
python/src/nemo_fabric/instead of generated API reference files.
Files:
python/src/nemo_fabric/runtime.py
python/src/nemo_fabric/**/*.{py,rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure native extension naming and placement remain compatible with downstream consumers, including the editable maturin build producing
nemo_fabric._native.
Files:
python/src/nemo_fabric/runtime.py
python/src/nemo_fabric/**/*
⚙️ CodeRabbit configuration file
python/src/nemo_fabric/**/*: Review Python SDK changes for typed API consistency, import-time dependency neutrality, async/session behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/src/nemo_fabric/runtime.py
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Implement new runtime or binding behavior in the shared Rust core first.
Files:
crates/fabric-core/src/runtime.rs
**/*.{rs,toml}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
For any Rust change, run
just test-rustandcargo fmt --all -- --check.For Rust core, CLI, or shared runtime semantic changes, run Rust formatting and tests, and add Python tests when behavior is exposed through the SDK.
Use Rust stable tooling; format Rust code with
cargo fmt --all, verify formatting withcargo fmt --all -- --check, and compile withcargo check --workspace --locked.
Files:
crates/fabric-core/src/runtime.rs
crates/fabric-core/**/*.{rs,py}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes under
crates/fabric-coremust run both the Rust and Python test suites.
Files:
crates/fabric-core/src/runtime.rs
**/*.{rs,rmeta}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If Rust code changes, run
cargo fmt --all -- --checkandjust test-rust.
Files:
crates/fabric-core/src/runtime.rs
crates/fabric-core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/fabric-corechanges in a way exposed through Python, run both the Rust and Python suites.For Rust API reference changes, update Rust documentation comments under
crates/fabric-core/instead of generated API reference files.
Files:
crates/fabric-core/src/runtime.rs
crates/fabric-core/src/**/*.rs
⚙️ CodeRabbit configuration file
crates/fabric-core/src/**/*.rs: Review the Rust core for runtime lifecycle correctness, handle validation, capability routing accuracy, schema stability, and error semantics.
Public API changes should match committed schemas, tests, and documentation.
Files:
crates/fabric-core/src/runtime.rs
🧠 Learnings (3)
📚 Learning: 2026-08-07T07:15:33.918Z
Learnt from: AnuradhaKaruppiah
Repo: NVIDIA/NeMo-Fabric PR: 186
File: schemas/adapter-contract/legacy/adapter-invocation.schema.json:176-176
Timestamp: 2026-08-07T07:15:33.918Z
Learning: For the NeMo Fabric v1alpha southbound adapter contract, treat the adapter descriptor's `contract_version` as the version of the complete contract, including `RuntimeContext`. Keep `RuntimeContext` strict by rejecting unknown properties, and require a negotiated contract-version change for additive shape changes.
Applied to files:
schemas/adapter-contract/legacy/openai-stream-invocation.schema.json
📚 Learning: 2026-07-24T16:07:22.255Z
Learnt from: AjayThorve
Repo: NVIDIA/NeMo-Fabric PR: 118
File: docs/reference/api/rust-library-reference/nemo-fabric-core/config/struct-adapterdescriptor.mdx:5-5
Timestamp: 2026-07-24T16:07:22.255Z
Learning: In this repo, files generated under `docs/reference/api/**` are NVIDIA NeMo Fabric API reference output. When reviewing changes to these generated pages, do not treat sidebar `position`/ordering updates as direct manual edits—these can be regenerated by running `just docs` after adding public types. Only flag substantive content changes that are not explained by generation.
Applied to files:
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdxdocs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx
📚 Learning: 2026-06-28T04:03:32.877Z
Learnt from: AjayThorve
Repo: NVIDIA/NeMo-Fabric PR: 26
File: python/tests/smoke_typed_config.py:163-177
Timestamp: 2026-06-28T04:03:32.877Z
Learning: In NVIDIA NeMo Fabric Python SDK serialization of `RuntimeCapabilities` (to satisfy the “parity contract” with Rust core and the CLI), do not emit metadata keys when the corresponding metadata is absent. Instead, omit those fields entirely so the produced JSON matches the Rust/CLI output (e.g., avoid `null`, empty objects, or placeholder metadata). During review, verify the serializer/builders follow this omission rule and that Python outputs/parity tests reflect the same shape.
Applied to files:
python/src/nemo_fabric/runtime.py
🪛 ast-grep (0.45.1)
python/src/nemo_fabric/runtime.py
[info] 194-194: use jsonify instead of json.dumps for JSON output
Context: json.dumps(self._runtime.to_mapping())
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 195-195: use jsonify instead of json.dumps for JSON output
Context: json.dumps(payload)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 207-207: use jsonify instead of json.dumps for JSON output
Context: json.dumps(dict(openai_stream_transport))
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
tests/python/test_openai_streaming.py
[info] 206-206: use jsonify instead of json.dumps for JSON output
Context: json.dumps(record, separators=(",", ":"))
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 224-224: use jsonify instead of json.dumps for JSON output
Context: json.dumps(record, separators=(",", ":"))
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 253-253: use jsonify instead of json.dumps for JSON output
Context: json.dumps(_result(request, runtime, invocation_id=invocation_id))
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 263-269: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(
request,
runtime,
invocation_id=f"invocation-{len(mock_native.requests)}",
)
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 327-329: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-empty")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 365-367: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-many")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 385-387: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-without-stream")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 417-419: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-after-probe")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 453-455: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-late-stream")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 483-489: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(
request,
runtime,
invocation_id="invocation-cancel-after-end",
)
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 556-563: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(
request,
runtime,
invocation_id="invocation-failed",
failed=True,
)
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 586-588: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_result(request, runtime, invocation_id="invocation-terminal")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 675-675: use jsonify instead of json.dumps for JSON output
Context: json.dumps(record)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 689-689: use jsonify instead of json.dumps for JSON output
Context: json.dumps(nonfinite)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 703-703: use jsonify instead of json.dumps for JSON output
Context: json.dumps(_record())
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 706-712: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_record(
sequence=1,
invocation_id="invocation-other",
chunk=_chunk("chunk-2", "other"),
)
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[info] 796-798: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
_record(record_type="end"), separators=(",", ":")
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🪛 LanguageTool
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdx
[grammar] ~26-~26: Ensure spelling is correct
Context: ...reaminvocation.mdx): One adapter-native OpenAI streaming invocation. - [OpenAiStreamSink](struct-openaistrea...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[grammar] ~41-~41: Ensure spelling is correct
Context: ...num-openaistreamprofile.mdx): Supported OpenAI-compatible chunk profile. - [OpenAiStreamProtocolVersion](enum-openai...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[style] ~50-~50: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...for adapter-native OpenAI streaming. - [OPENAI_STREAM_PROTOCOL_VERSION](constant-opena...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[grammar] ~50-~50: Ensure spelling is correct
Context: ...und protocol version for adapter-native OpenAI streaming. ## Functions - [invoke_openai_stream](fn-i...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 Ruff (0.16.1)
tests/adapters/test_adapters_common_lifecycle.py
[warning] 94-94: Missing return type annotation for private function start
Add return type annotation: None
(ANN202)
[warning] 97-97: Missing return type annotation for private function invoke
Add return type annotation: Never
(ANN202)
[warning] 98-98: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 100-100: Missing return type annotation for private function invoke_openai_stream
(ANN202)
[warning] 122-122: Missing return type annotation for private function stop
Add return type annotation: None
(ANN202)
[warning] 251-251: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 255-255: Missing return type annotation for private function open_connection
(ANN202)
[warning] 255-255: Missing type annotation for *_args
(ANN002)
[warning] 255-255: Missing type annotation for **_kwargs
(ANN003)
[warning] 290-290: Missing return type annotation for private function connect
(ANN202)
[warning] 299-299: Missing return type annotation for private function invoke_openai_stream
Add return type annotation: Never
(ANN202)
[warning] 341-341: Missing return type annotation for private function start
Add return type annotation: None
(ANN202)
[warning] 344-344: Missing return type annotation for private function invoke
(ANN202)
[warning] 347-347: Missing return type annotation for private function stop
Add return type annotation: None
(ANN202)
[warning] 432-432: Missing return type annotation for private function start
Add return type annotation: None
(ANN202)
[warning] 435-435: Missing return type annotation for private function invoke
Add return type annotation: None
(ANN202)
[warning] 438-438: Missing return type annotation for private function stop
Add return type annotation: None
(ANN202)
python/src/nemo_fabric/runtime.py
[error] 307-307: Function argument input is shadowing a Python builtin
(A002)
[warning] 307-307: Dynamically typed expressions (typing.Any) are disallowed in input
(ANN401)
[warning] 324-329: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 331-334: Avoid specifying long messages outside the exception class
(TRY003)
tests/python/test_openai_streaming.py
[warning] 19-19: Use from nemo_fabric import openai_streaming in lieu of alias
Replace with from nemo_fabric import openai_streaming
(PLR0402)
[warning] 125-125: Dynamically typed expressions (typing.Any) are disallowed in sequence
(ANN401)
[warning] 143-143: Dynamically typed expressions (typing.Any) are disallowed in stream
(ANN401)
[warning] 185-185: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 318-318: Missing return type annotation for private function invoke_empty
(ANN202)
[warning] 356-356: Missing return type annotation for private function invoke_many
(ANN202)
[warning] 383-383: Missing return type annotation for private function invoke_without_stream
(ANN202)
[warning] 406-406: Missing return type annotation for private function invoke_after_probe
(ANN202)
[error] 410-410: Possible hardcoded password assigned to argument: "token"
(S106)
[warning] 436-436: Missing return type annotation for private function invoke_before_stream
(ANN202)
[warning] 473-473: Missing return type annotation for private function invoke_after_end
(ANN202)
[warning] 498-499: Use asyncio.Event instead of awaiting asyncio.sleep in a while loop
(ASYNC110)
[warning] 547-547: Missing return type annotation for private function invoke_failed
(ANN202)
[warning] 577-577: Missing return type annotation for private function invoke_mismatch
(ANN202)
[warning] 718-718: Missing return type annotation for private function invoke_without_end
Add return type annotation: Never
(ANN202)
[warning] 731-731: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 743-743: Missing return type annotation for private function invoke_invalid_chunk
Add return type annotation: Never
(ANN202)
[warning] 759-759: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 781-781: Missing return type annotation for private function candidate
(ANN202)
[warning] 846-846: Boolean-typed positional argument in function definition
(FBT001)
[warning] 847-847: Boolean-typed positional argument in function definition
(FBT001)
[warning] 887-887: Missing return type annotation for private function fail_create_task
Add return type annotation: Never
(ANN202)
[warning] 889-889: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 891-891: Missing return type annotation for private function invoke
Add return type annotation: Never
(ANN202)
[warning] 892-892: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 920-920: Missing return type annotation for private function invoke_with_bad_token
Add return type annotation: Never
(ANN202)
[error] 929-929: Possible hardcoded password assigned to argument: "token"
(S106)
[warning] 931-931: Avoid specifying long messages outside the exception class
(TRY003)
🔇 Additional comments (35)
tests/adapters/test_adapters_common_lifecycle.py (5)
365-410: Validation branches in_validated_openai_chunkremain uncovered.The parametrized list covers missing
model, booleanindex, blankid, blankmodel,createdoverflow, andindexoverflow._validated_openai_chunkalso rejects a wrongobjectdiscriminator, a non-listchoices, a non-mappingdelta, and malformedtool_calls,logprobs, andusage. Those branches have no case here. This repeats an earlier review note.
15-15: LGTM!Also applies to: 30-78
80-157: LGTM!Also applies to: 160-190, 193-228
231-268: LGTM!Also applies to: 270-312, 314-363
418-450: LGTM!crates/fabric-core/src/runtime.rs (5)
40-46: LGTM!Also applies to: 349-560, 564-587, 618-627, 675-681
823-869: LGTM!
1245-1297: LGTM!Also applies to: 1332-1362
2130-2160: LGTM!
2912-3009: LGTM!Also applies to: 3076-3145, 3237-3394
python/src/nemo_fabric/runtime.py (4)
25-25: LGTM!Also applies to: 76-76
176-178: LGTM!Also applies to: 194-216, 250-251
304-329: LGTM!Also applies to: 335-352, 382-383
116-133: 🩺 Stability & AvailabilityKeep the current
adapter_descriptorlookup.RunPlaninheritsMapping.getand preservesadapter_descriptoras an extension field. The typedadapterfield does not include nestedcapabilities.> Likely an incorrect or invalid review comment.tests/python/test_openai_streaming.py (7)
1-62: LGTM!Also applies to: 65-140
143-229: LGTM!Also applies to: 231-295
298-403: LGTM!Also applies to: 405-469
472-496: LGTM!Also applies to: 503-543, 546-598
601-714: LGTM!Also applies to: 717-769
772-837: LGTM!Also applies to: 840-881
884-941: LGTM!schemas/adapter-contract/legacy/openai-stream-invocation.schema.json (2)
150-239: LGTM!Also applies to: 344-368
1-149: LGTM!Also applies to: 240-343
scripts/docs/generate_rust_library_reference.py (2)
399-401: LGTM!
514-514: LGTM!docs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatifstorageconfig.mdx (1)
26-42: LGTM!Also applies to: 54-86
docs/reference/api/rust-library-reference/nemo-fabric-core/config/enum-relayatofsinkconfig.mdx (1)
26-38: LGTM!Also applies to: 50-78
docs/reference/api/rust-library-reference/nemo-fabric-core/error/enum-fabricerror.mdx (2)
12-12: LGTM!Also applies to: 296-327
26-294: LGTM!Also applies to: 336-514
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/enum-openaistreamrecord.mdx (2)
24-68: The heading hierarchy is now correct:### Chunk→#### Fields→##### sequence: u64. The generator change atscripts/docs/generate_rust_library_reference.pylines 399-401 resolves the previously reported heading skip.
1-23: LGTM!Also applies to: 70-143
docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/index.mdx (2)
5-5: LGTM!Also applies to: 23-44
46-54: 🎯 Functional CorrectnessThe four linked pages exist and are committed. No documentation change is required.
> Likely an incorrect or invalid review comment.docs/reference/api/rust-library-reference/nemo-fabric-core/runtime/struct-openaistreamsink.mdx (2)
1-14: The full product name "NVIDIA NeMo Fabric" now appears in the description, and it flows from the Rust doc comment atcrates/fabric-core/src/runtime.rsline 395 into both this page andschemas/adapter-contract/legacy/openai-stream-invocation.schema.json. The earlier finding is resolved.
16-48: LGTM!Also applies to: 50-123
| fn run_local_host_openai_stream_adapter( | ||
| plan: &RunPlan, | ||
| runtime: &RuntimeHandle, | ||
| request: RunRequest, | ||
| transport: OpenAiStreamTransport, | ||
| ) -> Result<RunResult> { | ||
| run_local_host_invocation_with_timeout( | ||
| plan, | ||
| runtime, | ||
| request, | ||
| LocalHostInvocation::OpenAiStream(transport), | ||
| local_host_invoke_timeout(plan)?, | ||
| ) | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Inspect the invoke timeout constant, the lifecycle exchange, and any streaming-specific timeout handling.
set -euo pipefail
rg -n 'LOCAL_HOST_INVOKE_TIMEOUT|LOCAL_HOST_START_TIMEOUT|LOCAL_HOST_STOP_TIMEOUT' crates/fabric-core/src/runtime.rs
ast-grep run --pattern 'fn exchange_lifecycle_message($$$) { $$$ }' --lang rust crates/fabric-core/src/runtime.rs
# Check whether documentation states a streaming-specific timeout expectation.
rg -n -i 'timeout' docs/adapter-contract/execution.mdRepository: NVIDIA/NeMo-Fabric
Length of output: 511
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- runtime structure ---'
ast-grep outline crates/fabric-core/src/runtime.rs
printf '%s\n' '--- timeout and lifecycle call sites ---'
rg -n -C 8 'exchange_lifecycle_message|local_host_invoke_timeout|invalidate_timed_out_local_host|OpenAiStream|host_timeout' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- execution contract streaming requirements ---'
rg -n -C 5 -i 'stream|timeout|chunk|end record|terminal' docs/adapter-contract/execution.mdRepository: NVIDIA/NeMo-Fabric
Length of output: 46170
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- timeout resolution and invocation exchange ---'
sed -n '1240,1410p' crates/fabric-core/src/runtime.rs
sed -n '1789,1875p' crates/fabric-core/src/runtime.rs
sed -n '1510,1545p' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- stream transport implementation and host behavior ---'
rg -n -C 8 'invoke_openai_stream|100 Continue|chunked|OpenAiStreamRecord|recv_timeout|emit' --glob '*.rs' --glob '*.py' --glob '*.pyi' .
printf '%s\n' '--- timeout configuration contract and tests ---'
rg -n -C 6 'timeout_seconds|runtime.timeout|LOCAL_HOST_INVOKE_TIMEOUT|invoke_timeout' crates schemas docs pythonRepository: NVIDIA/NeMo-Fabric
Length of output: 50375
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
path = Path("crates/fabric-core/src/runtime.rs")
text = path.read_text()
stream_call = re.search(
r"fn run_local_host_openai_stream_adapter\b.*?"
r"local_host_invoke_timeout\(plan\)\?\s*,",
text,
re.S,
)
exchange = re.search(
r"fn exchange_lifecycle_message\b.*?"
r"host\.responses\.recv_timeout\(timeout\)",
text,
re.S,
)
progress_reset = re.search(
r"recv_timeout\(timeout\).*?(?:reset|deadline|timeout)",
text,
re.S | re.I,
)
print(f"stream_uses_shared_timeout={bool(stream_call)}")
print(f"exchange_waits_for_terminal_response={bool(exchange)}")
print(f"deadline_reset_or_progress_timeout_logic={bool(progress_reset)}")
PYRepository: NVIDIA/NeMo-Fabric
Length of output: 276
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- shared timeout resolution ---'
sed -n '1245,1265p' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- streaming invocation lifecycle ---'
sed -n '1290,1400p' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- terminal response wait ---'
sed -n '1789,1865p' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- exact timeout-related runtime tests ---'
sed -n '3470,3525p' crates/fabric-core/src/runtime.rs
sed -n '3660,3750p' crates/fabric-core/src/runtime.rs
printf '%s\n' '--- focused stream implementation files ---'
fd -t f -e py -e rs | while read -r file; do
rg -l -m 1 'class _OpenAIStreamListener|def invoke_openai_stream|async def invoke_openai_stream' "$file" 2>/dev/null || true
doneRepository: NVIDIA/NeMo-Fabric
Length of output: 13615
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path("crates/fabric-core/src/runtime.rs")
lines = p.read_text().splitlines()
for start, end in ((1245, 1265), (1290, 1400), (1789, 1865)):
print(f"--- runtime.rs:{start}-{end} ---")
for number in range(start, end + 1):
print(f"{number}: {lines[number - 1]}")
PYRepository: NVIDIA/NeMo-Fabric
Length of output: 9065
Use a streaming-specific timeout or reset the deadline on stream progress.
Native streaming shares local_host_invoke_timeout(plan), which defaults to LOCAL_HOST_INVOKE_TIMEOUT (one hour). exchange_lifecycle_message waits for one terminal stdout response with a fixed recv_timeout; stream chunks do not extend this wait. A longer stream therefore returns host_timeout and terminates the host, even while chunks are arriving.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/fabric-core/src/runtime.rs` around lines 1230 - 1243, Update
run_local_host_openai_stream_adapter to avoid the fixed
local_host_invoke_timeout deadline for active streams: use a streaming-specific
timeout and ensure each received stream chunk resets or extends the
terminal-response deadline in exchange_lifecycle_message. Preserve the existing
timeout behavior for non-streaming invocations while allowing healthy streams to
continue as chunks arrive.
| if self._current_stream is not None and not self._current_stream._finalized: | ||
| raise FabricStateError( | ||
| "a streaming invocation is active; fully consume it or call " | ||
| "`await stream.aclose()` before starting another turn" | ||
| ) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Extract the duplicated active-stream guard.
The same five-line guard now appears three times: lines 164-168 in invoke, lines 288-292 in invoke_stream, and lines 330-334 here. All three read the private _finalized attribute and raise the same FabricStateError with the same message.
Extract one private helper and call it from all three sites.
♻️ Proposed helper
def _ensure_no_active_stream(self) -> None:
if self._current_stream is not None and not self._current_stream._finalized:
raise FabricStateError(
"a streaming invocation is active; fully consume it or call "
"`await stream.aclose()` before starting another turn"
)- if self._current_stream is not None and not self._current_stream._finalized:
- raise FabricStateError(
- "a streaming invocation is active; fully consume it or call "
- "`await stream.aclose()` before starting another turn"
- )
+ self._ensure_no_active_stream()
self._ensure_invocable()🧰 Tools
🪛 Ruff (0.16.1)
[warning] 331-334: Avoid specifying long messages outside the exception class
(TRY003)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/src/nemo_fabric/runtime.py` around lines 330 - 334, Extract the
repeated active-stream validation into a private _ensure_no_active_stream method
on the containing class, preserving the existing _current_stream/_finalized
condition and FabricStateError message. Replace the guards in invoke,
invoke_stream, and the current method with calls to this helper.
| async def wait_for_end() -> None: | ||
| while not stream._end_observed: | ||
| await asyncio.sleep(0) | ||
|
|
||
| try: | ||
| await asyncio.wait_for(wait_for_end(), timeout=1) |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick win
Replace the zero-delay busy wait with a real sleep.
wait_for_end polls stream._end_observed with await asyncio.sleep(0). The producing work runs in a worker thread, so this loop spins the event loop at full speed for up to the one-second budget. It monopolizes CPU and can delay the loop callbacks that set _end_observed, which makes the test slower and flakier on loaded CI.
Use a small non-zero sleep.
💚 Proposed fix
async def wait_for_end() -> None:
while not stream._end_observed:
- await asyncio.sleep(0)
+ await asyncio.sleep(0.005)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async def wait_for_end() -> None: | |
| while not stream._end_observed: | |
| await asyncio.sleep(0) | |
| try: | |
| await asyncio.wait_for(wait_for_end(), timeout=1) | |
| async def wait_for_end() -> None: | |
| while not stream._end_observed: | |
| await asyncio.sleep(0.005) | |
| try: | |
| await asyncio.wait_for(wait_for_end(), timeout=1) |
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 498-499: Use asyncio.Event instead of awaiting asyncio.sleep in a while loop
(ASYNC110)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/python/test_openai_streaming.py` around lines 497 - 502, Update the
local wait_for_end function to use a small non-zero asyncio.sleep delay while
polling stream._end_observed, replacing the zero-delay yield; keep the existing
wait_for_end polling behavior and one-second asyncio.wait_for timeout unchanged.
Source: Linters/SAST tools
1 participant
Overview
Adds the consolidated adapter-native OpenAI Chat Completions streaming contract defined by FABRIC-162. This introduces
Runtime.invoke_openai_streamwithout changing or coupling it to the existing NeMo Relay-backedRuntime.invoke_streamAPI.This is intentionally the one larger PR in the three-PR sequence because it defines a public Rust/Python/adapter wire contract and carries its generated JSON Schema and API-reference output. It adds no dependencies or lockfile changes.
Breaking changes: none.
Details
fabric.adapter/v1alpha2; the authenticated side channel negotiates its independentfabric.openai_stream/v1alpha1transport version.OpenAIInvokeStreamasync iterator with separate immutable terminal results, empty-stream support, early-consumer-close draining, exact-once invocation, and one-active-turn enforcement.openai.chat_completions.chunk/v1profile, record identity, sequence monotonicity, explicit end records, record limits, and exact JSON numeric bounds throughu64::MAX.Validation
cargo fmt --all --check— passed.just test-rust— passed across the Rust workspace, including 75 core tests.just test-python— 771 passed, 16 skipped.just build-pythonrebuilt the native extension;tests/python/test_native_sdk.py::test_native_sdkthen passed the real subprocess/loopback streaming path.just docs— 0 errors; the unauthenticated Fern redirects check emitted its expected warning.git diff --checkpassed.Where should the reviewer start?
Start with
crates/fabric-core/src/runtime.rsfor capability gating and the transport contract, thenadapters/common/src/nemo_fabric_adapters/common/lifecycle.pyfor failure precedence, andpython/src/nemo_fabric/openai_streaming.pyfor consumer-side stream semantics. The end-to-end contract tests are intests/python/test_openai_streaming.pyandtests/adapters/test_adapters_common_lifecycle.py.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Relates to FABRIC-162
I confirm this contribution is my own work, or I have the right to submit it under this project's license.
I searched existing issues and open pull requests, and this does not duplicate existing work.
Summary by CodeRabbit
New Features
supports_openai_streaming.Documentation
Tests