feat(weave): add Custom Runtime conversation context

[mattliu-mygit]

Summary

• make the trace server authoritative for Custom Runtime conversation identity
• generate the ID before runtime dispatch when the caller does not already have one
• send the ID to explicit custom:: runtimes in the reserved, percent-encoded X-Weave-Conversation-Id header
• return the server-selected ID in the non-streaming response or a conversation-only streaming _meta record
• let the Custom Runtime own completion tracing, avoiding a duplicate W&B ingress span
• preserve the conversation across later turns and provider switches while recomputing trace ownership for every request

Existing behavior and decision

Today, when Trace LLM call is enabled, the Playground asks the trace server to record the completion boundary. For a Custom Runtime that produces one W&B-owned root chat span containing the submitted messages and final response. The public weave.chat client has the equivalent behavior through its local completion op. This is useful when the downstream model is opaque, but a Custom Runtime is not opaque: it can independently log its Turn, LLM, subagent, and tool spans to Weave.

If both sides log the same request, the Conversation view receives two representations of the boundary user and assistant messages. A conversation ID can correlate the records, but it cannot determine which duplicate should be hidden: message content and timestamps are not reliable identity, and the W&B span and runtime span do not share a span tree.

This PR therefore makes tracing ownership opinionated for explicit custom:: requests:

• the trace server establishes conversation identity before dispatch and forwards it to the selected runtime
• the Custom Runtime owns the complete trace, including boundary input/output and internal work
• hosted and CoreWeave requests retain W&B-owned completion tracing
• clients retain the returned identity for later turns but do not need to generate the first ID

This is the smallest implementation with one authoritative trace owner. It is decided per request in the shared completion dispatch path, so it covers Playground, synchronous, streaming, async, and raw API ingress without adding a UI setting or changing the storage model. It also requires no database migration, trace-merging protocol, deduplication heuristic, conversation-specific rendering logic, or coordinated Core deployment. Explicit custom:: resolution keeps the behavior from leaking to CoreWeave, which also uses LiteLLM's internal provider="custom" transport value.

Alternatives considered were materially broader:

• Log on both sides and deduplicate: requires a stable cross-system boundary-span identity plus storage or UI reconciliation; content/time heuristics would hide legitimate repeated messages.
• Pass W&B trace/span IDs and merge the trees: retains two logging owners and introduces parentage, partial-failure, and cross-ingress lifecycle rules.
• Return all runtime events for W&B to log: requires a new runtime event protocol, buffering/streaming semantics, and failure recovery.
• Add a customer setting: creates two Custom Runtime behaviors and allows the duplicate state we are trying to avoid.

The tradeoff is deliberate: an uninstrumented Custom Runtime will no longer get a W&B-generated boundary span. The runtime contract and public sample make runtime-owned logging explicit, because only the runtime can provide the complete chronology.

End-to-end flow

1. A client sends its existing conversation_id, or omits it on the first turn.
2. Before runtime dispatch, the trace server preserves the supplied ID or generates one.
3. The trace server resolves whether the request targets an explicit Custom Runtime.
4. For a Custom Runtime, the server overwrites any configured context header with the authoritative, percent-encoded X-Weave-Conversation-Id value and disables the W&B completion span for that request.
5. The runtime logs its boundary input/output, LLM, tool, and subagent spans with that conversation ID.
6. A non-streaming response returns conversation_id at the top level. A streaming response first emits {"_meta":{"conversation_id":"..."}} and then the normal completion chunks.
7. Core already consumes _meta; weave.chat now consumes it internally, exposes stream.conversation_id, and never yields the metadata as an LLM chunk. Later turns and provider switches send the retained ID back to the trace server.

The _meta record for an untracked Custom Runtime contains only conversation identity—no synthetic call, span, or trace IDs are created. Existing tracked-stream metadata remains unchanged.

Provider switching

Trace ownership is decided per request rather than cached for the whole conversation. Automated coverage and the local E2E exercise one conversation moving through:

1. Custom Runtime: receives the context header and owns tracing
2. hosted CoreWeave: reuses the conversation ID and keeps normal W&B tracing
3. Custom Runtime: receives the same context header and again owns tracing

The local trace data preserved the same conversation ID across all three providers. The hosted request retained a W&B-owned Weave Chat Playground span; returning to the Custom Runtime rejoined the original runtime-owned conversation.

The hosted CoreWeave turn returned a blank/error response locally because of the existing reasoning-response normalization behavior. That is separate from conversation identity: the persisted span still demonstrated the expected ID and per-request trace ownership, and this PR does not alter response normalization.

Customer-visible result

The screenshots use the same natural two-turn Seattle-planning conversation. The deterministic OpenAI-compatible downstream keeps the content stable; the conversation storage and runtime-owned Weave spans are real.

Before: only the boundary input and output

Current production records one root chat span per turn. The customer sees the final answers, but none of the runtime's planning, research, or tool work: 2 spans, 0 tools, 2 traces.

Open the before conversation

After: runtime work appears chronologically

The runtime receives one conversation ID across both turns and logs the complete workflow: 18 runtime-owned spans, 8 tools, 2 traces. The app inspection showed 2 user messages and 4 assistant messages, with planning, research, LLM, subagent, and tool activity in chronological order and no synthetic internal user messages.

Open the production-hosted after example

Turn 1: planning, research, four tools, and the final response

Turn 2: the follow-up remains in the same conversation

Trace view: one nine-span runtime-owned hierarchy

The public sample implements the runtime side of the contract, including use of the received conversation ID and assistant/tool-oriented trace data. The E2E version is published in commit 4e6948d.

Verification

• affected-file ClickHouse-backed Weave suite — 154 passed
• affected-file in-memory Weave suite — 142 passed, 12 skipped
• focused type checking — passed
• Ruff formatting and diff checks — passed
• public sample: npm test — 104 passed
• local two-turn Playground E2E — 18 spans, 8 tools, 2 traces, with complete runtime chronology
• local Custom Runtime → CoreWeave → Custom Runtime E2E — same conversation ID preserved and trace ownership recomputed per request

Code review

Reviewed the complete diff after moving conversation-ID ownership back into the trace server. The retained tests cover server generation/reuse, streaming _meta consumption, non-streaming responses, sync/async/in-memory servers, reserved-header override, CoreWeave isolation, Custom Runtime trace ownership, provider switching, OpenAI SSE compatibility, and stream resource cleanup.

[codecov]

Codecov Report

❌ Patch coverage is 87.50000% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
weave/chat/stream.py	75.00%	3 Missing and 1 partial ⚠️
weave/trace_server/in_memory_trace_server.py	0.00%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mattliu-mygit]

Superseded by #7696, which isolates the first mergeable slice to Custom Runtime conversation-context propagation and preserves the existing tracing policy. The logging-default and UI behavior will follow in separate PRs.