Feature Request
Enable shared ownership of chat_history in streaming requests, allowing callers to access the conversation history after streaming completes.
Motivation
Currently, when using stream_prompt().with_history() or stream_chat(), callers pass a Vec<Message> which is consumed by the request. After streaming completes, there's no way to access the updated conversation history (including the user's prompt and assistant's response).
This limitation makes it difficult to:
- Build multi-turn chatbots: Callers must manually reconstruct the history by tracking prompts and accumulating streamed responses
- Share history across requests: Multiple streaming requests cannot share and update a common history
- Inspect conversation state: No way to examine the full conversation after streaming
Example of current pain point:
let history = vec![];
// First request - history is consumed
let mut stream = agent
.stream_prompt("Hello")
.with_history(history) // moved
.await;
// Consume stream...
// Problem: Can't access updated history!
// Must manually track: history.push(user_msg); history.push(assistant_msg);Proposal
Change chat_history parameter type from Vec<Message> to Arc<RwLock<Vec<Message>>>:
1. Update StreamChat trait and StreamingPromptRequest:
// streaming.rs
fn stream_chat(
&self,
prompt: impl Into<Message> + WasmCompatSend,
chat_history: Arc<RwLock<Vec<Message>>>, // shared ownership
) -> StreamingPromptRequest<M, ()>;
// streaming.rs - with_history method
pub fn with_history(mut self, history: Arc<RwLock<Vec<Message>>>) -> Self {
self.chat_history = Some(history);
self
}2. Use std::sync::RwLock instead of tokio::sync::RwLock:
The lock is never held across await points, so synchronous RwLock is sufficient and avoids async overhead. And the std::RwLock is better than a third-party RwLock in public API.
3. Ensure history is updated before streaming completes:
When the stream finishes (no tool calls), push the user message and assistant response to history:
if !did_call_tool {
{
let mut history = chat_history.write().expect("lock poisoned");
history.push(current_prompt.clone());
if !last_text_response.is_empty() {
history.push(Message::assistant(&last_text_response));
}
}
// ... yield final response and break
}Usage after this change:
use std::sync::{Arc, RwLock};
let history = Arc::new(RwLock::new(vec![]));
// First request
let mut stream = agent
.stream_prompt("Hello")
.with_history(history.clone()) // clone Arc, not the data
.await;
// Consume stream...
// Now we can access the updated history!
let updated = history.read().unwrap();
assert_eq!(updated.len(), 2); // user message + assistant response
// Second request - same shared history
let mut stream2 = agent
.stream_prompt("Follow-up question")
.with_history(history.clone())
.await;Alternatives
1. Return history from the stream:
Could yield the final history as part of FinalResponse. Drawback: Doesn't allow concurrent access or sharing between requests.
2. Callback-based approach:
Provide a callback that receives updated history. Drawback: More complex API, doesn't integrate naturally with Rust ownership patterns.
3. Keep Vec<Message> but add getter:
Return a reference to internal history. Drawback: Lifetime issues with streaming, can't share across requests.
The Arc<RwLock<...>> approach was chosen because:
- Idiomatic Rust pattern for shared mutable state
- No lifetime complexity
- Caller controls when to read/write
- Works naturally with async code
- Uses standard library (no external crate dependency)
Feature Request
Enable shared ownership of
chat_historyin streaming requests, allowing callers to access the conversation history after streaming completes.Motivation
Currently, when using
stream_prompt().with_history()orstream_chat(), callers pass aVec<Message>which is consumed by the request. After streaming completes, there's no way to access the updated conversation history (including the user's prompt and assistant's response).This limitation makes it difficult to:
Example of current pain point:
Proposal
Change
chat_historyparameter type fromVec<Message>toArc<RwLock<Vec<Message>>>:1. Update
StreamChattrait andStreamingPromptRequest:2. Use
std::sync::RwLockinstead oftokio::sync::RwLock:The lock is never held across await points, so synchronous
RwLockis sufficient and avoids async overhead. And the std::RwLock is better than a third-party RwLock in public API.3. Ensure history is updated before streaming completes:
When the stream finishes (no tool calls), push the user message and assistant response to history:
Usage after this change:
Alternatives
1. Return history from the stream:
Could yield the final history as part of
FinalResponse. Drawback: Doesn't allow concurrent access or sharing between requests.2. Callback-based approach:
Provide a callback that receives updated history. Drawback: More complex API, doesn't integrate naturally with Rust ownership patterns.
3. Keep
Vec<Message>but add getter:Return a reference to internal history. Drawback: Lifetime issues with streaming, can't share across requests.
The
Arc<RwLock<...>>approach was chosen because: