fix(token_count): add OpenAI Responses API usage extraction - #714
fix(token_count): add OpenAI Responses API usage extraction#714yossiovadia wants to merge 1 commit into
Conversation
The token_count filter only parsed the Chat Completions usage format (top-level usage.prompt_tokens). The Responses API nests usage under response.usage with different field names (input_tokens/output_tokens), causing all Responses API clients (e.g. Codex CLI) to report 0 tokens. Extend parse_openai to try the Responses API format as a fallback when the Chat Completions format doesn't match. Fixes praxis-proxy#713 Signed-off-by: Yossi Ovadia <yovadia@redhat.com>
praxis-bot
left a comment
There was a problem hiding this comment.
Review Summary
Clean, well-scoped fix for streaming Responses API token extraction. The fallback approach in parse_openai is correct for SSE response.completed events and the test coverage for that path is solid.
One gap: non-streaming Responses API responses are silently missed (see inline comment). 1 medium finding.
Automated review by praxis-bot
| usage.input_tokens, | ||
| usage.output_tokens, | ||
| usage.total_tokens, | ||
| )); |
There was a problem hiding this comment.
[Medium] Non-streaming Responses API responses silently return 0 tokens. When stream: false, the /v1/responses endpoint returns usage at the top level with Responses API field names:
{"id": "resp_123", "object": "response", "usage": {"input_tokens": 150, "output_tokens": 42, "total_tokens": 192}}Neither parse path matches this:
OpenAiResponsefails becauseOpenAiUsagerequiresprompt_tokens/completion_tokens, which are absent.ResponsesApiEventsucceeds butresponseisNone(noresponsekey at the top level).
Add a third fallback using a struct with top-level usage that expects input_tokens/output_tokens:
#[derive(Deserialize)]
struct ResponsesApiDirectResponse {
usage: Option<ResponsesApiUsage>,
}Then after the ResponsesApiEvent branch:
if let Ok(resp) = serde_json::from_slice::<ResponsesApiDirectResponse>(body)
&& let Some(usage) = resp.usage
{
return Some(TokenUsage::new(
usage.input_tokens,
usage.output_tokens,
usage.total_tokens,
));
}This reuses ResponsesApiUsage and covers both streaming and non-streaming Responses API paths.
Summary
token_countfilter to extract usage from OpenAI Responses API streaming events/v1/responses) report 0 tokensFixes #713
Root cause
parse_openaionly handled Chat Completions format (usage.prompt_tokens). The Responses API puts usage inresponse.completedevents with a different shape:{"type": "response.completed", "response": {"usage": {"input_tokens": N, "output_tokens": M}}}Two differences vs Chat Completions:
response.usage, not top-levelusageinput_tokens/output_tokens, notprompt_tokens/completion_tokensFix
Extend
parse_openaito try the Responses API format as a fallback. 3 new structs, one fallback branch. The existing SSE streaming path (try_complete_usage) already callsparse_openaion each SSE data payload — no changes needed in the streaming handler.What's included
filters/src/token_usage/providers.rs— 3 new Responses API structs + fallback inparse_openairesponse.completedevent, missing total, null responseTest plan
cargo clippy -p praxis-ai-filters -- -D warnings— zero warnings