Skip to content

fix(token_count): add OpenAI Responses API usage extraction - #714

Open
yossiovadia wants to merge 1 commit into
praxis-proxy:mainfrom
yossiovadia:fix/token-count-responses-api
Open

fix(token_count): add OpenAI Responses API usage extraction#714
yossiovadia wants to merge 1 commit into
praxis-proxy:mainfrom
yossiovadia:fix/token-count-responses-api

Conversation

@yossiovadia

Copy link
Copy Markdown

Summary

  • Fix token_count filter to extract usage from OpenAI Responses API streaming events
  • Without this fix, all Responses API clients (Codex CLI, OpenAI SDK with /v1/responses) report 0 tokens

Fixes #713

Root cause

parse_openai only handled Chat Completions format (usage.prompt_tokens). The Responses API puts usage in response.completed events with a different shape:

{"type": "response.completed", "response": {"usage": {"input_tokens": N, "output_tokens": M}}}

Two differences vs Chat Completions:

  1. Usage nested under response.usage, not top-level usage
  2. Field names input_tokens/output_tokens, not prompt_tokens/completion_tokens

Fix

Extend parse_openai to try the Responses API format as a fallback. 3 new structs, one fallback branch. The existing SSE streaming path (try_complete_usage) already calls parse_openai on each SSE data payload — no changes needed in the streaming handler.

What's included

  • filters/src/token_usage/providers.rs — 3 new Responses API structs + fallback in parse_openai
  • 3 new tests: response.completed event, missing total, null response

Test plan

  • 3 new tests for Responses API format
  • All existing provider parser tests pass unchanged (39 total)
  • cargo clippy -p praxis-ai-filters -- -D warnings — zero warnings
  • Validated end-to-end: Codex CLI → Praxis → metering dashboard shows real token counts

The token_count filter only parsed the Chat Completions usage format
(top-level usage.prompt_tokens). The Responses API nests usage under
response.usage with different field names (input_tokens/output_tokens),
causing all Responses API clients (e.g. Codex CLI) to report 0 tokens.

Extend parse_openai to try the Responses API format as a fallback
when the Chat Completions format doesn't match.

Fixes praxis-proxy#713

Signed-off-by: Yossi Ovadia <yovadia@redhat.com>
@yossiovadia
yossiovadia requested review from a team and leseb August 11, 2026 19:55

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Clean, well-scoped fix for streaming Responses API token extraction. The fallback approach in parse_openai is correct for SSE response.completed events and the test coverage for that path is solid.

One gap: non-streaming Responses API responses are silently missed (see inline comment). 1 medium finding.


Automated review by praxis-bot

usage.input_tokens,
usage.output_tokens,
usage.total_tokens,
));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] Non-streaming Responses API responses silently return 0 tokens. When stream: false, the /v1/responses endpoint returns usage at the top level with Responses API field names:

{"id": "resp_123", "object": "response", "usage": {"input_tokens": 150, "output_tokens": 42, "total_tokens": 192}}

Neither parse path matches this:

  • OpenAiResponse fails because OpenAiUsage requires prompt_tokens/completion_tokens, which are absent.
  • ResponsesApiEvent succeeds but response is None (no response key at the top level).

Add a third fallback using a struct with top-level usage that expects input_tokens/output_tokens:

#[derive(Deserialize)]
struct ResponsesApiDirectResponse {
    usage: Option<ResponsesApiUsage>,
}

Then after the ResponsesApiEvent branch:

if let Ok(resp) = serde_json::from_slice::<ResponsesApiDirectResponse>(body)
    && let Some(usage) = resp.usage
{
    return Some(TokenUsage::new(
        usage.input_tokens,
        usage.output_tokens,
        usage.total_tokens,
    ));
}

This reuses ResponsesApiUsage and covers both streaming and non-streaming Responses API paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(token_count): add OpenAI Responses API usage extraction

2 participants