feat(token-count): report prompt cache token breakdown

[noyitz]

Adds token.cache_read and token.cache_write filter metadata keys so consumers
can distinguish cached prompt tokens from fresh ones.

Part of #577.

Why

Providers that support prompt caching price cached input very differently from
fresh input: a cache read is roughly a tenth of the fresh input price, while a
cache write is roughly 1.25x. token_count folded all of it into a single input
count, so no downstream consumer could tell the two apart and a metered
deployment had no way to bill correctly.

The counts were never wrong — they were never split, which is exactly why a
metering consumer reads zero cached tokens on a workload that is mostly cache
hits.

Per-provider semantics

The providers disagree on whether cached tokens are already counted in the
reported prompt total, so each is handled on its own terms rather than through a
shared assumption:

• OpenAI / Azure report prompt_tokens inclusive of cached tokens, with
  prompt_tokens_details.cached_tokens as a subset of it. Neither reports a
  cache-write count.
• Anthropic reports input_tokens exclusive of cached tokens, so the true
  input total is the sum of input_tokens, cache_creation_input_tokens, and
  cache_read_input_tokens.
• Google reports promptTokenCount inclusive, with
  cachedContentTokenCount as a subset.

Bedrock Converse is deliberately left uncovered, since its inclusion semantics
are not documented clearly enough to bill against. Claude served over Bedrock
InvokeModel continues to fall through to the Anthropic parser and gains the
breakdown with it.

Both the JSON and SSE paths are covered. TokenUsage now derives Default so
the cache fields follow the same construction pattern as the existing counts
rather than being special cased at each call site.

Scope note

This PR deliberately does not surface the breakdown as response headers.
token_usage_headers reads metadata in on_response, the header phase, while
body-derived token counts are only available in on_response_body, which runs
strictly later — the header block has already been written downstream by then.
That is a pre-existing constraint affecting the three existing token headers as
well, so it is tracked separately in #583 rather than worked around here.

Tests

• 6 new tests in apis/src/token_usage/providers.rs covering the cache
  breakdown per provider, including the Anthropic inclusive/exclusive
  distinction.
• 4 new tests in apis/src/token_usage/tests.rs for the streaming path.
• 3 new tests in filters/src/token_count/tests.rs asserting the metadata keys
  are written for JSON and SSE.

Verification

make build, make test, and make lint all pass on this branch
(3,134 tests, 0 failures).

[praxis-bot]

Review: feat(token-count): report prompt cache token breakdown

Solid, well-designed PR. The per-provider cache semantics (Anthropic exclusive vs OpenAI/Google inclusive) are handled correctly throughout both JSON and SSE paths. The StreamingTokens struct cleanly replaces the old tuple return, the with_cache builder pattern on TokenUsage is a clean extension, and the merge_reported_count refactor eliminates duplication. Test coverage is thorough: 21 new tests across the three test modules covering every provider, both content paths, absent/null/zero cache fields, and the header-only Bedrock path.

One consistency concern flagged inline.

Severity	Count
Critical	0
Large	0
Medium	1

[praxis-bot]

[Medium] finalize_streaming_counts unconditionally calls set_cache_token_usage with defaults of 0 when no accumulator was populated. For Bedrock ConverseStream, parse_bedrock_event returns cache_read: None / cache_write: None, so merge_reported_count skips them and nothing is accumulated -- yet this line writes token.cache_read = "0" and token.cache_write = "0". The sibling extract_bedrock_headers path deliberately leaves these keys absent (asserted in bedrock_invoke_model_headers_record_no_cache).

A downstream metering consumer distinguishing "no cache activity" ("0") from "cache info unavailable" (absent) would see inconsistent semantics depending on which Bedrock code path ran. Consider guarding:

if ctx.filter_metadata.contains_key(META_CACHE_READ)
    || ctx.filter_metadata.contains_key(META_CACHE_WRITE)
{
    set_cache_token_usage(ctx, cache_read, cache_write);
}

This preserves the current behavior for providers that report cache info (Anthropic streaming, OpenAI/Google via try_complete_usage) while leaving keys absent for those that don't (Bedrock ConverseStream), consistent with the header path.

[crstrn13]

I agree with the bot here, besides this I'd say the PR can be merged.

[praxis-bot]

Review: feat(token-count): report prompt cache token breakdown (pass 2)

Clean, well-structured PR. The per-provider cache semantics are handled correctly, the StreamingTokens struct is a good replacement for the old tuple, and the 21 new tests are thorough across providers and content paths.

The prior review flagged the cache metadata inconsistency in finalize_streaming_counts. The same root cause also applies to the JSON body path (flagged below) and the try_complete_usage SSE path, where set_cache_token_usage is called unconditionally regardless of whether the provider actually reports cache info. The suggested guard from the prior review only covers the streaming accumulation path; a holistic fix would be to make TokenUsage.cache_read and cache_write into Option<u64> (matching StreamingTokens), then only call set_cache_token_usage when either is Some. This fixes all three paths from a single change.

Severity	Count
Critical	0
Large	0
Medium	1 (+ 1 in prior review)

[praxis-bot]

[Medium] This call is unconditional: when provider.extract_token_usage() returns a TokenUsage, cache_read_tokens() and cache_write_tokens() are always 0 for providers that never called .with_cache() (Bedrock Converse JSON). The result is token.cache_read = "0" and token.cache_write = "0" in metadata, inconsistent with extract_bedrock_headers which deliberately leaves them absent.

The prior review flagged the same semantic issue in finalize_streaming_counts, but the guard suggested there (contains_key(META_CACHE_READ)) does not apply here because the JSON path writes directly to consumer-visible metadata without intermediate accumulators.

A single structural fix covers all three affected paths (this one, try_complete_usage, and finalize_streaming_counts): make TokenUsage.cache_read and cache_write Option<u64> instead of u64, then gate set_cache_token_usage on presence:

if let Some(cr) = usage.cache_read_tokens() {
    set_cache_token_usage(ctx, cr, usage.cache_write_tokens().unwrap_or(0));
}

This keeps the existing behavior for providers that do report cache info (Anthropic, OpenAI, Google) while leaving keys absent for those that don't (Bedrock Converse), consistent with the header-only path.

[praxis-bot]

Review: feat(token-count): report prompt cache token breakdown (pass 3)

Third review pass. Examined all six changed files in full: count.rs, mod.rs, providers.rs, streaming.rs, count/tests.rs, and the example config.

No new issues found. The prior two findings (unconditional set_cache_token_usage in finalize_streaming_counts and handle_json_body) remain the only substantive concerns. Specifically:

• The StreamingTokens struct correctly uses Option<u64> to distinguish reported-zero from absent, and merge_reported_count correctly skips None values. The gap is in finalize_streaming_counts where those absent values collapse to 0 via unwrap_or(0) and are then unconditionally published.
• The JSON path in handle_json_body unconditionally calls set_cache_token_usage with u64 values from TokenUsage, which defaults cache fields to 0 for providers that never called .with_cache().
• The try_complete_usage SSE path has the same unconditional write pattern.

All three are the same root cause (already flagged), which is why the prior review's suggestion to make TokenUsage.cache_read/cache_write into Option<u64> would fix all paths at once.

Everything else is clean: the Anthropic inclusive/exclusive normalization is consistent between providers.rs and streaming.rs, the with_cache builder pattern is sound, Bedrock Converse correctly avoids cache claims in both JSON and streaming, and the 21 new tests cover the matrix well.

Severity	Count
Critical	0
Large	0
Medium	0 (2 from prior rounds still apply)