Skip to content

fix: cache_control ttl model check + batch create with explicit names#25

Merged
koljahuang merged 3 commits intomainfrom
fix/cache-ttl-and-batch-names
Apr 14, 2026
Merged

fix: cache_control ttl model check + batch create with explicit names#25
koljahuang merged 3 commits intomainfrom
fix/cache-ttl-and-batch-names

Conversation

@koljahuang
Copy link
Copy Markdown
Contributor

Summary

  • Cache TTL fix: Only add ttl field to cache_control for Claude 4.5 family models (Opus 4.5, Sonnet 4.5, Haiku 4.5). Non-4.5 models like Sonnet 4 reject ttl with ValidationException: Extra inputs are not permitted. Also strips ttl from pre-existing client breakpoints on unsupported models.
  • Batch token creation: Replace count + name_prefix (auto-generated prefix-001, prefix-002) with comma-separated names input. Frontend uses a textarea; backend auto-trims whitespace and ignores empty entries.

Changed files

File Change
backend/app/services/bedrock.py _model_supports_cache_ttl(), _new_cache_marker(), _inject_prompt_cache_breakpoints() accept model_id; _build_anthropic_body() passes model_id through
backend/app/api/admin/endpoints/tokens.py BatchCreateTokenRequest: count+name_prefixnames (comma-separated string)
backend/app/services/token.py create_tokens_batch(): count+name_prefixnames: List[str]
backend/tests/test_prompt_cache_injection.py Updated for model-aware TTL; added supported/unsupported model tests
backend/tests/test_stream_failover.py Mock signature fix for model_id param
frontend/src/stores/tokens.ts BatchCreateTokenRequest interface updated
frontend/src/pages/TokensPage.vue Batch dialog: textarea replaces prefix+count inputs

Test plan

  • 27 prompt cache injection tests pass (including new TTL model checks)
  • 65 total tests pass (excluding unrelated pricing test)
  • Pre-commit hooks pass
  • Verify Sonnet 4 + X-Bedrock-Auto-Cache: true + system prompt no longer returns Extra inputs are not permitted
  • Verify Sonnet 4.5 still gets ttl: "1h" in cache_control
  • Verify batch create with comma-separated names creates multiple tokens

Chao Huang added 3 commits April 14, 2026 16:32
…h explicit names

1. Cache TTL fix: Only Claude 4.5 family models support the `ttl` field
   in `cache_control`. Non-4.5 models (Sonnet 4, Opus 4, etc.) reject it
   with "Extra inputs are not permitted". The proxy now checks the model
   before adding `ttl` and strips it from pre-existing breakpoints on
   unsupported models.

2. Batch token creation: Replace `count` + `name_prefix` (auto-generated
   names like prefix-001) with comma-separated `names` string input.
   Frontend textarea replaces the two input fields. Backend auto-trims
   whitespace around each name.
Split on ASCII comma, Chinese full-width comma, semicolons, and
newlines so names entered with non-ASCII input methods are parsed
correctly.
@koljahuang koljahuang merged commit 431f181 into main Apr 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant