Add cache directive to Bedrock system prompt via providerOptions#663
Open
timko212 wants to merge 1 commit into
Open
Add cache directive to Bedrock system prompt via providerOptions#663timko212 wants to merge 1 commit into
timko212 wants to merge 1 commit into
Conversation
Anthropic already accepts cache_control through providerOptions and it flows
straight into the request body. Bedrock needs the marker emitted as a separate
cachePoint block in the system list, so providerOptions alone is not enough.
This adds a 'cache' directive that the Bedrock Converse gateway translates into
a cachePoint marker on the system blocks:
return ['cache' => [['target' => 'system']]];
The directive is stripped before the remaining providerOptions are merged into
the Converse parameters, so it never reaches the AWS API.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #634, which was closed because it overlapped with how Anthropic already works.
Bedrock is the part that's actually missing. The Converse API doesn't accept
cache_controlon the system string; it wants an extracachePointblock appended to thesystemlist, which isn't something a generic passthrough can do. So there's no clean way to enable Bedrock prompt caching from an agent today.This PR adds a
cachedirective thatBedrockTextGatewayrecognises and turns into thecachePointblock. From the agent side it looks like this:The directive is consumed by the gateway and stripped before the rest of
providerOptionsis merged into the Converse parameters, so AWS never sees thecachekey.The new test (`tests/Feature/Providers/Bedrock/CacheControlTest.php`) checks three things: the marker shows up when the directive is set, it doesn't leak through to AWS, and the system block stays untouched when no directive is set. There's also a small `capturingBedrockClient` helper in `BedrockHelpers` so the test can read the exact Converse payload.