[FLINK-AGENTS-523] Add Amazon Bedrock chat model and embedding model integrations#534
[FLINK-AGENTS-523] Add Amazon Bedrock chat model and embedding model integrations#534avichaym wants to merge 4 commits intoapache:mainfrom
Conversation
…grations Add two new integration modules for Amazon Bedrock: - Chat model using the Converse API with native tool calling support, SigV4 auth via DefaultCredentialsProvider, and token metrics reporting. Supports all Bedrock models accessible via Converse API. - Embedding model using Titan Text Embeddings V2 via InvokeModel. Batch embed(List<String>) parallelizes via configurable thread pool (embed_concurrency parameter, default 4). Includes unit tests for constructors, parameter handling, and inheritance.
…ockEmbeddingModelConnection
github-actions
bot
added
priority/major
wenjin272
added
doc-needed
avichaym
force-pushed
the
feature/bedrock-integration
branch
from
a10b9dd to
34d38c2
Compare
| } | ||
| return trimmed; | ||
| } | ||
| int start = trimmed.indexOf('{'); |
There was a problem hiding this comment.
This fallback path extracts the substring between the first '{'and last '}' from any response. If the model returns a normal text response that happens to contain braces (e.g., "Use the format {key: value} for config"), this would discard everything outside the braces.
Should this extraction only apply when structured output is expected (i.e., when an outputSchema is provided), rather than on every response? The other integrations (OpenAI, Anthropic) don't modify the text content at the connection layer.
There was a problem hiding this comment.
Good catch. Removed the {...} extraction fallback entirely — stripMarkdownFences now only strips markdown code fences ( json... ), which is a lossless operation. Additionally, fence stripping is now
only applied to non-tool-call responses, since tool-call responses should pass through unmodified.
The connection layer doesn't have visibility into whether outputSchema is set , so scoping it that way would require changing the BaseChatModelConnection interface.
Removing the extraction altogether addresses the core concern — normal text responses containing braces are no longer corrupted.
Also Added 5 unit tests for stripMarkdownFences covering: text with braces preserved, clean JSON passthrough, json fence stripping, plain fence stripping, and null handling.
Fixed in 6d50c47
| } | ||
|
|
||
| return convertResponse(response); | ||
| } catch (Exception e) { |
There was a problem hiding this comment.
Bedrock frequently returns ThrottlingException under load. The current code wraps all exceptions into a RuntimeException without retry. It might be worth handling ThrottlingException and ServiceUnavailableException with retry here - or at minimum, preserving the original exception type so callers can distinguish transient from permanent failures. (PR #531 is adding similar retry logic to MCPServer for the same reason.)
There was a problem hiding this comment.
Added exponential backoff retry around client.converse(), consistent with the retry already in BedrockEmbeddingModelConnection in this same PR.
MAX_RETRIES=5, backoff with jitter (2^attempt 200ms (0.5 + random 0.5)), covering ThrottlingException, ServiceUnavailableException, ModelErrorException, 429, and 503. Non-retryable exceptions still fail immediately with the original exception preserved as the cause.
Fixed in 6d50c47
…ences
- Add exponential backoff retry (MAX_RETRIES=5) for ThrottlingException,
ServiceUnavailableException, ModelErrorException, 429, 503 — consistent
with BedrockEmbeddingModelConnection in this PR.
- Remove {..} JSON extraction fallback from stripMarkdownFences that could
corrupt normal text responses containing braces.
- Only apply markdown fence stripping on non-tool-call responses.
- Add 5 unit tests for stripMarkdownFences covering: text with braces,
clean JSON, json fences, plain fences, and null input.
avichaym
force-pushed
the
feature/bedrock-integration
branch
from
8d23262 to
6d50c47
Compare
3 participants
Linked issue: #523
Purpose of change
Add Amazon Bedrock as a chat model and embedding model provider for Flink Agents.
BedrockChatModelConnection— Converse API with native tool calling, SigV4 authBedrockChatModelSetup— model, temperature, max_tokens configurationBedrockEmbeddingModelConnection— Titan Text Embeddings V2 with parallel batch embeddingBedrockEmbeddingModelSetup— model, dimensions configurationNew modules:
integrations/chat-models/bedrock/,integrations/embedding-models/bedrock/Tests
BedrockChatModelConnectionTest(4),BedrockChatModelSetupTest(3),BedrockEmbeddingModelTest(4)API
No public API changes. New integration modules only.
Documentation
doc-neededdoc-not-neededdoc-included