[FLINK-AGENTS-524] Add Amazon OpenSearch and S3 Vectors vector store integrations#533
Open
avichaym wants to merge 8 commits intoapache:mainfrom
Open
[FLINK-AGENTS-524] Add Amazon OpenSearch and S3 Vectors vector store integrations#533avichaym wants to merge 8 commits intoapache:mainfrom
avichaym wants to merge 8 commits intoapache:mainfrom
Conversation
added 2 commits
February 10, 2026 08:02
…grations Add two new integration modules for Amazon Bedrock: - Chat model using the Converse API with native tool calling support, SigV4 auth via DefaultCredentialsProvider, and token metrics reporting. Supports all Bedrock models accessible via Converse API. - Embedding model using Titan Text Embeddings V2 via InvokeModel. Batch embed(List<String>) parallelizes via configurable thread pool (embed_concurrency parameter, default 4). Includes unit tests for constructors, parameter handling, and inheritance.
…ockEmbeddingModelConnection
github-actions
bot
added
priority/major
avichaym
changed the title
avichaym
force-pushed
the
feature/aws-vector-stores
branch
from
56b9836 to
9f4f768
Compare
github-actions
bot
added
the
doc-label-missing
|
@avichaym Please add the following content to your PR description and select a checkbox: |
github-actions
bot
added
doc-not-needed
wenjin272
added
doc-needed
avichaym
force-pushed
the
feature/aws-vector-stores
branch
from
9f4f768 to
c492b13
Compare
added 4 commits
February 19, 2026 09:47
…ences
- Add exponential backoff retry (MAX_RETRIES=5) for ThrottlingException,
ServiceUnavailableException, ModelErrorException, 429, 503 — consistent
with BedrockEmbeddingModelConnection in this PR.
- Remove {..} JSON extraction fallback from stripMarkdownFences that could
corrupt normal text responses containing braces.
- Only apply markdown fence stripping on non-tool-call responses.
- Add 5 unit tests for stripMarkdownFences covering: text with braces,
clean JSON, json fences, plain fences, and null input.
…grations Add two new vector store integration modules: - OpenSearch: supports Serverless (AOSS) and Service domains, IAM (SigV4) or basic auth. Implements CollectionManageableVectorStore. ANN search via knn query with ef_search, min_score, and filter_query support. Bulk writes chunked by configurable max_bulk_mb. - S3 Vectors: uses S3 Vectors SDK for PutVectors/QueryVectors/ GetVectors/DeleteVectors. PutVectors chunked at 500 (API limit). Both override add() for batch embedding via embed(List<String>). Includes unit tests and integration tests (auto-enabled via OPENSEARCH_ENDPOINT / S3V_BUCKET environment variables). Validated against real OpenSearch domain and S3 Vectors bucket.
…docs Bedrock (chat + embedding): - Add close() to release AWS SDK clients and thread pools - Wire max_tokens through BedrockChatModelSetup into InferenceConfiguration - Add retry jitter to BedrockEmbeddingModelConnection - Add typed getConnection() override to BedrockEmbeddingModelSetup - Document stripMarkdownFences necessity and future work OpenSearch vector store: - Add close() to release SdkHttpClient - Cache DefaultCredentialsProvider (was creating new instance per request) - Add constructor validation for required endpoint/index params - Add limit support in get() via extraArgs - Add TODO for Aws4Signer deprecation and batch add() dedup S3 Vectors vector store: - Add close() to release S3VectorsClient - Add constructor validation for required vector_bucket/vector_index params - size() now throws UnsupportedOperationException instead of returning -1 - Add TODO for batch add() dedup All files: expand wildcard imports, add usage example Javadocs
avichaym
force-pushed
the
feature/aws-vector-stores
branch
from
c492b13 to
dd77d50
Compare
OpenSearchVectorStore: - Add retry with exponential backoff for 429/502/503 in executeRequest() - Only ignore 404s in getCollection/deleteCollection (not all exceptions) - Close credentialsProvider in close() S3VectorsVectorStore: - Add retry with backoff for putVectors (ThrottlingException, 429, 503) Consistent with retry patterns in BedrockChatModelConnection and BedrockEmbeddingModelConnection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #524
Depends on #534 — please merge that first.
Purpose of change
Add Amazon OpenSearch and S3 Vectors as vector store providers.
OpenSearchVectorStore— Supports Serverless (AOSS) and Service domains, IAM/basic auth, implementsCollectionManageableVectorStorefor Long-Term Memory, KNN search with filter support, chunked bulk writesS3VectorsVectorStore— S3 Vectors SDK, PutVectors chunked at 500 (API limit)Both override
add()for batch embedding optimization.New modules:
integrations/vector-stores/opensearch/,integrations/vector-stores/s3vectors/Tests
OPENSEARCH_ENDPOINT,S3V_BUCKET): collection CRUD, document CRUD, filtered queryAPI
No public API changes. New integration modules only.
Documentation
doc-neededdoc-not-neededdoc-included