Gigamap jvector update tests updade#569
Merged
zdenek-jonas merged 7 commits intogigamap-jvector-updatefrom Feb 21, 2026
Merged
Conversation
fh-ms
approved these changes
Feb 21, 2026
fh-ms
added a commit
that referenced
this pull request
Feb 21, 2026
* Bump jvector version to 4.0.0-rc.8 in pom.xml * Add parallel on-disk write support to VectorIndex * Introduce BackgroundIndexingManager for eventual consistency in VectorIndex - Added `BackgroundIndexingManager` interface to manage background graph indexing. - Implemented a default asynchronous queue-based manager for deferred HNSW operations. - Added unit tests to validate eventual indexing behavior across add, update, remove, and bulk operations. * Add concurrent stress tests for VectorIndex thread safety - Introduced stress test cases to validate VectorIndex under heavy concurrent operations. - Tests cover various configurations, including in-memory and on-disk setups with/without PQ compression and background tasks. - Ensured thread safety via assertions for no exceptions or deadlocks. - Included targeted eventual indexing and heavy load scenarios for robustness. * Add performance tests for synchronous vs eventual indexing in VectorIndex - Introduced test cases to evaluate insertion performance (single and batch adds) between synchronous and eventual indexing modes. - Verified search quality to ensure correctness of deferred graph indexing. - Included detailed metrics on caller-visible speedup and indexing throughput. * Refactor BackgroundIndexingManager to delegate operation handling via execute() - Replaced the applyOperation method with per-operation execute() implementations in IndexingOperation types. - Simplified indexing logic by encapsulating operation-specific behavior within each record. - Improved maintainability and reduced duplication in BackgroundIndexingManager. * Update default value for parallel on-disk write to false in VectorIndexConfiguration * Update tests for parallel on-disk write default change to false in VectorIndexConfiguration * Refactor VectorIndex synchronization and builder operation handling - Replaced `persistenceLock` with `builderLock` for unified read/write access control over builder operations. - Introduced deferred operation handling for sync-mode mutations during cleanup phases. - Improved thread-safety for concurrent graph updates by coordinating via builder read/write locks. - Added `cleanupInProgress` flag and `deferredBuilderOps` queue to manage in-flight operations during cleanup and persistence tasks. - Removed redundant `synchronized(parentMap)` calls to avoid lock-ordering issues. * Documented `eventualIndexing` and `parallelOnDiskWrite` features in VectorIndexConfiguration with examples and configuration details. * Document eventual indexing and parallel on-disk write features with examples and configuration guidance in VectorIndex JavaDoc. * Remove redundant default case check for unsupported similarity function in VectorIndex and add comment * Add braces to conditional statements in VectorEntry equals() for consistency * Consolidate BackgroundIndexingManager, BackgroundOptimizationManager, and BackgroundPersistenceManager into unified BackgroundTaskManager. * Gigamap jvector update tests updade (#569) * move configuration tests to VICT file, remove duplicite tests. * remove test, duplicate of testBackgroundOptimizationTriggersAfterIntervalAndThreshold * test refactoring * change indents - to see diff in PR * reverts original formats * add vector indices unit tests * remove dulicite tests --------- Co-authored-by: Zdenek Jonas <z.jonas@microstream.one>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A small optimisation of the tests — overall shorter execution time thanks to the use of await.
Some assertions were removed.
An
AssertNotNullfollowed immediately by another assertion that would throw an NPE anyway is unnecessary.Configuration tests have been moved into the dedicated configuration test class.
Plus a few minor adjustments.