Skip to content

feat(mongodb-to-mongodb]: Read/Write Performance and Reliability Improvements #4079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
michaeltle-goog wants to merge 20 commits into
base: main
Choose a base branch
from
Open

feat(mongodb-to-mongodb]: Read/Write Performance and Reliability Improvements #4079

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
4516199
Improve MongoDB to MongoDB template throughput by removing GroupIntoB…
michaeltle-goog Jul 29, 2026
6799cf4
Add parallel index-slice reader to MongoDbToMongoDb template for high…
michaeltle-goog Jul 29, 2026
4a53f4c
Add comprehensive operational and debugging logs across MongoDB templ…
michaeltle-goog Jul 29, 2026
e29199f
Use official FindQuery in MongoDbIO.read().withQueryFn to fix Illegal…
michaeltle-goog Jul 29, 2026
fef5af7
Apply spotless formatting and checkstyle validation
michaeltle-goog Jul 29, 2026
ce844f2
Group parallel read slices under ReadSlices(collection) in Dataflow UI
michaeltle-goog Jul 29, 2026
e63787e
Add parameterized linear write rate ramp-up with Guava RateLimiter
michaeltle-goog Jul 29, 2026
6f51b96
Add data-driven $sample quantile split discovery and automatic key-ty…
michaeltle-goog Jul 29, 2026
9ede4d0
Remove useDataDrivenReadSplits flag and automatically attempt data-dr…
michaeltle-goog Jul 29, 2026
644b257
Rename numReadPrefixSplits parameter to numReadSplits and remove unus…
michaeltle-goog Jul 29, 2026
4731f2d
fix: implement Type-Aware Data-Driven Splitting to handle mixed BSON …
michaeltle-goog Jul 29, 2026
b4cff74
test: add mockito tests for Type-Aware splits
michaeltle-goog Jul 29, 2026
0387807
fix(mongodb): convert verbose batch flush logging to debug level
michaeltle-goog Jul 29, 2026
cca114d
docs(mongodb): update template parameter descriptions and group param…
michaeltle-goog Jul 29, 2026
a0d5f31
style(mongodb): apply spotless java formatting rules to ReadSplitGene…
michaeltle-goog Jul 30, 2026
2ef36a3
style(mongodb): replace inline fully qualified imports in tests and t…
michaeltle-goog Jul 30, 2026
ea28d8b
feat(mongodb): log explicit DLQ directories and inspection command at…
michaeltle-goog Jul 31, 2026
9321030
feat(mongodb): rate-limit DLQ reason logging to every 30s per worker …
michaeltle-goog Jul 31, 2026
fefb81d
Merge remote-tracking branch 'origin/main' into monogo-to-mongo-impro…
michaeltle-goog Aug 7, 2026
1e3d277
feat(mongodb): add standard UUID representation support and update DL…
michaeltle-goog Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
253 changes: 201 additions & 52 deletions ...odb-to-mongodb/src/main/java/com/google/cloud/teleport/v2/templates/MongoDbToMongoDb.java
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading