feat(Spanner): integrate SourceConfigParser to centralize shard configuration loading for SourceDbToSpanner pipelines. - #3854
Open
pratickchokhani wants to merge 16 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3854 +/- ##
=============================================
+ Coverage 38.08% 56.17% +18.08%
- Complexity 511 7336 +6825
=============================================
Files 195 1126 +931
Lines 11764 68711 +56947
Branches 1179 7778 +6599
=============================================
+ Hits 4480 38596 +34116
- Misses 6919 27619 +20700
- Partials 365 2496 +2131
🚀 New features to boost your workflow:
|
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
June 22, 2026 05:50
453a9f9 to
b7b20a3
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
6 times, most recently
from
June 29, 2026 07:25
a19cf9e to
581ffd3
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
4 times, most recently
from
June 30, 2026 14:13
0da7475 to
e13bdd2
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
July 1, 2026 09:19
c90217f to
c58f5cb
Compare
pratickchokhani
commented
Jul 1, 2026
pratickchokhani
force-pushed
the
shard-config-bulk
branch
3 times, most recently
from
July 2, 2026 10:46
41f2355 to
461938b
Compare
pratickchokhani
marked this pull request as ready for review
July 2, 2026 10:55
pratickchokhani
requested review from
jsuhani-2026 and
sm745052
and removed request for
sm745052
July 2, 2026 10:55
pratickchokhani
force-pushed
the
shard-config-bulk
branch
3 times, most recently
from
July 28, 2026 08:38
c7f9560 to
2649e0a
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
August 4, 2026 10:00
2649e0a to
157262e
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
August 4, 2026 11:07
f2d0565 to
2ddbd87
Compare
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
August 6, 2026 08:17
23b9871 to
7007b5f
Compare
…guration loading for SourceDbToSpanner pipelines
…nectionConfig instead of pipeline options for improved configuration handling.
…es and update JDBC migration logic to utilize shard configurations
…object directly to builder methods
…e standardized shard config GCS file
pratickchokhani
force-pushed
the
shard-config-bulk
branch
from
August 7, 2026 04:55
7007b5f to
2d3b2a3
Compare
| "additionalExperiments", Collections.singletonList("disable_runner_v2")); | ||
|
|
||
| if (!multiSharded) { | ||
| builder.addParameter("jdbcDriverClassName", "com.mysql.jdbc.Driver"); |
Contributor
There was a problem hiding this comment.
In the updated flow, the sharded and non-sharded flows are basically the same. Why do we need this special handling ?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.