[Fix][Connector-V2] Close JDBC source read transactions after each split - #1
Open
brownie-cake wants to merge 2 commits into
Open
[Fix][Connector-V2] Close JDBC source read transactions after each split#1brownie-cake wants to merge 2 commits into
brownie-cake wants to merge 2 commits into
Conversation
…esense and Assert connector docs (apache#11751) Co-authored-by: DanielCarter-stack <danielcarter-stack@users.noreply.github.com>
brownie-cake
force-pushed
the
fix/jdbc-source-read-transaction-cleanup
branch
from
August 12, 2026 10:59
054f94f to
7e92698
Compare
End cursor/manual transactions when a JDBC source split closes, restore the configured auto-commit mode, and discard only connections whose transaction state cannot be cleaned safely. Add regression coverage for clean failure recovery and multi-split connection reuse.
brownie-cake
force-pushed
the
fix/jdbc-source-read-transaction-cleanup
branch
from
August 12, 2026 11:05
7e92698 to
7f70804
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of this pull request
Fix JDBC source sessions that can remain
idle in transactionafter a source split has finished reading or has failed during initialization.PostgreSQL and Redshift cursor reads disable auto-commit in their dialect-specific prepared-statement creation.
JdbcInputFormat.close()currently closes only theResultSetandPreparedStatement; closing those resources does not end a manual JDBC transaction. Because the source connection is reused across splits, the database backend can retain the old transaction snapshot while the reader is waiting for another split or while the rest of the job is being cleaned up.On PostgreSQL-compatible databases this is visible as sessions with:
The retained transaction can hold
backend_xmin, delay vacuum/tuple cleanup, retain transaction-scoped locks and consume connection slots.This patch closes the transaction at the JDBC source split lifecycle boundary:
close()calls idempotent;closeInputFormat()throughfinally.rollback()is intentional. JDBC source processing is read-only from the connector's perspective. Rolling back ends the server-side cursor transaction, releases its snapshot and also clears a transaction that entered an aborted state. Cursor-based incremental fetching remains enabled; the patch does not force PostgreSQL/Redshift reads into auto-commit mode while the result set is active.Does this PR introduce any user-facing change?
Yes, as a bug fix.
Before this change, PostgreSQL-compatible JDBC source sessions could remain in
idle in transactionafter a split completed, until the whole reader connection was eventually closed.After this change, the read transaction ends immediately at split cleanup. The configured auto-commit mode is restored before the connection is reused.
There is no configuration-key, default-value or public-API change. JDBC Sink, XA and exactly-once commit semantics are not modified.
How was this patch tested?
Added
JdbcInputFormatTestcoverage for:truetofalse; cleanup rolls back and restorestrue.auto_commit=false: cleanup rolls back while preserving manual-commit mode.finally.close()calls: cleanup is safe and idempotent.The production and test sources were also compiled against Java 8-compatible type stubs, and the transaction state transitions were exercised with a standalone JDBC
Connectionproxy harness. The repository's connector-JDBC unit, formatting and static checks are delegated to this PR's GitHub CI.Check list