[server] Make leader-only rebalance tasks execute sequentially#3071
Open
swuferhong wants to merge 1 commit intoapache:mainfrom
Open
[server] Make leader-only rebalance tasks execute sequentially#3071swuferhong wants to merge 1 commit intoapache:mainfrom
swuferhong wants to merge 1 commit intoapache:mainfrom
Conversation
swuferhong
force-pushed
the
rebalance-leader-switch
branch
from
eda0fad to
e60a784
Compare
swuferhong
force-pushed
the
rebalance-leader-switch
branch
from
e60a784 to
6bf8f61
Compare
LiebingYu
reviewed
Apr 14, 2026
Contributor
LiebingYu
left a comment
LiebingYu
left a comment
There was a problem hiding this comment.
@swuferhong Thanks for the PR, only minor comments for me, not block the PR.
| // server to acknowledge the leader change before proceeding to the next migration. | ||
| for (NotifyLeaderAndIsrResultForBucket notifyLeaderAndIsrResultForBucket : | ||
| notifyLeaderAndIsrResultForBuckets) { | ||
| tryToCompleteRebalanceTask(notifyLeaderAndIsrResultForBucket.getTableBucket()); |
Contributor
There was a problem hiding this comment.
After switching to sequential leader transfer one bucket at a time, a new issue arises: if the leader transfer fails for any single bucket, it may block all subsequent leader transfers. However, I understand it is acceptable to not address this issue for now, but users must manually handle such blocking situations. WDYT?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: #3070
Previously, leader-only rebalance tasks (where only the leader changes
but the replica set stays the same) were completed immediately after
triggering the leader election, without waiting for the TabletServer
to acknowledge the change. This caused all leader migrations to fire
simultaneously, putting excessive pressure on tablet servers, especially
for KV tables.
This patch removes the immediate
finishRebalanceTask()call forleader-only changes and instead completes the task in
processNotifyLeaderAndIsrResponseReceivedEvent()when the TabletServerresponds. This ensures leader migrations execute one at a time, matching
the behavior of replica migration tasks.
Also fixes
TestTabletServerGatewayto return per-bucket success resultsin
NotifyLeaderAndIsrresponses, which is necessary for the coordinatorto identify acknowledged buckets in the response callback.
Brief change log
Tests
API and Format
Documentation