Support multiple regions in MIGRATE REGION and multiple DataNodes in REMOVE DATANODE#18046
Open
CRZbulabula wants to merge 2 commits into
Open
Support multiple regions in MIGRATE REGION and multiple DataNodes in REMOVE DATANODE#18046CRZbulabula wants to merge 2 commits into
CRZbulabula wants to merge 2 commits into
Conversation
MIGRATE REGION previously accepted a single region id. Extend it to accept a comma-separated list of region ids while keeping a single source (FROM) and destination (TO) DataNode: MIGRATE REGION 1, 2, 3 FROM 4 TO 5 The list of regions is migrated from the source DataNode to the destination DataNode and the per-region results are aggregated into the response sub-status, mirroring the existing EXTEND/REMOVE REGION reporting. Both the tree model and table model grammars are updated. Changes span grammar (IoTDBSqlParser.g4, RelationalSql.g4), the thrift TMigrateRegionReq (regionId -> list<i32> regionIds), the parsers, the MigrateRegion AST node / MigrateRegionStatement, and ProcedureManager, which now resolves the fixed source/destination DataNodes once and loops over the deduped region ids submitting one RegionMigrateProcedure each. Adds tree- and table-model parser unit tests and a 1C5D integration test that migrates multiple regions in a single statement.
REMOVE DATANODE previously accepted a single DataNode id. Extend it to accept a comma-separated list of DataNode ids in one statement: REMOVE DATANODE 3, 4, 5 The removal already flowed through a multi-node path internally (RemoveDataNodeStatement holds a Set, TDataNodeRemoveReq carries a list<TDataNodeLocation> and RemoveDataNodesProcedure is plural); only the grammar, parsers and a 'size() != 1' guard restricted it to one id. Both the tree model and table model grammars now accept a list, the parsers collect every id, and ClusterConfigTaskExecutor.removeDataNode fails the whole statement (instead of silently dropping ids) when any requested id is not a registered DataNode, and rejects an empty set. Adds tree- and table-model parser unit tests and enables a 1C5D integration test that removes two DataNodes in a single statement.
CRZbulabula
changed the title
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18046 +/- ##
============================================
+ Coverage 41.40% 41.41% +0.01%
Complexity 318 318
============================================
Files 5286 5286
Lines 369547 369583 +36
Branches 47815 47820 +5
============================================
+ Hits 153008 153071 +63
+ Misses 216539 216512 -27 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two cluster-management commands that previously accepted a single id now accept a comma-separated list in one statement (the single-id form keeps working):
Why
Operators draining or rebalancing nodes routinely act on several regions / DataNodes at once. Today that needs one statement per target. Accepting a list makes these one-liners and aligns
MIGRATE REGIONwith the multi-region shape already used byEXTEND/REMOVE/RECONSTRUCT REGION.How
MIGRATE REGION — multiple regions, fixed FROM/TO
migrateRegion/migrateRegionStatementacceptregionIds+=INTEGER_LITERAL (COMMA regionIds+=INTEGER_LITERAL)*.TMigrateRegionReq:i32 regionId->list<i32> regionIds.MigrateRegion, statementMigrateRegionStatement: carryList<Integer> regionIds.ProcedureManager#migrateRegion: resolves the fixed source/destination DataNodes once, then loops over the de-duplicated region ids submitting oneRegionMigrateProcedureper region. Per-region results are aggregated into theTSStatussub-status, mirroringEXTEND/REMOVE REGIONreporting. Unknown source/destination ids fail fast with a clear message.REMOVE DATANODE — multiple DataNodes
removeDataNode/removeDataNodeStatementacceptdataNodeIds+=INTEGER_LITERAL (COMMA dataNodeIds+=INTEGER_LITERAL)*.RemoveDataNodeStatementholds aSet,TDataNodeRemoveReqcarrieslist<TDataNodeLocation>, andRemoveDataNodesProcedureis plural. The only restriction was asize() != 1guard inClusterConfigTaskExecutor#removeDataNode, now replaced so the statement fails as a whole (instead of silently dropping ids) when any requested id is not a registered DataNode, and rejects an empty set.Tests
MIGRATE REGIONandREMOVE DATANODE(4 new test classes).IoTDBMigrateMultiRegionForIoTV1IT(1C5D): migrates multiple regions from one DataNode to another in one statement, asserting every region left the source and joined the destination.IoTDBRemoveDataNodeNormalITthat removes two DataNodes in a single statement.Build verified: thrift + ANTLR regeneration,
iotdb-core/datanodeandiotdb-core/confignodecompile,integration-testtest-compiles,spotless:checkpasses, and all new parser unit tests pass.