core: scan a copy-on-write snapshot in GetRegionSizeByRange - #11140
core: scan a copy-on-write snapshot in GetRegionSizeByRange#11140vldmit wants to merge 6 commits into
Conversation
Three defects made the existing benchmarks unable to measure the region heartbeat and range-scan paths: - BenchmarkRandomSetRegionWithGetRegionSizeByRange and its Parallel variant scanned an empty key range, which is answered from regionTree.totalSize in O(1) and never reaches the range scan. Both now scan a partial range. - Their background reader goroutines never stopped, so they outlived their benchmark and perturbed every later one in the same binary. - BenchmarkRandomSetRegion mutates an already published *RegionInfo and re-reports the same pointer, so origin == region and updateStat adds and subtracts the same values. It does not model a heartbeat. Add benchmarks for the three paths that matter: - BenchmarkSetRegionSameRange alternates two immutable same-range versions of every region, which is what a region heartbeat does. - BenchmarkSetRegionRangeChanged alternates two end keys, the only heartbeat shape that touches the btree structure. - BenchmarkBatchScanRegionsHighConcurrency runs many short scans concurrently with heartbeat writers and point readers, which is the realistic client shape; a single wide scan amortises away the per-call cost. Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
regionItem embedded a *RegionInfo, and setRegionLocked replaces that pointer in place while other goroutines may be reading the item. Today every reader holds RegionsInfo.t, so the replacement is safe; it stops being safe as soon as anything reads a region tree without that lock. Replace the embedded pointer with an atomic.Pointer slot reached through getRegion/setRegion. atomic.Pointer is one word, so regionItem stays 8 bytes and there is no per-region memory cost. Two notes recorded on the type, because both are load-bearing and neither was written down before: - Ordering immutability: an item that is in a tree must never change its key range. This is what a later copy-on-write snapshot will depend on. - Publish-once: the atomic protects the slot, not the RegionInfo behind it, so a stored *RegionInfo must not be mutated afterwards. RegionInfo.ref and RegionInfo.reportBuckets are the exceptions, as both carry their own atomic. Dropping the embedded field also removes the shadowing of RegionInfo's GetID, GetStartKey and GetEndKey by the identically named methods on regionItem, which previously made it impossible to tell at a call site which of the two was being invoked. No behaviour change. Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
setRegionLocked reused the regionItem from r.regions even when the region's key range changed, rewriting the item's start and end keys in place. That is safe only while every reader holds RegionsInfo.t, because the item's keys are what orders it in the btree: a reader holding a reference to that item across the change would see it at one position while it reports another range. Allocate a fresh item on the range-changed path instead and repoint the map entry. The path already performs tree.remove plus overlaps, Delete and ReplaceOrInsert, so one 8-byte allocation is not measurable against it. The same-range path is untouched and keeps replacing the value in place, which is what the region heartbeat does. The reference count is unchanged. tree.remove still finds the old item, which still holds the origin, so origin loses its reference; tree.update then adds one for the region behind the fresh item. Two things that were previously relied on but not written down are now comments: that tree.remove has to run before the item is replaced, and why the new map assignment cannot be undone by the overlap cleanup that follows. TestSetRegionRangeChangedRepointsMap covers the map, the tree and the reference counts together. Without the repointing it fails with the pre-shrink region. Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
GetRegionSizeByRange walked the region tree in ScanRegionLimit-sized chunks, releasing and immediately re-acquiring RegionsInfo.t between them. On a cluster with 1.5M regions one call therefore took the lock about 1500 times, and because Go's RWMutex is write-preferring, each acquisition let at most one waiting writer through. checkStores calls this for every preparing store on a 10s ticker, so a scale-out could hold region-heartbeat writes near that rate for as long as the stores stayed in the preparing state. Chunking is also not exact. Each chunk resumed at the previous region's end key, and scanRange resolves a key to the region containing it, so a merge across a chunk boundary made the merged region contribute its whole size to a range that had already been partly counted. pkg/btree is copy-on-write and its Clone is O(1), so add rootRangeSnapshot: a clone of the root tree that can be walked with no lock at all. Taking it needs the write lock, because Clone rewrites the source tree's copy-on-write context, but that critical section is O(1) regardless of region count. A snapshot freezes the tree's shape, not its values. The RegionInfo behind an item may still be replaced concurrently, though only by one covering the same key range, so a scan sees exactly one region per key. That is weaker than a scan under one read lock, where every region comes from a single instant, and the type documents the difference. Measured on 1M regions, arm64, with a partial-range scanner running against concurrent writers, writer cost per operation: GOMAXPROCS=1 2.1-6.2us -> 1.3-2.8us GOMAXPROCS=4 12.1-12.9us -> 0.81-1.19us GOMAXPROCS=16 13.7-15.9us -> 1.08-1.16us BatchScanRegions is deliberately left on the read lock. It has the opposite shape -- short scans at a high call rate -- so an exclusive clone per call costs much more than the read lock it would replace: eight-region scans went from 3.1us to 20.7us at GOMAXPROCS=4. Only a shared snapshot invalidated on structural change would suit that caller. A comment records the measurement so the next reader does not have to rediscover it. Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
btree Clone was covered by one test that compares Ascend order and nothing else,
while the test that checks the per-node index bookkeeping never clones. Nothing
exercised mutableFor's copying of the indices array, even though GetAt and
GetWithIndex depend on it, and through them GetCountByRange and RandomRegions. A
fault there would return a wrong rank or a wrong item with no panic and no error.
- TestBTreeSizeInfoAfterCloneG clones repeatedly while filling and then emptying
a tree, and re-checks the length, rank and index invariants on the original and
on every clone.
- TestCloneReadWhileWritingOriginalG reads a clone while the original is written,
which is how PD uses Clone. Writes are serialised, because BTreeG does not
support concurrent writers; only the lock-free reads are concurrent.
Both pass at degree 2, 32 and 64.
On the core side:
- TestRootRangeSnapshotIsolation applies a same-range update, a split and a
removal after taking a snapshot, and pins down both halves of the contract: the
key ranges are frozen, the values are not.
- TestRootRangeSnapshotEquivalence checks a snapshot against the live tree, checks
GetRegionSizeByRange against a brute-force sum over ScanRegions for random
ranges, and checks that taking a snapshot changes no reference count, which is
what keeps it O(1) and keeps RegionInfo.ref meaning "present in a tree".
- TestGetRegionSizeByRangeConcurrent runs the lock-free walk against writers
doing same-range updates, shrinks and restores, and cross-checks the subtrees
afterwards.
- TestRegionItemRangeImmutability holds a snapshot across a thousand updates and
re-checks the ordering of the live tree and the snapshot after each one.
- TestRootRangeSnapshotAdjacencyUnderMerges checks that a snapshot of a
contiguous key space stays contiguous while the tree changes underneath it.
Merges drive the churn, and the split half of each cycle is made atomic with
respect to taking a snapshot: a split reaches PD as two separate updates, so
between them the key space really is missing a range and a scanner is right to
report a hole however it locks.
- TestGetRegionSizeByRange gains cases for boundaries inside a region, a range
past the last region's start key, and an empty range.
The last two fail without the fresh-item allocation, reporting start keys that
go backwards -- the signature of a shared item that moved in one tree while a
clone still holds it at its old position.
Benchmarks for the snapshot itself, on 1M regions, arm64:
BenchmarkSnapshotRootTree ~95-107ns, 4 allocs, flat from
100k to 4M regions
BenchmarkSetRegionSameRangeWithSnapshot ~365ns, 0 allocs, same as without
...RangeChangedWithSnapshot 2557ns vs 2496ns, +36B/op
Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
Satisfies the unparam linter. Signed-off-by: Vlad Dmitriev <vldmit@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @vldmit. Thanks for your PR. I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Welcome @vldmit! |
📝 WalkthroughWalkthroughThe change adds atomic region-item storage, copy-on-write root snapshots, reusable range scans, and snapshot-based region-size queries. Tests cover index consistency, snapshot isolation, range updates, concurrent access, reference counts, and benchmark workloads. ChangesRegion tree snapshot flow
Estimated code review effort: 4 (Complex) | ~60 minutes Mergeability Score: ⚪ Minimal · up to The change moves range scans onto copy-on-write snapshots to reduce lock contention and corrects merge-boundary size overcounting; no actionable merge-blocking risk remains beyond normal checks and review. Sequence Diagram(s)sequenceDiagram
participant RegionsInfo
participant snapshotRootTree
participant rootRangeSnapshot
RegionsInfo->>snapshotRootTree: clone root tree under lock
snapshotRootTree-->>RegionsInfo: return rootRangeSnapshot
RegionsInfo->>rootRangeSnapshot: scan requested range
rootRangeSnapshot-->>RegionsInfo: return region size
Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
pkg/core/region.go (1)
1456-1467: 🚀 Performance & Scalability | 🔵 TrivialConsider recording the snapshot lock-hold time.
snapshotRootTreetakes the exclusiver.tlock on every partial-range call. The critical section is O(1), so the hold time should stay flat as the region count grows. A metric or the existingRWLockStatscounters would make a regression visible in production instead of only in benchmarks.This is optional. No behavior change is required.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/core/region.go` around lines 1456 - 1467, Optionally instrument snapshotRootTree to record the duration of its exclusive r.t lock hold, reusing the existing metric or RWLockStats mechanism if available. Start timing immediately before r.t.Lock and record the elapsed duration after unlocking, without changing snapshot behavior or lock scope.pkg/core/region_test.go (1)
1150-1155: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRotate the published version in the background writer.
The writer picks
versions[i]and publishes the same*RegionInfopointer many times. On repeat writes for the same ID,origin == region, soupdateStatadds and subtracts identical values andupdateRefdecrements and increments the same object. That is the shape the new comment onBenchmarkRandomSetRegionat Line 806 warns about, so the background load is slightly less realistic than a heartbeat.Two pre-built versions per region would keep
origin != regionon every write.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/core/region_test.go` around lines 1150 - 1155, Update the background writer in the region test to rotate between two pre-built RegionInfo versions for each region, selecting a version that differs from the currently published one on every write. Preserve the existing SetRegion and UpdateSubTree flow while ensuring repeated writes for the same ID use distinct pointers.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@pkg/core/region_test.go`:
- Around line 1150-1155: Update the background writer in the region test to
rotate between two pre-built RegionInfo versions for each region, selecting a
version that differs from the currently published one on every write. Preserve
the existing SetRegion and UpdateSubTree flow while ensuring repeated writes for
the same ID use distinct pointers.
In `@pkg/core/region.go`:
- Around line 1456-1467: Optionally instrument snapshotRootTree to record the
duration of its exclusive r.t lock hold, reusing the existing metric or
RWLockStats mechanism if available. Start timing immediately before r.t.Lock and
record the elapsed duration after unlocking, without changing snapshot behavior
or lock scope.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 45966cae-ba3c-44fd-9cc5-66fb0ffe8a89
📒 Files selected for processing (5)
pkg/btree/btree_generic_test.gopkg/core/region.gopkg/core/region_test.gopkg/core/region_tree.gopkg/core/region_tree_test.go
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #11140 +/- ##
==========================================
- Coverage 79.43% 79.42% -0.02%
==========================================
Files 542 542
Lines 77117 77127 +10
==========================================
- Hits 61259 61258 -1
- Misses 11571 11584 +13
+ Partials 4287 4285 -2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
What problem does this PR solve?
Issue Number: ref #9574
Problem. While a store is
Preparing,checkStorescallsgetThresholdfor it everynodeStateCheckJobInterval. With placement rules that split the keyspace, that reachesGetRegionSizeByRangewith a non-empty range, which walks the region tree inScanRegionLimit-sized chunks and releases and immediately re-acquiresRegionsInfo.tbetween them. On a cluster with 1.5M regions one call therefore takes the lock about 1500
times. Go's
sync.RWMutexis write-preferring, so each of those acquisitions lets at most onewaiting writer through, and region-heartbeat writes are held near that rate for as long as the
stores stay in
Preparing.This is the part of #9574 that the two merged changes do not reach. As @lhy1024 summarised on
that issue, #9580 removed the full-range case and #11072 removed the
Preparing stores x matching rulesamplification, but "each unique range is still scanned inO(N) time once per round".
This PR takes a different angle from the two options listed there. It does not make the
query O(log N), and it does not reduce the CPU spent walking. It removes the lock coupling
instead: the walk runs on a snapshot with no lock held at all. That is deliberate, because the
harmful part of the walk is not its CPU cost — during the incident that motivated this work PD
was using 5-10 of 64 cores with no GC pressure — but the ~1500 lock hand-offs it forces on the
heartbeat writers. An O(log N) weighted aggregate is still worth doing and is complementary;
note though that maintaining a range-size aggregate means updating a root-to-leaf sum whenever
approximate_sizechanges, which puts O(log n) work on the region-heartbeat path, whereas thischange puts none there.
Why not chunk the other scans as well. #9671 tried that for
ScanRegionsand #9844 revertedit, to preserve "view consistency of the region between two scans". That objection is right and
it is inherent to chunking: re-entering the tree at the last end key lets the key space change
between chunks. A copy-on-write snapshot has no second scan to be inconsistent with, so it gives
a bounded lock hold and a consistent key-range view.
It also fixes a small correctness bug. The chunked loop resumed each chunk at the previous
region's end key, and
scanRangeresolves a key to the region containing it, so a merge acrossa chunk boundary made the merged region contribute its whole size to a range that had already
been partly counted.
What is changed and how does it work?
Three steps, one per commit, plus benchmarks and tests.
1.
regionItemholds its*RegionInfoin anatomic.Pointer.setRegionLockedreplaces that pointer while other goroutines may read the item. That is safeonly while every reader holds
RegionsInfo.t.atomic.Pointeris one word, soregionItemstays 8 bytes and there is no per-region memory cost. Dropping the embedded field also removes
the shadowing of
RegionInfo'sGetID,GetStartKeyandGetEndKeyby the identically namedmethods on
regionItem.2. A fresh
regionItemon range change. An item that is in a tree must never change its keyrange, because its keys are what orders it in the btree: a reader holding a reference across the
change would see it at one position while it reports another range.
setRegionLockednowallocates a new item on the range-changed path and repoints
r.regions. That path already doestree.removeplusoverlaps,DeleteandReplaceOrInsert, so one 8-byte allocation is notmeasurable against it. The same-range path is untouched and still replaces the value in place,
which is what a region heartbeat does. The reference count is unchanged.
3.
rootRangeSnapshot, andGetRegionSizeByRangewalks one.pkg/btreeis already copy-on-write and itsCloneis O(1), but nothing called it. Taking asnapshot needs the write lock, because
Clonerewrites the source tree's copy-on-write context,but that critical section is O(1) regardless of region count.
Consistency contract, stated plainly. A snapshot freezes the tree's shape — which items
exist, their key ranges, the coverage, the order, the length. It does not freeze the values:
the
RegionInfobehind an item may still be replaced concurrently, though only by one coveringthe same key range, so a scan sees exactly one region per key. That is weaker than a scan
under one read lock, where every region comes from a single instant. The type documents the
difference, and callers that need agreement between two different regions' values rather than
agreement about which key ranges exist must not use a snapshot. A snapshot also takes no
reference on the regions it holds, since that would make it O(n) and
RegionInfo.refcarries thefunctional "already in the subtree" signal.
BatchScanRegionsis deliberately left on the read lock. I tried converting it, because itis the function #9844 reverted chunking on, and measured a large regression: it has the opposite
shape, short scans at a high call rate, so an exclusive clone per call costs much more than the
read lock it replaces. Eight-region scans went from 3.1us to 20.7us at GOMAXPROCS=4. Only a
shared snapshot invalidated on structural change would suit that caller. A comment in the code
records the measurement so the next reader does not have to rediscover it.
Measurements
arm64 (Apple M4 Max), 1M regions,
benchstat, before/after measured back to back in the samethermal state. Not yet measured on amd64.
The path this PR is about — region-heartbeat writers with a partial-range scanner running
concurrently,
BenchmarkRandomSetRegionWithGetRegionSizeByRangeParallel:Cost added elsewhere:
The same-range write path is the one that matters for acceptance, and it is unchanged including
allocations, with and without a snapshot outstanding. It never calls
ReplaceOrInsertorDelete, so it never reaches btree's copy-on-write node copy.Copy-on-write is paid only by structural writes, and less than I expected: re-taking a snapshot
every 1000 structural writes costs +36 B/op over not having one, because most writes hit paths
the tree already owns.
An item layout with an immutable
startKey, which would keep the atomic off the btreecomparison path and make the ordering invariant structural, was prototyped and rejected: it
recovered ~1.7% on the structural path but quadrupled its allocation (40 -> 160 B/op, since every
scratch item grows from 8 to 32 bytes), on top of ~36 MB of steady state at 1.5M regions.
Check List
Tests
New tests, all passing under
-race:pkg/btree:TestBTreeSizeInfoAfterCloneGandTestCloneReadWhileWritingOriginalG.Clonewas covered by one test that compares
Ascendorder and nothing else, while the test thatchecks the per-node
indicesbookkeeping never clones. Nothing exercisedmutableFor'scopying of
indices, even thoughGetAtandGetWithIndexdepend on it, and through themGetCountByRangeandRandomRegions. Both pass at degree 2, 32 and 64.pkg/core:TestRootRangeSnapshotIsolation,TestRootRangeSnapshotEquivalence,TestGetRegionSizeByRangeConcurrent,TestRegionItemRangeImmutability,TestRootRangeSnapshotAdjacencyUnderMerges,TestSetRegionRangeChangedRepointsMap, and extracases on
TestGetRegionSizeByRangefor boundaries inside a region, a range past the lastregion's start key, and an empty range.
The adjacency test uses merges rather than splits on purpose: a split reaches PD as two separate
updates, the shrunk parent and then the new child, so between them the key space really is
missing a range and a scanner is right to report a hole however it locks. The split half of each
cycle is therefore made atomic with respect to taking a snapshot.
Two of the tests fail without the fresh-item allocation, reporting start keys that go backwards,
which is the signature of a shared item that moved in one tree while a clone still holds it at
its old position.
Benchmark hygiene, in the first commit, so the before/after above compares like with like:
BenchmarkRandomSetRegionWithGetRegionSizeByRangeand itsParallelvariant scanned anempty key range, which is answered from
regionTree.totalSizein O(1) and never reachesthe range scan. Both now scan a partial range.
perturbed every later one in the same binary.
BenchmarkRandomSetRegionmutates an already published*RegionInfoand re-reports the samepointer, so
origin == regionandupdateStatadds and subtracts the same values. It does notmodel a heartbeat, so
BenchmarkSetRegionSameRangewas added, alternating two immutablesame-range versions of every region.
Code changes
Side effects
(split/merge/create/delete). The dominant same-range heartbeat path is unchanged. See
Measurements.
Related changes
yet on release-8.5, and this change sits on top of them.
Release note
Summary by CodeRabbit
Performance
Reliability