Skip to content

Align default BanyanDB stage segmentInterval to integer multiples#13910

Open
mrproliu wants to merge 3 commits into
apache:masterfrom
mrproliu:fix/banyandb-stage-segment-interval-multiple
Open

Align default BanyanDB stage segmentInterval to integer multiples#13910
mrproliu wants to merge 3 commits into
apache:masterfrom
mrproliu:fix/banyandb-stage-segment-interval-multiple

Conversation

@mrproliu

Copy link
Copy Markdown
Contributor

Improve the efficiency of BanyanDB hot → warm → cold lifecycle stage migration (default config)

Each coarser lifecycle stage's segmentInterval should be an integer multiple of the immediately finer (preceding) stage's. BanyanDB segments are epoch-anchored, so when the coarser interval is a multiple of the finer one, every source segment nests entirely inside exactly one target segment and the tier migration stays on the cheap whole-segment fast path (byte-copy). A non-multiple interval makes source segments straddle target boundaries, forcing a row-by-row decode-and-re-bucket slow path that costs significantly more CPU, IO, and peak memory, and produces more small parts for the background merge loop to compact. It does not affect correctness — only migration efficiency.

This aligns the shipped defaults that previously violated the rule:

Group Field Old → New
records cold segmentInterval 3 → 4 (multiple of warm=2)
metricsMinute cold segmentInterval 5 → 6 (multiple of warm=3)
metricsHour warm segmentInterval 7 → 10 (multiple of hot=5)
metricsHour cold segmentInterval 15 → 20 (multiple of warm=10)

All adjusted values still satisfy hot ≤ warm ≤ cold and segmentInterval ≤ that stage's ttl. The other groups already comply and are unchanged.

  • The improvement is structural (whole-segment byte-copy vs row-by-row re-bucketing), not a micro-optimization, so no micro-benchmark applies; the fast/slow path mechanism is documented in the stages guide.

  • Theory proof: see the new "integer multiple" guideline in docs/en/banyandb/stages.md, and the BanyanDB migration fast/slow path behavior.

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

  • Update the CHANGES log.

Make each coarser lifecycle stage's segmentInterval an integer multiple of
the finer (preceding) stage's, so hot -> warm -> cold migration stays on the
cheap whole-segment fast path instead of the row-by-row slow path. Adjusted
defaults: records cold 3->4, metricsMinute cold 5->6, metricsHour warm 7->10
and cold 15->20. Also document the multiple rule in the stages guide.
@mrproliu mrproliu added the enhancement Enhancement on performance or codes label Jun 14, 2026
@mrproliu mrproliu added this to the 11.0.0 milestone Jun 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request updates SkyWalking’s shipped BanyanDB default lifecycle stage segmentInterval values so that each coarser stage uses an integer-multiple interval of the preceding stage, improving hot → warm → cold migration efficiency by keeping migrations on the whole-segment fast path.

Changes:

  • Adjust default segmentInterval values in BanyanDB bydb.yml (runtime + test resource) for records, metricsMinute, and metricsHour to be integer multiples across stages.
  • Update the configuration vocabulary documentation to reflect the new defaults.
  • Document the “integer multiple” guideline in the BanyanDB stages guide and record the change in the changelog.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
oap-server/server-storage-plugin/storage-banyandb-plugin/src/test/resources/bydb.yml Updates test fixture defaults for stage segmentInterval values to match the new integer-multiple alignment.
oap-server/server-starter/src/main/resources/bydb.yml Updates shipped default BanyanDB stage segmentInterval values for more efficient lifecycle migrations.
docs/en/setup/backend/configuration-vocabulary.md Updates documented default values for the affected segmentInterval settings.
docs/en/changes/changes.md Adds a changelog entry describing the default segmentInterval alignment improvement.
docs/en/banyandb/stages.md Updates the stages guide example default and adds an “integer multiple” guideline to explain migration efficiency implications.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/en/setup/backend/configuration-vocabulary.md
Comment thread docs/en/banyandb/stages.md Outdated
cold:
shardNum: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SHARD_NUM:2}
segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:5}
segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:6}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the stages.md from metricsMin to metricsMinute.

…mple naming

- Update the third copy of the BanyanDB defaults in storages/banyandb.md to the
  new integer-multiple values (records cold 4, metricsMinute cold 6, metricsHour
  warm 10 / cold 20), matching bydb.yml and configuration-vocabulary.md.
- Align the stages.md example to the real config: group metricsMin -> metricsMinute
  and env vars SW_STORAGE_BANYANDB_GM_MINUTE_* -> SW_STORAGE_BANYANDB_METRICS_MINUTE_*
  (the GM_MINUTE names do not exist in any shipped config).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Enhancement on performance or codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants