Skip to content

Fix shard distribution bias: restore default expiration, add WithMaxKeyLength docs, and getShardID tests#150

Merged
kpango merged 2 commits intobugfix/shard-distribution-biasfrom
copilot/sub-pr-148
Feb 25, 2026
Merged

Fix shard distribution bias: restore default expiration, add WithMaxKeyLength docs, and getShardID tests#150
kpango merged 2 commits intobugfix/shard-distribution-biasfrom
copilot/sub-pr-148

Conversation

Copy link

Copilot AI commented Feb 25, 2026

Shard ID computation had a logic bug when maxKeyLength=0 (hashed empty string instead of full key), the 30s default expiration was silently dropped from New[V](), and the new WithMaxKeyLength option lacked documentation.

Changes

  • Restore default expiration: New[V]() now applies WithDefaultExpiration[V](30 * time.Second) alongside WithMaxKeyLength[V](256) — its removal was an unintended breaking change
  • Fix getShardID logic: normalize kl=0 to use the full key length before branching; clamp kl to min(len(key), kl) consistently across both paths
  • Document WithMaxKeyLength: clarifies the 256-byte default, the maphash/xxh3 selection threshold (≤32 bytes vs longer), and the meaning of kl=0
  • Add getShardID unit tests: covers all behavioral boundaries — kl=0, kl=1, kl∈[2,32], kl>32, key shorter than kl, prefix isolation, and result range invariants
// Before: kl=0 caused key[:0] (empty string) to be hashed → all short keys → same shard
// After: kl=0 falls through to full-key path correctly
func getShardID(key string, kl uint64) (id uint64) {
    if kl != 0 {
        kl = min(uint64(len(key)), kl)
        // ... maphash or xxh3 on key[:kl]
    }
    // full key path when kl == 0
}

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: kpango <9798091+kpango@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix shard distribution bias due to key length and structure Fix shard distribution bias: restore default expiration, add WithMaxKeyLength docs, and getShardID tests Feb 25, 2026
Copilot AI requested a review from kpango February 25, 2026 03:43
@kpango kpango marked this pull request as ready for review February 25, 2026 03:45
@kpango kpango merged commit 5488f17 into bugfix/shard-distribution-bias Feb 25, 2026
2 checks passed
@kpango kpango deleted the copilot/sub-pr-148 branch February 25, 2026 03:47
kpango added a commit that referenced this pull request Feb 25, 2026
…eyLength docs, and getShardID tests (#150)

* Initial plan

* Restore default 30s expiration and add docs to WithMaxKeyLength

Co-authored-by: kpango <9798091+kpango@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kpango <9798091+kpango@users.noreply.github.com>
Signed-off-by: kpango <kpango@vdaas.org>
kpango added a commit that referenced this pull request Feb 25, 2026
Apply suggestions from code review
Add unit tests for getShardID (#149)
Fix shard distribution bias: restore default expiration, add WithMaxKeyLength docs, and getShardID tests (#150)

Signed-off-by: kpango <kpango@vdaas.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants