Skip to content

fix(scoring): use normalised key weights in _searchObjectList#834

Closed
JSap0914 wants to merge 1 commit into
krisk:mainfrom
JSap0914:fix/key-weight-normalization-in-object-search
Closed

fix(scoring): use normalised key weights in _searchObjectList#834
JSap0914 wants to merge 1 commit into
krisk:mainfrom
JSap0914:fix/key-weight-normalization-in-object-search

Conversation

@JSap0914

Copy link
Copy Markdown
Contributor

Problem

_searchObjectList built its key list from this._myIndex.keys, which stores the raw user-specified weights — before the normalisation that KeyStore performs (weights sum to 1). _searchObjectList never read from KeyStore, so it applied the original values directly.

This caused two observable bugs:

1. Score underflow for extreme weight values

A single key always normalises to weight = 1.0 (only key = 100 % share). With a raw weight of 100, the exact-match formula becomes:

Math.pow(Number.EPSILON, 100 * fieldNorm)  // underflows to exactly 0

whereas the normalised exponent (1.0) gives ~2.2e-16.

const f1   = new Fuse([{ text: 'hello' }], { keys: [{ name: 'text', weight: 1   }], includeScore: true })
const f100 = new Fuse([{ text: 'hello' }], { keys: [{ name: 'text', weight: 100 }], includeScore: true })

f1.search('hello')[0].score   // ~2.2e-16  ✓
f100.search('hello')[0].score // 0         ✗  (underflow)

2. Score inconsistency between query forms

  • Plain string queries → _searchObjectListraw weights
  • Logical Expression queries → _searchLogicalthis._keyStore.get(keyId)normalised weights

Identical searches against identical data produced different absolute scores depending solely on how the query was expressed.

Fix

Replace

const { keys, records } = this._myIndex   // raw weights

with

const { records } = this._myIndex
const keys = this._keyStore.keys()        // normalised weights

KeyStore and FuseIndex both iterate this.options.keys in the same order, so the positional indexing item[keyIndex] is unaffected.

Tests

Three new tests in test/key-weight-normalization.test.js (import from source):

Test RED before GREEN after
weight:1 and weight:2 (single key) produce the same score
weight:100 does not underflow to 0
Scaling all weights by the same factor does not change ordering or scores

All 374 existing tests continue to pass.

Test Files  21 passed (21)
     Tests  374 passed (374)

Closes #833

_searchObjectList built its key list from this._myIndex.keys, which
stores the raw user-specified weights (before the normalisation that
KeyStore performs). KeyStore normalises weights so they sum to 1, but
_searchObjectList never read from KeyStore — it used the original
values directly.

This caused two bugs:

1. Score underflow for extreme weight values.
   A single key always normalises to weight=1.0 (only key = 100%),
   so 'weight:1' and 'weight:100' must produce the same score.
   With the raw weight in the exponent the formula
     Math.pow(Number.EPSILON, 100 * fieldNorm)
   underflows to exactly 0, whereas the normalised exponent (1)
   gives ~2.2e-16.

2. Score inconsistency between query forms.
   Plain string queries went through _searchObjectList (raw weights);
   logical Expression queries went through _searchLogical which
   already called this._keyStore.get(keyId) (normalised weights).
   Identical searches produced different absolute scores depending
   on how the query was expressed.

Fix: replace 'const { keys, records } = this._myIndex' with
'const keys = this._keyStore.keys()' so the keys seen by
_findMatches — and therefore by computeScoreSingle — carry
the same normalised weights that KeyStore guarantees.

Closes krisk#833
Copilot AI review requested due to automatic review settings June 30, 2026 03:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@krisk

krisk commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Thanks, the diagnosis is spot on. _searchObjectList scored off the raw index weights while the keyed logical path already went through the KeyStore, so the two disagreed and big weights underflowed.

I landed a slightly wider fix on main instead of merging this directly, for two reasons:

  1. The keyless logical form had the same bug. search({ $or: ['hello'] }) also read this._myIndex.keys, so weight: 100 still underflowed there even with _searchObjectList fixed. Wanted all three forms consistent.

  2. Swapping in _keyStore.keys() positionally misaligns if a prebuilt index's key order differs from options.keys. I used a small _normalizedKeys() helper that looks each key up by id, so the weight stays pinned to the right value.

Your tests carried over almost as-is. Closing in favor of the version on main. Credit's yours.

@krisk krisk closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: _searchObjectList uses raw (unnormalized) key weights, causing score underflow for extreme weight values

3 participants