Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/index-optimization-partial-and-or.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
'@tanstack/db': patch
---

Fix incorrect results from index-optimized `where` clauses that combine indexed and non-indexed conditions.

- `OR` expressions are now only served from indexes when every disjunct can use an index; otherwise the query falls back to a full scan. Previously, rows matched only by a non-indexed disjunct were missing from the result.
- `AND` expressions still use indexes for the conditions that have them, but the remaining conditions are now enforced by re-checking each candidate row against the full expression. Previously, non-indexed conditions were silently dropped, returning rows that did not match the query.
- Compound range conditions (e.g. `age > 5 AND age < 10`) combined with conditions on other fields no longer ignore those other conditions.
- Compound range conditions sharing the same boundary value (e.g. `age >= 5 AND age > 5`) now apply the strictest bound regardless of the order the conditions appear in, using the same value comparison semantics as the indexes (dates, locale strings, ...).
- Compound range conditions that only bound one side (e.g. `age > 5 AND age >= 8`) no longer return an empty result.
- Strict range comparisons (`gt`/`lt`) on BTree-indexed fields holding normalized values such as dates now correctly exclude the boundary value.
9 changes: 7 additions & 2 deletions packages/db/src/collection/change-events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,16 @@ export function currentStateAsChanges<
)

if (optimizationResult.canOptimize) {
// Use index optimization
// Use index optimization. When the index lookup is inexact, the keys
// are a superset of the true result (some conditions could not be
// served by an index), so re-check each row against the full expression.
const filterFn = optimizationResult.isExact
? undefined
: createFilterFunctionFromExpression(expression)
const result: Array<ChangeMessage<WithVirtualProps<T, TKey>, TKey>> = []
for (const key of optimizationResult.matchingKeys) {
const value = collection.get(key)
if (value !== undefined) {
if (value !== undefined && (filterFn?.(value) ?? true)) {
result.push({
type: `insert`,
key,
Expand Down
5 changes: 4 additions & 1 deletion packages/db/src/indexes/btree-index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,10 @@ export class BTreeIndex<
toKey,
toInclusive,
(indexedValue, _) => {
if (!fromInclusive && this.compareFn(indexedValue, from) === 0) {
// Compare against the normalized key: indexed values are stored
// normalized (e.g. dates as timestamps), the raw `from` would
// never compare equal to them
if (!fromInclusive && this.compareFn(indexedValue, fromKey) === 0) {
// the B+ tree `forRange` method does not support exclusive lower bounds
// so we need to exclude it manually
return
Expand Down
Loading
Loading