Skip to content

linq: table point-lookup conjunct extraction — where(key==X && residual) probes#3106

Merged
borisbat merged 3 commits into
masterfrom
bbatkin/linq-table-point-lookup-conjuncts
Jun 12, 2026
Merged

linq: table point-lookup conjunct extraction — where(key==X && residual) probes#3106
borisbat merged 3 commits into
masterfrom
bbatkin/linq-table-point-lookup-conjuncts

Conversation

@borisbat

Copy link
Copy Markdown
Collaborator

What

Stage B of the table arc: the point-lookup matcher now extracts a key-equality conjunct from a compound where predicate instead of requiring the whole predicate to be a bare key == X. where(kv.key == X && residual) — including the chained spelling where(key == X) |> where(residual), which collapse_chained_wheres merges into the same shape at flatten time — folds the whole walk to an O(1) probe that evaluates the residual on the probed element only, routing a false residual to the same miss path the key-miss takes.

Semantics rule

The key-equality must be the leftmost conjunct. That positioning is what makes the fold exact, not a simplification:

  • Keys are unique, so under && short-circuit every conjunct right of the key-equality runs at most once on both paths (scan and probe) — no invariance/purity gate is needed on the residual. A bump-counter regression test pins this (residual runs exactly once, fused or not).
  • A conjunct left of the key-equality runs per element in the scan vs once in a probe, so that order declines and keeps the scan's exact panic/side-effect behavior. The existing gates on X (loop-invariant, side-effect-free, either operand order) are unchanged.

All probe terminators ride it: first / first_or_default (± trailing select) take the element probe with the residual check inserted before the hit return; predicate-form any(p) / count(p) with a residual bind the probed element and return hit && residual / ? 1 : 0. Residual-free shapes keep their existing bare key_exists emissions.

The second commit removes a where-run coalescing loop the first draft carried: try_table_point_lookup is only reached after collapse_chained_wheres, so the matcher can never see two where_ entries — the conjunct peel alone covers both spellings.

Numbers (m7, 2026-06-11 sweep)

point_lookup_residual — the exact shape that was a full scan before this PR — probes at 466 ns/op INTERP vs the scan control's 9967 (~21×) and 1270 vs 9049 JIT (~7×); it normalizes to 0.0 ns/elem in the results.md matrix, same cell as the bare probe. The bench trio replaces the old pair: point_lookup (bare probe) / point_lookup_residual (newly fused) / point_lookup_scan (flipped to residual-FIRST conjunct order so it remains an honest decline control).

Tests

8 new/extended sub-tests in test_table_point_lookup: hit + residual-true/false, miss, any/count with residual, three-conjunct chains with trailing select, the key == 2 && key == 3 oddity (probes 2, residual re-check fails — agrees with the scan), keys-lane residual, the two-where spelling, residual-eval-count pinning, first panic on a failed residual, and residual-first staying a scan.

Gates

🤖 Generated with Claude Code

borisbat and others added 3 commits June 11, 2026 16:58
…al) probes

extract_key_probe peels the left-assoc && spine: the leftmost conjunct must be
the key-equality (keys are unique, so under short-circuit everything right of
it runs at most once on both paths — no purity gate needed on the residual);
try_table_point_lookup coalesces a leading run of consecutive where_ calls
before extraction, and the emission evaluates the residual on the probed
element only, routing false to the same miss path. any/count with a residual
ride the element probe; residual-first conjunct order still declines to the
scan. Bench trio: point_lookup / point_lookup_residual (newly fused) /
point_lookup_scan (flipped to residual-first as the decline control).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rges at flatten time

try_table_point_lookup is only reached after flatten_linq's
collapse_chained_wheres pass, so a leading run of consecutive where_ calls
is always ONE call here; the nw-loop generality could never trigger. Restore
the original chain-shape match — the conjunct peel in extract_key_probe alone
covers both the explicit && spelling and the chained-where spelling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
point_lookup_residual (the pre-stage-B scan shape) probes at 466 ns/op INTERP
vs the residual-first scan control's 9967 (~21x); 1270 vs 9049 JIT (~7x) —
0.0 ns/elem in the matrix, same cell as the bare probe. LINQ_TO_TABLE.md
status -> stage B, findings recorded, conjunct-extraction deferred edge
removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 12, 2026 00:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands LINQ TableAdapter point-lookup folding so a where(key == X && residual...) predicate can still fold to an O(1) table probe, evaluating residual conjuncts only on the probed element (with a leftmost-key-equality rule to preserve scan semantics/side effects).

Changes:

  • Enhances try_table_point_lookup to extract a leftmost key == X conjunct from a compound && predicate and treat the remaining conjuncts as a post-probe residual filter.
  • Extends tests and benchmarks to cover residual-true/false behavior, miss-path equivalence, predicate-form any/count, chained where merging, and residual evaluation count.
  • Updates user-facing fold-pattern documentation and benchmark result matrices to reflect the new fused shape.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/linq/test_linq_table_source.das Adds/extends regression tests for residual-conjunct probing semantics and edge cases.
doc/source/reference/linq_fold_patterns.rst Documents residual-conjunct point-lookup folding and the leftmost-conjunct rule.
daslib/linq_fold_table.das Implements conjunct extraction + residual handling in table point-lookup folding.
benchmarks/sql/table.das Adds a residual-probe benchmark and adjusts the scan control to force decline.
benchmarks/sql/results.md Updates benchmark matrix text and incorporates the new benchmark row/results.
benchmarks/sql/LINQ_TO_TABLE.md Updates stage/status notes and records the stage-B findings and semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@borisbat borisbat merged commit 25f496a into master Jun 12, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants