linq: table group_by fusion — TableAdapter rides plan_group_by_core#3103
Merged
Conversation
each_kv/keys/values |> group_by chains declined at can_group_by_source (TableAdapter inherited the base false) and cascaded to tier-2: materialize a kv array, re-enter the array lane. The group_by splice pattern is fully adapter-generic, so enabling tables is two overrides mirroring DecsAdapter: - can_group_by() -> true - build_group_by_adapter() -> fresh TableAdapter (clone tabExpr, fresh srcName, same elemType/lane). The upstream_join arm returns null: join |> group_by over a table lead stays on tier-2 (named deferred edge in LINQ_TO_TABLE.md; the stage-5 terminator-path join is untouched). plan_group_by_core hands the bucket-fill body to wrap_source_loop, so the kv usage-pruner sees the whole accumulation (key expr + reducer updates + upstream where/select segments) — a group key over kv.value.brand walks values(tab) alone. The non-copyable-values gate and the -const elemType scrub are inherited from adapter construction. m7 groupby_* (ns/op): INTERP 144-201 -> 30-50 (count 163->31, ~5x); JIT 44-73 -> 8.4-11 (count 43.5->8.4, rides #3100's inline slot walk). join_groupby_* unchanged (cascade). INTERP matrix flat elsewhere. Tests: 8 new group_by sub-tests in test_linq_table_source.das (67/67 INTERP+JIT; AOT-compiled in test_aot) — kv/keys/values lanes, upstream segments, having + trailing where + order, count terminator, empty table, fused-vs-tier-2 agreement. Gates: full INTERP 11001/0, full AOT 10313/0, JIT linq 1971 + jit_tests 295, CI lint 0, Sphinx latex+html -W clean + pdflatex pass 1, formatter clean. results.md re-swept; linq_fold_patterns.rst table row + new group_by pattern row; LINQ_TO_TABLE.md stage 7 status + findings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Enables group_by fusion for table-backed LINQ sources (each_kv(tab), keys(tab), values(tab)) by letting TableAdapter participate in the existing adapter-generic plan_group_by_core path, avoiding the tier-2 “materialize kv array then regroup” cascade.
Changes:
- Enable table-source
group_byfusion viaTableAdaptercapability + adapter construction hook. - Add new table-source
group_bytests covering kv/keys/values lanes and composed segments (having/where/order/count/empty). - Update docs and benchmark notes/results to reflect the new fused behavior and performance deltas.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
daslib/linq_fold_table.das |
Turns on group_by support for table sources and wires build_group_by_adapter to reuse plan_group_by_core with a fresh TableAdapter (declines upstream-join shape to keep the documented deferred edge). |
tests/linq/test_linq_table_source.das |
Adds a new test_table_group_by suite validating fused behavior across lanes and common operator compositions. |
doc/source/reference/linq_fold_patterns.rst |
Documents the new table-lead group_by fusion pattern and its limits (e.g., join→group_by over table lead still cascades). |
benchmarks/sql/results.md |
Updates benchmark narrative and measured numbers to reflect the new fused table groupby_* behavior. |
benchmarks/sql/LINQ_TO_TABLE.md |
Advances table-fold “stage” status and documents the new group_by fusion plus remaining deferred edges. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… inference cleanups Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…catch) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
each_kv(tab)/keys(tab)/values(tab)chains withgroup_bydeclined at thecan_group_by_sourcegate (TableAdapter inherited the basefalse) and cascaded to tier-2: materialize a kv array, re-enter the array lane. That cascade was the entiregroupby_*gap in the m7 column.The group_by splice pattern is fully adapter-generic (
can_group_by_source→build_group_by_adapter→plan_group_by_core), so enabling tables is two overrides onTableAdapter, mirroringDecsAdapter:can_group_by()→ truebuild_group_by_adapter()→ freshTableAdapter. Theupstream_joinarm returns null:join |> group_byover a table lead stays on tier-2 (named deferred edge inLINQ_TO_TABLE.md; the stage-5 terminator-path join is untouched).plan_group_by_corehands the bucket-fill body towrap_source_loop, so the kv usage-pruner sees the whole accumulation (key expr + reducer updates + upstream where/select segments) — a group key overkv.value.brandwalksvalues(tab)alone. The non-copyable-values gate and the elemType const-scrub are inherited from adapter construction.Numbers
benchmarks/sql m7 (table)
groupby_*family, ns/op (results.md re-swept in this PR):~5× in both tiers — the JIT side compounds with #3100's inline slot walk.
join_groupby_*unchanged (cascade, by design); INTERP matrix flat elsewhere.Tests
8 new sub-tests in
tests/linq/test_linq_table_source.das(67/67 INTERP + JIT, AOT-compiled in test_aot): kv/keys/values lanes, upstream where+select segments, having + trailing where + order, count terminator, empty table, fused-vs-tier-2 agreement.Gates
-Wclean, pdflatex pass 1 clean (linq_fold_patterns.rsttable-source row updated + new table group_by pattern row;LINQ_TO_TABLE.mdstage 7 status + findings + deferred edge)🤖 Generated with Claude Code