linq: tables (each_kv / keys / values) as the 6th _fold source + the to_table sink

[borisbat]

Tables become a first-class _fold source — each_kv / keys / values chains fuse through a new TableAdapter — plus the selector-free to_table() sink for every direct-loop source. Plan of record with settled decisions, per-stage findings, and named deferred edges: benchmarks/sql/LINQ_TO_TABLE.md.

Stage commits (each independently reviewable)

1. each_kv builtin (8751bb9) — (key, value) named-tuple iteration over tables (strict can_copy gate, reject overloads for void/dim values). Also fixes a pre-existing generator-lowering bug: yield-for emitted short-circuiting loop &&= _builtin_iterator_first(...), so an empty first source closed a never-opened iterator ("magic mismatch on unlock").
2. TableAdapter (571fe87) — kv usage-pruned slot walks: a chain touching only .value walks values(tab) alone, key-only shapes walk keys(tab) alone; O(1) bare count(); redundant distinct over keys/kv dropped. New m7 bench lane.
3. %linq! table sources (29d23ba) — from kv in tab dispatches each_kv (map) / keys (set) by argument type, no annotation needed.
4. Point-lookup folds (ac441c4) — where(kv.key == X) + any/count/first[_or_default]/contains folds the whole walk to an O(1) key_exists / tab?[X] probe when X is loop-invariant and side-effect-free; everything else keeps the scan (per-element evaluation of impure X is observable, covered by a regression test).
5. Table joins (2742f6d) — emit_array_join generalized to an adapter-driven lead loop (any direct-return source rides it), so table leads join through the pruned slot walk. A table srcB joined on its bare key (d.key / bare set element) skips the join's internal hash entirely and probes the user's table per lead row — unique table keys make the probe ≡ hash semantics exactly.
6. to_table sink (b72f625) — fused insert-loop terminator (a k => v projection splits so each side evaluates once; reserve from O(1) length on unfiltered walks) + selector-free tier-2 forms over iterators and arrays. Duplicate keys keep the last occurrence (das insert semantics, not C#'s throw).

Mid-arc, master's fixed-array rework was merged in (1ab3e6a) and each_kv's dim-array reject overloads re-validated against the new auto(valT)[] matching rules. A final bench pass (9331bbc) fills the m7 column — 26 empty cells down to 7 genuinely-inapplicable ones — and lights to_table lanes across array/decs/XML/JSON.

Numbers (INTERP ns/elem, n = 100k; full matrix in benchmarks/sql/results.md)

shape	before (tier-2)	fused
sum over values	—	13.5 (array 2.1, SQL 30.2, XML 53.9)
point_lookup vs same query as scan	8.4	0.0
join_count (table lead)	195.0	66.0
join_probe vs forced hashed build	79.9	46.6
to_table vs materialize-then-convert	69.0	32.2 (array: 54.8 → 18.7)

Gates

Full INTERP 10984 tests / 0 failed; full AOT 10304 / 0 failed; JIT tests/linq 1962/1962 on a clean cache + verifier smoke; CI lint 0 issues across all 20 changed .das files; detect-dupe clean (bench-lane parallelism only); das2rst no stubs / no Uncategorized; Sphinx -W clean; results.md re-swept on an idle machine.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR makes table<K;V> / table<K> first-class _fold sources (via a new TableAdapter) and adds a selector-free to_table() sink that can fuse as a terminator, enabling better fusion and O(1) shortcuts/probes for common table-shaped LINQ patterns.

Changes:

• Add each_kv builtin iteration for tables (with compile-time rejections for unsupported value types) and fix generator zip lowering to always call first() for every source.
• Introduce TableAdapter for _fold to fuse each_kv/keys/values chains, including usage-pruned walks, point-lookup folding, and join optimizations (table-lead joins + table-srcB probe mode).
• Add selector-free to_table() tier-2 overloads and a fused _fold sink; expand tests, docs, and benchmarks to cover the new behavior and add an m7 “table” benchmark lane.

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/linq/test_linq_table_source.das	New _fold-level correctness tests for table sources, joins, point-lookup folds, and to_table() fusion.
tests/linq/test_linq_das.das	Adds %linq! coverage for untyped table/table-set from dispatch and join scenarios.
tests/linq/failed_linq_das_table.das	Adds compile-fail coverage for table-source inference rejections in %linq!.
tests/language/table_each_kv.das	Adds language-level tests for the new each_kv iterator behavior.
tests/language/generator_zip_empty.das	Regression test for generator multi-source loop lowering with empty sources.
tests/language/failed_each_kv.das	Compile-fail coverage for each_kv rejection cases (non-copyable/void/dim-array values).
src/ast/ast_generate.cpp	Fixes generator lowering to ensure _builtin_iterator_first runs for each zipped source.
skills/linq.md	Documents table sources and selector-free to_table() usage at a high level.
doc/source/stdlib/handmade/function-builtin-each_kv-0xdb81e5ca7a0e3baa.rst	Adds builtin docs entry for each_kv.
doc/source/reference/linq_fold_patterns.rst	Documents new table-source adapter behavior, point-lookup folds, join probe mode, and to_table() sink pattern.
doc/source/reference/linq_das.rst	Updates %linq! docs to reflect from_in dispatch for untyped sources, including tables.
doc/reflections/das2rst.das	Adds each_kv to the containers grouping for docs generation.
daslib/linq.das	Adds selector-free to_table overloads for iterators/arrays (map + set forms).
daslib/linq_fold.md	Updates architecture notes to include the new linq_fold_table module.
daslib/linq_fold.das	Wires in linq_fold_table and adds dispatcher logic for table source extraction/splicing.
daslib/linq_fold_table.das	Implements TableAdapter, table-source extraction, point-lookup folding, and redundant-distinct dropping.
daslib/linq_fold_decs.das	Declines to_table for decs to avoid incorrect fallback behavior.
daslib/linq_fold_common.das	Adds invoke_param_type capability, to_table as a terminator lane, and join refactors + probe-mode support.
daslib/linq_das.das	Updates %linq! transpilation to use from_in(...) for untyped sources and extends FromInMacro for table dispatch.
daslib/builtin.das	Adds each_kv builtin + rejection overloads and docs comments.
benchmarks/sql/xml.das	Adds XML lane benchmarks for fused to_table vs staged baseline.
benchmarks/sql/table.das	New m7 lane benchmarks for table source fusion, probes, joins, and to_table.
benchmarks/sql/results.md	Extends benchmark report to include the new table lane and to_table results.
benchmarks/sql/LINQ_TO_TABLE.md	Adds/updates the “arc plan” document detailing design decisions and deferred edges for the table arc.
benchmarks/sql/json.das	Adds JSON lane benchmarks for fused to_table vs staged baseline.
benchmarks/sql/decs.das	Adds decs lane benchmarks for to_table (tier-2 cascade) vs staged baseline.
benchmarks/sql/array.das	Adds array lane benchmarks for fused to_table vs staged baseline.
benchmarks/sql/_update_results.das	Extends the results generator to include the new m7 lane and header.
benchmarks/sql/_common.das	Adds a shared fixture_table helper for the new m7 benchmark lane.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.