linq late-stage cleanups: reducer shapes, adapter hygiene, match.das AST patterns (+ preflight pdflatex gate)

[borisbat]

Late-stage cleanup arc for linq (plan of record: benchmarks/sql/LINQ_TO_TABLE.md "Late stage"): user-facing reducer-shape fixes, adapter hygiene, and match.das adoption for AST matchers — plus the preflight side-quest (first mac run + a new docs/pdflatex gate).

Reducer shapes (items 1+2)

• Selector overloads sum / min / max / average(src, selector) in daslib/linq.das (iterator + array forms) — the C# 2-arg spellings. first / count deliberately excluded: their C# 2-arg forms take a predicate, not a selector.
• Untyped lambdas on the group-bucket surface — _._1 |> select(@(x) => x * 2) |> sum failed with 30303 on BOTH tiers because the fully generic tier-2 params can't flow a type into an untyped lambda, and the fold macro reads ._type so tier-2 must accept the spelling first. Fix: BucketLambdaStamper in linq_boost stamps the bucket element type onto untyped single-param lambdas of direct _._1 |> op(lam) calls before inference (same mechanism as the existing outer-param injection in visit()).
• 2-arg recognizer arm in is_bucket_reducer_call — the direct-selector spelling fuses like the inner-select form; an identity lambda (max(@(v) => v)) canonicalizes to the bare reducer.
• 18 new tests (tests/linq/test_linq_reducer_shapes.das), green on interp / JIT / AOT; fused-vs-unfused parity asserted.

Adapter hygiene (item 4, partial by design)

• 4A: the upstream-join validation block was byte-identical in the Array / Json / Xml build_group_by_adapter arms — extracted extract_upstream_join_core + extract_upstream_join_array_srcb into linq_fold_common; decs keeps extract_decs_bridge and shares only the core.
• 4B: the transitional getters arrayTop() / arraySrcName() renamed to loop_source_expr() / loop_source_name() with an honest contract: they're the generic-lane source feed (the shared array-shaped lanes emit for/length() against the name and read the expr for compile-time facts; null = decline), not an array-only leftover. linq_fold.md updated to match.
• Deferred with reasons (recorded in LINQ_TO_TABLE.md): stringly-Captures → typed ChainView; per-source dispatch can't become a registry (macro modules compile into separate contexts).

match.das adoption (item 3) + ExprRef2Value transparency

• daslib/match now peels ExprRef2Value for AST class patterns and $v captures (match_peel_r2v), mirroring qmatch's transparency rule — the typer's wrappers have no surface syntax, so a clean ExprField(...) pattern should match a wrapped source. An explicit ExprRef2Value(...) pattern still matches the wrapper itself. 7 new tests (tests/match/test_match_r2v_peel.das).
• match_expr now accepts das_string fields compared against string locals (language-level compare; previously a false "mismatching expression type" error).
• Converted the hand-rolled is/as/peel ladders that fit: the key-probe matchers in linq_fold_table and the bare-key join matcher in linq_fold_common (now one shared is_bare_key_ref); extract_key_probe's == leaf goes through qmatch. Declined with reasons: is_bucket_reducer_call (match is statement-shaped; the ladder isn't the function) and extract_decs_bridge (match.das array patterns reject das-vector scrutinees).
• Toolbox docs: skills/das_macros.md gains a "match (daslib/match) — pairs with qmatch" section (division of labor, limits, canonical conversions) and the stale "qmatch auto-peel is TODO" claim is fixed; CLAUDE.md idiomatic-forms row added.

preflight: first mac run + docs/pdflatex gate

• Tool discovery off PATH: sphinx-build via ~/Library/Python/*/bin (mac pip --user) / ~/.local/bin, latexmk via /Library/TeX/texbin.
• New gate 7 (docs/pdflatex): compiles both PDFs via latexmk -cd, deliberately stricter than CI — CI's PDF steps are continue_on_error, so an undeclared unicode char ships broken release PDFs without going red (the U+2261 incident). Locally it's a FAIL with a one-line remedy.
• doc/source/conf.py declares 7 more unicode chars (≈ ∈ ∉ √ ∞ ∧ ∨) per the "support them as they come" policy. No Windows behavior change.

Bench re-sweep

benchmarks/sql/results.md re-swept (m1–m7, INTERP + JIT) per the living-doc rule. No perf-path changes in this PR — deltas are run-to-run noise (JIT on Apple silicon is high-variance per the results.md preamble).

Also

• skills/writing_tests.md documents the tests/.das_test directory filter and its root-path caveat (the filter only loads at the --test ROOT, so subtree sweeps walk gated dirs unfiltered), plus the current options no_aot AOT-hash limitation.
• detect-dupe pass over the diff folded two findings (one bare-key matcher, shared preflight skip-gate helper).

Validation

• preflight --full: 16 gates pass (format, lint, dasgen, ci-das, all seven docs gates incl. pdflatex, tests-cpp, interp, JIT, AOT, sequence smoke; cpp-syntax skipped — no C++ changed).
• Full suites: interp / JIT / AOT green; new tests covered on all three tiers.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR advances the “late stage” LINQ cleanup work by (1) expanding reducer call shapes to better match C# (selector overloads + canonicalization), (2) improving fold adapter hygiene via shared helpers and clearer adapter contracts, and (3) adopting daslib/match patterns (including ExprRef2Value transparency) for AST matchers. It also strengthens local tooling by adding a stricter docs/pdflatex preflight gate and updating docs/benchmarks accordingly.

Changes:

• Add selector overloads for sum/min/max/average(src, selector) and update fold recognition to fuse direct-selector and inner-select forms (incl. identity-lambda canonicalization).
• Make daslib/match peel ExprRef2Value transparently for AST class patterns and $v captures; convert select LINQ fold match ladders to match/qmatch.
• Enhance preflight docs tooling discovery and add a local docs/pdflatex gate; update docs and benchmark living docs.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
utils/preflight/README.md	Updates preflight documentation to reflect 7 docs gates.
utils/preflight/main.das	Adds tool discovery for sphinx-build/latexmk, introduces docs/pdflatex gate, and refactors skip helpers.
tests/match/test_match_r2v_peel.das	Adds tests ensuring daslib/match peels ExprRef2Value for AST patterns and captures.
tests/linq/test_linq_reducer_shapes.das	Adds tests for selector reducers, bucket-surface untyped lambdas, and reducer-shape parity.
skills/writing_tests.md	Documents .das_test directory filter behavior and AOT/JIT subtree caveat.
skills/preflight.md	Documents the new 7th docs gate (pdflatex) and tool discovery behavior.
skills/das_macros.md	Updates matcher guidance, adds match vs qmatch guidance, and documents ExprRef2Value transparency.
modules/dasPUGIXML/daslib/linq_fold_xml.das	Deduplicates upstream-join validation via shared helpers; renames adapter getter override to loop_source_name.
doc/source/reference/linq_fold_patterns.rst	Documents accepted reducer spellings and bucket-surface lambda stamping behavior.
doc/source/conf.py	Extends LaTeX unicode declarations and clarifies policy comments.
doc/reflections/das2rst.das	Hides match_peel_r2v in “Implementation details” for generated docs grouping.
daslib/match.das	Adds match_peel_r2v, peels ExprRef2Value for AST class patterns and $v captures, and relaxes match_expr string-like compares.
daslib/linq.das	Adds selector overloads for max/min/sum/average for iterator and array sources.
daslib/linq_fold.md	Updates adapter contract docs from arrayTop/arraySrcName to loop_source_expr/loop_source_name.
daslib/linq_fold_table.das	Adopts daslib/match, updates adapter getters, and simplifies key-probe recognition using shared bare-key matcher + qmatch.
daslib/linq_fold_json.das	Deduplicates upstream-join validation via shared helpers; renames adapter getter override to loop_source_name; removes unused strings require.
daslib/linq_fold_decs.das	Deduplicates upstream-join validation via shared helpers for the decs join-by-group-by adapter path.
daslib/linq_fold_common.das	Renames generic-lane source feed API, adds reducer-shape recognition updates (2-arg reducers + identity canonicalization), adds shared upstream-join extraction helpers, and converts bare-key matcher to match.
daslib/linq_fold_array.das	Updates adapter getters, deduplicates upstream-join validation via shared helpers; removes unused strings require.
daslib/linq_boost.das	Adds bucket-surface lambda stamping pass to type untyped lambdas over group buckets pre-inference.
CLAUDE.md	Updates guidance on replacing is/as/peel ladders with qmatch/match.
benchmarks/sql/results.md	Refreshes benchmark results sweep (living doc update).
benchmarks/sql/LINQ_TO_TABLE.md	Updates “Late stage” plan status and records what’s completed/deferred.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 1 comment.