Skip to content

perf: concatenate per-chunk Schur edge lists in one pre-sized copy - #255

Merged
schroedk merged 1 commit into
mainfrom
perf/par-emit-collect
Aug 13, 2026
Merged

perf: concatenate per-chunk Schur edge lists in one pre-sized copy#255
schroedk merged 1 commit into
mainfrom
perf/par-emit-collect

Conversation

@schroedk

@schroedk schroedk commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

par_emit collected sampled star edges with reduce(Vec::new, |a, b| a.append(b)), so rayon's reduction tree recopied every edge once per level. Collecting the chunks and calling concat() copies each edge exactly once into one pre-sized allocation.

Setup time on fixest_comparison, interleaved rounds with a rebuild between arms:

500K 2M 5M 20M
LSMR(exact) (null control) +1.5% +2.1% +1.4% +0.3%
LSMR(approx) -3.2% -4.1% -4.1% -5.0%
LSMR(2-2) -9.0% -5.6% -6.4% -6.2%

Run ranges separate at 5M and 20M for approx, and at every size for 2-2.

Scope: null on slopes, akm_panel, akm_scaling and six small-problem suites — Schur assembly is ~30% of within CPU on fixest-difficult 3FE but only 5-7% there. No regression in any suite measured.

Results are unchanged: sort_and_dedup imposes a total (lo, hi, weight) order immediately after, and iteration counts are bit-identical across every case/config pair in all runs.

Note: 5% sits inside this repo's normal noise band, so reproducing it needs the interleaved protocol — a casual before/after will not show it.

@schroedk schroedk changed the title perf: splice per-worker Schur edge lists instead of reducing by append perf: concatenate per-chunk Schur edge lists in one pre-sized copy Aug 13, 2026
@schroedk
schroedk force-pushed the perf/par-emit-collect branch from 3a0a57c to e82fb90 Compare August 13, 2026 11:40
@schroedk
schroedk merged commit 64c9d3a into main Aug 13, 2026
5 checks passed
@schroedk
schroedk deleted the perf/par-emit-collect branch August 13, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant