perf(prover): parallelize + vectorize hot path (4.4× Prove speedup)

[gbotrel]

Summary

Stacks four optimizations from optim_plan_ideas.md on the proving hot path. On the bench workload (40 PLONK instances of size 2^17, Poseidon2, 32-core box):

metric	baseline	first commit	this PR	total delta
Prove wall	41.87s	24.45s	9.60s	−4.36×
Prove par	1.09×	2.13×	5.40×	+4.95×
Prove allocated	12.89 GiB	12.27 GiB	12.39 GiB	−4%
peak heap	4.86 GiB	7.66 GiB	7.69 GiB	+58%

Wall time drops from ~42s → ~10s; par rises from ~1× → ~5.4× as the per-module proving work overlaps and the surviving serial loops cost less per element.

Changes (commit by commit)

1. perf(prover): parallelize ComputeAIRQuotients (item #1 from the plan)

• Per-module ComputeQuotient* dispatched via parallel.Execute on a sorted module list.
• dag.(*DAG).Clone() deep-copies nodes + *expr.Leaf so each goroutine mutates Leaf.Idx independently (required because cross-module Lookup / CopyConstraint shares leaves).
• poly.DomainCache.Get is now sync.Mutex-protected for concurrent access; pr.airTrace.SetBase/SetExt + chunkDomains writes batched under a tiny lock window.
• computeExt/BaseAIRChunks now stop at canonical-Normal form (skip FFT + BitReverse + FFTInverse + BitReverse identity); chunks are 3-index sub-slices of the quotient backing array instead of make + copy.

2. perf(dag,poly): use ext/koalabear Vector ops in hot inner loops (item #4)

• dag.{add,sub,mul}ChildVectorIntoExt and poly.{Ext,}EvaluateLagrangeWithWeights were scalar Go loops over N≈2^17. Switched to gnark-crypto ext.Vector / koalabear.Vector AVX-512 entry points (Add, Sub, Mul, MulByElement, InnerProduct[ByElement]).
• mulChildVectorIntoExt alone was ~15% of Prove CPU (cumulative, including E4.Mul).
• Base-rail add/sub paths touch dst[j].B0.A0 directly since the lifted base coefficient is zero in the other slots — drops the lift indirection.

3. perf(rs): drop two BitReverse passes per Encode via DIT FFT (item #3 option B)

• Encode path was FFTInverse(DIF) → BitReverse → FFT(DIF) → BitReverse. The two BitReverse calls on sizes up to 2^20 fell through gnark-crypto's threshold to bitReverseNaive (~5.9% of Prove CPU).
• New path scatters the n bit-reversed inverse-FFT coefficients directly into N bit-reversed slots (zero-padding the gaps) and runs the forward FFT in DIT order. DIT consumes bit-reversed input and produces normal output — no explicit BitReverse needed.
• New helper scatterBitReversedCoeffs[T any] is generic over both poly.Polynomial (Base) and poly.ExtPolynomial.

4. perf(prover): parallelize zeta evaluations, vectorize DeepQuotient accumulation

• ComputeAIRQuotients per-chunk evaluate-at-zeta + ComputeEvaluationsAtZeta per-module are now parallelized; results merged serially under Proof.SetValueAtZetaExt.
• ComputeDeepQuotient: the inner C_s[x] += alpha · col[x] accumulation now uses ext.Vector.MulAccByElement (base columns) or ScalarMul into a per-size scratch buffer + Add (ext columns). Constant columns (len == 1) keep the scalar broadcast form.

Bench tooling (bench/main.go)

• Fixed par metric: was /cpu/classes/total (= GOMAXPROCS × wall, counts idle slots). Now uses /cpu/classes/user + /cpu/classes/gc (real on-CPU time). A serial section now correctly reports par ≈ 1.0×.
• Default workload bumped to -instances 40 -log2-size 17 for usable pprof samples.

Docs

• optim_plan_ideas.md catalogues remaining perf opportunities with profile evidence and file:line pointers. This PR implements items feat: Easier to use Fiat-Shamir transcript #1, chore: pin GitHub Actions to commit SHAs #3 (option B), Feat/recursion #4 and a partial perf(prover): parallelize + vectorize hot path (4.4× Prove speedup) #6 (parallel zeta evals; full FRI-query parallelism still pending).

Test plan

• go test -count=1 ./internal/... ./prover/... ./integration_test/... — all green, including end-to-end verifier checks in integration_test/gnark_plonk and integration_test/go_corset/zkc.
• go run ./bench → Prove wall 9.60s, par 5.40× (vs 41.87s / 1.09× baseline).
• Reviewer: confirm no consumer outside prover/ writes to DomainCache concurrently in a way that conflicts with the new mutex.
• Reviewer: spot-check scatterBitReversedCoeffs against the original BitReverse → zero-pad → BitReverse semantics (a unit test in internal/reedsolomon would be welcome).
• Reviewer: confirm the len(col) == 1 broadcast path in addScaled{Ext,Base}Column matches the original branch behavior.

Remaining (in optim_plan_ideas.md)

After re-profiling on this branch, the biggest remaining targets are #2 (batch-invert in DeepQuotientExt), #5 (reuse EvalWorkspace across modules per worker), and #9 (pool the per-iteration N-sized E4 scratch buffers to bring peak heap back below 5 GiB).

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes core proving math paths (parallel DAG/quotient, FFT layout in RS encode, shared DomainCache locking); correctness relies on clone/scatter semantics matching prior behavior—tests pass but peak heap rises and RS scatter lacks dedicated unit tests called out in the PR.

Overview
This PR speeds up the Prove path (reported ~4× wall time on the bundled bench workload) by parallelizing per-module work, cutting redundant FFT/allocation work, and vectorizing large field loops.

Prover: ComputeAIRQuotients runs per module in parallel (parallel.Execute), with dag.DAG.Clone() per worker so ComputeQuotient* can mutate Leaf.Idx safely when modules share leaves. Quotient chunking stops at coset-Lagrange → canonical (Coset*NormalToCanonicalWithCache) and sub-slices the backing array instead of extra FFT round-trips and make+copy per chunk. ComputeEvaluationsAtZeta (and AIR chunk evals at ζ) are parallelized per module; results are merged into the proof serially. ComputeDeepQuotient uses gnark-crypto ext.Vector helpers (MulAccByElement, ScalarMul/Add) for column accumulation.

Supporting libs: DomainCache is mutex-protected for concurrent Get. DAG mixed ext add/sub/mul paths and Lagrange inner products use vector APIs. Reed–Solomon encode avoids two BitReverse passes via scatterBitReversedCoeffs and a DIT forward FFT.

Tooling: New bench/main.go drives aggregated PLONK instances, writes pprof profiles, and reports par from real on-CPU time (user + GC), not idle-inflated totals. .gitignore ignores profile output dirs.

^{Reviewed by Cursor Bugbot for commit 58e62e2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

This PR improves prover performance by parallelizing per-module AIR quotient computation and reducing memory churn in the quotient chunking pipeline, while also updating bench tooling and adding a written optimisation plan for future work.

Changes:

• Parallelizes proverRuntime.ComputeAIRQuotients() across modules, adding per-goroutine DAG cloning and locking around shared trace writes.
• Adds “coset-Lagrange Normal → canonical Normal” conversion helpers and switches quotient chunking to bounded subslices (avoiding per-chunk make+copy).
• Introduces a new end-to-end benchmark (bench/main.go) and adds an optimisation planning document.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
prover/prover.go	Parallel per-module AIR quotient computation; new chunking helpers; synchronized writes to shared trace/maps.
internal/poly/domain_cache.go	Makes DomainCache.Get mutex-protected for concurrent use.
internal/poly/compute_quotient.go	Splits coset conversion into canonical-only helpers to avoid redundant FFT round-trips.
internal/dag/dag.go	Adds (*DAG).Clone() deep copy to avoid cross-goroutine mutation races on shared leaves/nodes.
bench/main.go	Adds an end-to-end prover benchmark with improved CPU parallelism metric and profiling output.
optim_plan_ideas.md	Adds a draft optimisation plan with profiling evidence and implementation ideas.
.gitignore	Ignores generated .pprof files and bench_profiles/ output directory.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.