Add generalized semiring reductions to einsum VM and JIT - #132
Conversation
Generalize the einsum contraction to an arbitrary commutative semiring:
a Reduce operator {Sum, Prod, Max, Min} folding contributions into the
output, over a Combine operator {Mul, Add, Min, Max} merging the input
operands — so max-product (`A[i,j,k] = max_q B[j]*C[i,k,q]*D[k,q]`) and
min-plus (tropical / shortest-path) contractions run through the same
machinery as plain sum-product einsum. The design is modelled after the
Julia library Tullio.jl (noted as a footnote in the module docs).
- New `Scalar` element trait (tensor.rs) carrying the per-operator
identity table: Sum→0, Prod→1, Max→-inf/T::MIN, Min→+inf/T::MAX.
Empty reductions yield the identity.
- VM: `compile_reduce` / `einsum_reduce` / `einsum_reduce_dyn`. The
einsum_reduce wrappers pre-fill outputs with the reduce identity
(overwrite semantics); compile_reduce + Program::exec keeps
accumulate semantics.
- Sparse row iteration (structural-zero skipping) is only sound when a
missing operand annihilates its contribution to the reduce identity —
precisely (Sum, Mul). Other op pairs iterate sparse inputs densely in
the VM; the JIT rejects them with Unsupported.
- JIT: `EinsumF32Jit::compile_reduce` / `einsum_jit_reduce`, swapping
fmax/fmin/fmul/fadd into the existing register-accumulator and RMW
paths. (Sum, Mul) emits IR identical to before.
- Tests: semiring unit tests (incl. integer MIN-identity, empty ranges,
AccFlush RMW edge, sparse-with-max dense fallback) plus a 481k-case
differential sweep (tests/reduce_sweep.rs) comparing VM and JIT
against a naive reference over all specs on {a,b,c}, 7 op pairs, all
sparse masks — bit-exact. Existing 19.5M-case einsum_sweep passes
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A small decoder-only transformer whose every contraction runs through the einsum VM, cross-checked against a NumPy reference and a PyTorch trainer that all share one set of einsum specs. - train.py: PyTorch nanoGPT-style trainer; defines the arch with torch.einsum (same specs as the decode step), trains char-level on an embedded corpus, exports weights.bin/config.txt/vocab.bin/prompt.txt. - gpt2_reference.py: NumPy reference forward (np.einsum, identical specs); greedy-decodes and dumps ref_logits.bin. - main.rs: loads the trained weights at runtime (dims from config), runs the KV-cached decode through einsum_homogenous, and reports max-abs logit diff and argmax agreement vs the reference. Falls back to random-weight smoke test when weights/ is absent. - README.md documents the architecture, specs, and run steps. Rust vs NumPy: 24/24 argmax agreement, max abs logit diff 1.7e-5 (float summation order). No library changes needed — the whole transformer expresses in the existing einsum plus trivial elementwise helpers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a unary `map(x, f)` helper and rewrite relu, scale, and rmsnorm's scaling step to use it instead of reading Dense.data directly. add (binary zip) and softmax_last (per-row) aren't single-function maps and are left as-is. No behavior change — the example still matches the NumPy reference (24/24 argmax, 1.7e-5 max logit diff). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`cargo run --release --example gpt2 -- bench` times full cold-cache greedy decodes (best of several batches) and reports per-token latency and tokens/s. Extracts the decode loop into a reusable run_decode(); the normal compare path is unchanged and still matches the NumPy reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The intro claimed the whole forward pass "expresses entirely in einsum". RMSNorm, ReLU, softmax, residual adds, and the attention scale are plain loops, not einsum — reword to scope the claim to the linear-algebra contractions, matching the main.rs doc comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
the VM regression looks like per-element dispatch: tried lifting jit_bench.rs is byte-identical across both builds so the whole delta is the einsum change, and the JIT path is in that total too so the VM-only drop is a bit bigger. correctness holds with it applied: the linalg suite, your 481,278-case reduce sweep, and the 19,465,300-case exhaustive sweep all pass. two caveats: i only measured (Sum, Mul), jit_bench doesn't run min-plus or max-product so i haven't checked those. and the extra instantiations cost size, ~+4.8% here and more on the rlib. patch's here if it's useful: https://gist.github.com/MesTTo/0d470c97bfecfe503f79e026a537abe5 |
|
accidental comment |
Generalize the einsum contraction to an arbitrary commutative semiring: a Reduce operator {Sum, Prod, Max, Min} folding contributions into the output, over a Combine operator {Mul, Add, Min, Max} merging the input operands — so max-product (
A[i,j,k] = max_q B[j]*C[i,k,q]*D[k,q]) and min-plus (tropical / shortest-path) contractions run through the same machinery as plain sum-product einsum. The design is modelled after the Julia library Tullio.jl (noted as a footnote in the module docs).Scalarelement trait (tensor.rs) carrying the per-operator identity table: Sum→0, Prod→1, Max→-inf/T::MIN, Min→+inf/T::MAX. Empty reductions yield the identity.compile_reduce/einsum_reduce/einsum_reduce_dyn. The einsum_reduce wrappers pre-fill outputs with the reduce identity (overwrite semantics); compile_reduce + Program::exec keeps accumulate semantics.EinsumF32Jit::compile_reduce/einsum_jit_reduce, swapping fmax/fmin/fmul/fadd into the existing register-accumulator and RMW paths. (Sum, Mul) emits IR identical to before.VM (interpreted) einsum regressed ~5–15%:
einsum_homogenous(concrete)einsum_homogenous(concrete)einsum_homogenous(concrete)einsum_homogenous(concrete)einsum(&dyn)einsum(&dyn)einsum(&dyn)einsum(&dyn)einsum_homogenous(Tensor)einsum_homogenous(Tensor)einsum_homogenous(Csr)einsum_homogenous(Csr)einsum_homogenous(Csr)einsum(&dyn)einsum(&dyn)einsum(&dyn)