Add a C port, and speedup Rust - #56
Draft
certik wants to merge 7 commits into
Draft
Conversation
Documents expression representation, ExprRef handles, interner mechanics (two-level IndexSet structure, shallow hashing/equality), automatic canonicalization rules, and traced walkthroughs of x+x and diff(sin(x), x). Linked from spec/design.md and README.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GcRnPSHE7EazAEDdrfUcj
C11 implementation of the Number/Symbol/Pow/Mul/Add subset with the same design and algorithms as the Rust core: arena-backed hash-consing interner (dense node array + open-addressing index table, FxHash, shallow hashing/equality), canonicalizing constructors with the same rule order and binary fast paths, structural ordering, and memoized expand with multinomial expansion using overflow-checked __int128 coefficients. Custom infrastructure, no hot-path malloc: uint32_t ExprRef handles, node/child/name arenas, open-addressing i128 accumulator maps and expand cache, reusable scratch buffers. Tests verify hash-consing identity, canonicalization rules, exact display strings, benchmark term counts (21/66/2^n/56/5984/6272/153), and that expansion preserves value under a modular evaluator; suite is ASan/UBSan-clean. bench/bench.c mirrors the Criterion expand scenarios (fresh context per iteration, scenario body only); C runs 1.2-2x faster than the Rust core on this machine. Implemented per c/PLAN.md by a Sonnet 5 subagent. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GcRnPSHE7EazAEDdrfUcj
Speeds up the expand benchmarks 10-32% with identical algorithms and
canonical forms (all 1479 Rust + 255 Python tests unchanged):
- ExprRef is u32 instead of usize: halves every child slice and handle.
- New default interner (FastInterner): dense Vec<Expr> + hashbrown
HashTable of (cached hash, index) slots, mirroring the C interner.
Borrowed-candidate methods (intern_{add,mul,ncmul}_{slice,vec},
intern_mul_tail, intern_add_head) probe without building a Box, so
interning an existing node performs no allocation; Vec-based variants
reuse the buffer when the node is new. IndexSetInterner retained as a
reference implementation.
- Context-owned pools for the Add/Mul accumulator maps and finalizer
scratch buffers (cleared, capacity retained) instead of fresh
FxHashMap/Vec per canonicalization.
- Binary add/mul dispatch on iterator size_hint: two-element calls run
the fast paths with zero allocation; larger inputs keep in-place Vec
collection.
- try_mul_fast/add_number_to_sorted read children in place (plan phase
with shared borrows, then one exact-capacity build) instead of
defensively copying; collect_mul_child takes &Context and walks
nested children without to_vec; collect_add_child re-fetches by index.
- is_commutative memoized per node (nodes are immutable and interned
bottom-up, so commutativity is fixed once computed).
- propagate_errors short-circuits via a context-level has_errors flag:
hash-consing guarantees no child can be an Error if none was interned.
- #[inline] on Number add/mul.
expand benchmarks vs the C port (median, same machine): expand3 52.5ms
(C 53.4ms), product_of_binary_sums_20 822ms (C 810ms), expand1 3.55ms
(C 3.30ms), expand2 220ms (C 167ms). add1 34.9ms (was 42.6ms).
Adds profile_expand2 bin target for sampling the expand2 workload.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014GcRnPSHE7EazAEDdrfUcj
Replace Box<[ExprRef]> with a new Children type for every slice-children Expr variant (Add, Mul, NCMul, TensorProduct, Function, Sum, Product, Integral, Derivative, Operator, Ket, Bra, And, Or, Xor, Piecewise, FiniteSet, Union, Intersection). Lists of up to 8 children are stored directly in the node — no heap allocation and no pointer chase on access; longer lists spill to a heap slice as before. Expr stays 64 bytes (the Number variant dominates), so the inline storage is free space-wise. The representation is canonical (len <= 8 is always inline), and Children hashes and compares as its slice view, so the interner's borrowed-candidate probes are unaffected. Children derefs to [ExprRef]; destructuring match sites are unchanged. Expand benchmarks improve another 3-5% (quiet-machine medians: binomial_20 7.89us, trinomial_10 22.7us, polynomial_5 17.4us, expand1 3.41ms, expand2 214ms, expand3 53.0ms). The C port comparison now stands at: expand3 and product_of_binary_sums_20 at parity or faster, expand1 within 3%, expand2 at 1.28x. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GcRnPSHE7EazAEDdrfUcj
…steps Adds spec/performance.md: the C port comparison methodology, current benchmark table (Rust at parity with C on expand1/expand3/pobs_20, 1.2-1.7x elsewhere), the optimizations that landed with their measured effects, negative results preserved so they are not re-tried (target-cpu=native, pre-sized interner tables, single- vs two-probe interning, and the 48-byte-node experiment on the experiment-48-byte-nodes branch), where the remaining gap lives, and ranked next steps (INLINE_CAP 11, integer fast-path accumulation, Tier-2 arena redesign as a proposed spec, PGO). Updates spec/internals.md and spec/accepted/expressions.md to describe the current implementation (ExprRef(u32), FastInterner with cached-hash slots and borrowed-candidate probes, inline Children storage), and links performance.md from design.md and the README. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GcRnPSHE7EazAEDdrfUcj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a draft, we'll extract useful parts into separate PRs / projects.