Skip to content

replace hashbrown with std::hashmap #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aneubeck merged 3 commits into from
Jun 3, 2026
Merged

replace hashbrown with std::hashmap #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
19c0ca7
replace hashbrown with std::hashmap
aneubeck Jun 3, 2026
392c620
Potential fix for pull request finding
aneubeck Jun 3, 2026
93f929f
Potential fix for pull request finding
aneubeck Jun 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 50 additions & 49 deletions crates/hash-sorted-map/OPTIMIZATIONS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@ and an **optimized growth strategy**. It is generic over key type, value type,
and hash builder.

This document analyzes the design trade-offs versus
[hashbrown](https://github.com/rust-lang/hashbrown) and records the
experimental results that guided the current design.
[hashbrown](https://github.com/rust-lang/hashbrown) — the Swiss-table
implementation that backs `std::collections::HashMap` — and records the
experimental results that guided the current design. The benchmark suite
drives `std::collections::HashMap` directly with various explicit `BuildHasher` configurations.

---

Expand Down Expand Up @@ -197,70 +199,69 @@ Hardware used for the current local snapshot:

### Insert (1000 trigrams, pre-sized)

| Implementation | Time (µs) | vs hashbrown |
|----------------------|-----------|--------------|
| FoldHashMap | 13.88 | −5% |
| FxHashMap | 14.60 | ~0% |
| hashbrown+Identity | 14.44 | baseline |
| hashbrown::HashMap | 14.55 | +1% |
| std::HashMap+FNV | 15.55 | +8% |
| AHashMap | 15.59 | +8% |
| **HashSortedMap** | **9.40** | **−35%** |
| std::HashMap | 25.26 | +75% |
| Implementation | Time (µs) | vs `std::HashMap+Identity` |
|-----------------------------|-----------|----------------------------|
| `std::HashMap+FoldHash` | 13.88 | −4% |
| `FxHashMap` | 14.60 | +1% |
| `std::HashMap+Identity` | 14.44 | baseline |
| `std::HashMap+FNV` | 15.55 | +8% |
| `std::HashMap+AHash` | 15.59 | +8% |
| **`HashSortedMap`** | **9.40** | **−35%** |
| `std::HashMap` (RandomState)| 25.26 | +75% |

### Reinsert (1000 trigrams, all keys exist)

| Implementation | Time (µs) |
|----------------------|-----------|
| **HashSortedMap** | **6.59** |
| hashbrown+Identity | 6.95 |
| Implementation | Time (µs) |
|-------------------------|-----------|
| **`HashSortedMap`** | **6.59** |
| `std::HashMap+Identity` | 6.95 |

### Growth (128 → 1000 trigrams, 3 resize rounds)

| Implementation | Time (µs) |
|----------------------|-----------|
| hashbrown+Identity | 26.66 |
| **HashSortedMap** | **27.50** |
| Implementation | Time (µs) |
|-------------------------|-----------|
| `std::HashMap+Identity` | 26.66 |
| **`HashSortedMap`** | **27.50** |

### Count (4000 trigrams, mixed insert/update)

| Implementation | Time (µs) |
|----------------------------------|-----------|
| hashbrown+Identity entry() | 15.49 |
| **HashSortedMap get_or_default** | **15.88** |
| **HashSortedMap entry().or_default()** | **16.15** |
| Implementation | Time (µs) |
|-----------------------------------------|-----------|
| `std::HashMap+Identity` `entry()` | 15.49 |
| **`HashSortedMap get_or_default`** | **15.88** |
| **`HashSortedMap entry().or_default()`**| **16.15** |

### Iteration (1000 trigrams)

| Implementation | Time (µs) |
|-------------------------------|-----------|
| **HashSortedMap iter()** | **3.02** |
| hashbrown+Identity iter() | 3.04 |
| **HashSortedMap into_iter()** | **3.03** |
| hashbrown+Identity into_iter()| 3.56 |
| Implementation | Time (µs) |
|--------------------------------------|-----------|
| **`HashSortedMap iter()`** | **3.02** |
| `std::HashMap+Identity` `iter()` | 3.04 |
| **`HashSortedMap into_iter()`** | **3.03** |
| `std::HashMap+Identity` `into_iter()`| 3.56 |

### Sort (100K trigrams)

| Implementation | Time (ms) |
|-----------------------------|-----------|
| **HashSortedMap sort_by_hash** | **1.66** |
| Vec::sort_unstable | 2.20 |
| Implementation | Time (ms) |
|--------------------------------|-----------|
| **`HashSortedMap sort_by_hash`** | **1.66** |
| `Vec::sort_unstable` | 2.20 |

### Merge (100 maps × 100K keys each → sorted output)

| Implementation | Time (ms) | vs HSM merge+sort |
|-----------------------------------|-----------|--------------------|
| hashbrown merge presized | 160.79 | +6% |
| **HashSortedMap merge presized** | **117.01**| **−23%** |
| **HashSortedMap merge (no sort)** | **141.57**| **−7%** |
| hashbrown merge | 163.59 | +7% |
| **HashSortedMap merge + sort** | **152.34**| **baseline** |
| hashbrown merge + Vec sort | 193.37 | +27% |
| k-way merge sorted vecs | 445 | +192% |
| Implementation | Time (ms) | vs HSM merge+sort |
|-----------------------------------------------|-----------|--------------------|
| `std::HashMap+Identity` merge presized | 160.79 | +6% |
| **`HashSortedMap` merge presized** | **117.01**| **−23%** |
| **`HashSortedMap` merge (no sort)** | **141.57**| **−7%** |
| `std::HashMap+Identity` merge | 163.59 | +7% |
| **`HashSortedMap` merge + sort** | **152.34**| **baseline** |
| `std::HashMap+Identity` merge + Vec sort | 193.37 | +27% |
| k-way merge sorted vecs | 445 | +192% |

**Key takeaways:**
- Pre-sized insert is **~35% faster** than hashbrown+Identity
- Reinsert and iter paths are now close to parity with hashbrown+Identity
- Growth path is currently **~3% slower** than hashbrown+Identity
- sort_by_hash is **~24% faster** than Vec::sort_unstable
- merge + sort is **~21% faster** than hashbrown merge + Vec sort
- Pre-sized insert is **~35% faster** than `std::HashMap+Identity`
- Reinsert and iter paths are now close to parity with `std::HashMap+Identity`
- Growth path is currently **~3% slower** than `std::HashMap+Identity`
- sort_by_hash is **~24% faster** than `Vec::sort_unstable`
- merge + sort is **~21% faster** than `std::HashMap+Identity` merge + Vec sort
22 changes: 12 additions & 10 deletions crates/hash-sorted-map/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,21 +53,23 @@ Hardware used for this snapshot:
- CPU frequency range: 800 MHz to 2800 MHz
- Memory: 7.8 GiB RAM

| Scenario | HashSortedMap | Comparison | Result |
| :------------------------------------------- | ------------: | :------------------------------------- | :---------- |
| Insert 1000 trigrams (pre-sized) | 9.40 µs | hashbrown::HashMap: 14.55 µs | ~35% faster |
| Grow from capacity 128 | 27.50 µs | hashbrown+Identity: 26.66 µs | ~3% slower |
| Count 4000 trigrams (`entry().or_default()`) | 16.15 µs | hashbrown+Identity `entry()`: 15.49 µs | ~4% slower |
| Iterate 1000 trigrams (`iter()`) | 3.02 µs | hashbrown+Identity `iter()`: 3.04 µs | ~1% faster |
| Sort 100000 trigrams by hash | 1.66 ms | `Vec::sort_unstable`: 2.20 ms | ~24% faster |
| Merge 100 sorted maps + final sort | 152.34 ms | hashbrown merge + vec sort: 193.37 ms | ~21% faster |
| Scenario | HashSortedMap | Comparison | Result |
| :------------------------------------------- | ------------: | :---------------------------------------- | :---------- |
| Insert 1000 trigrams (pre-sized) | 9.40 µs | `std::HashMap+FoldHash`: 14.55 µs | ~35% faster |
| Grow from capacity 128 | 27.50 µs | `std::HashMap+Identity`: 26.66 µs | ~3% slower |
| Count 4000 trigrams (`entry().or_default()`) | 16.15 µs | `std::HashMap+Identity` `entry()`: 15.49 µs | ~4% slower |
| Iterate 1000 trigrams (`iter()`) | 3.02 µs | `std::HashMap+Identity` `iter()`: 3.04 µs | ~1% faster |
| Sort 100000 trigrams by hash | 1.66 ms | `Vec::sort_unstable`: 2.20 ms | ~24% faster |
| Merge 100 sorted maps + final sort | 152.34 ms | `std::HashMap+Identity` merge + vec sort: 193.37 ms | ~21% faster |

> Note: `std::collections::HashMap` is `hashbrown` under the hood, so the benchmark drives `std::collections::HashMap` directly with the same custom `BuildHasher`s that were previously passed to `hashbrown`.

Key takeaways:

- Pre-sized inserts, sorting, and merge+sort remain the strongest paths.
- Iteration is now roughly on par with `hashbrown+Identity`.
- Iteration is now roughly on par with `std::HashMap+Identity`.
- Growth and count/update workloads are currently slightly slower than
`hashbrown+Identity` in this run.
`std::HashMap+Identity` in this run.

## Running

Expand Down
1 change: 0 additions & 1 deletion crates/hash-sorted-map/benchmarks/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ criterion = "0.8"
rand = "0.10"
rustc-hash = "2"
ahash = "0.8"
hashbrown = "0.15"
foldhash = "0.1"
fnv = "1"
itertools = "0.14"
68 changes: 29 additions & 39 deletions crates/hash-sorted-map/benchmarks/performance.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,12 @@ fn bench_insert(c: &mut Criterion) {
);
});

group.bench_function("hashbrown::HashMap", |b| {
group.bench_function("std::HashMap+FoldHash", |b| {
b.iter_batched(
|| hashbrown::HashMap::with_capacity(trigrams.len()),
|| std::collections::HashMap::<u32, usize, foldhash::fast::FixedState>::with_capacity_and_hasher(
trigrams.len(),
foldhash::fast::FixedState::default(),
),
|mut map| {
for (i, &key) in trigrams.iter().enumerate() {
map.insert(key, i);
Expand All @@ -52,24 +55,11 @@ fn bench_insert(c: &mut Criterion) {
);
});

group.bench_function("AHashMap", |b| {
group.bench_function("std::HashMap+AHash", |b| {
b.iter_batched(
|| ahash::AHashMap::with_capacity(trigrams.len()),
|mut map| {
for (i, &key) in trigrams.iter().enumerate() {
map.insert(key, i);
}
map
},
BatchSize::SmallInput,
);
});

group.bench_function("FoldHashMap", |b| {
b.iter_batched(
|| hashbrown::HashMap::<u32, usize, foldhash::fast::FixedState>::with_capacity_and_hasher(
|| std::collections::HashMap::<u32, usize, ahash::RandomState>::with_capacity_and_hasher(
trigrams.len(),
foldhash::fast::FixedState::default(),
ahash::RandomState::default(),
),
|mut map| {
for (i, &key) in trigrams.iter().enumerate() {
Expand Down Expand Up @@ -99,10 +89,10 @@ fn bench_insert(c: &mut Criterion) {
);
});

group.bench_function("hashbrown+Identity", |b| {
group.bench_function("std::HashMap+Identity", |b| {
b.iter_batched(
|| {
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
trigrams.len(),
Default::default(),
)
Expand Down Expand Up @@ -142,11 +132,11 @@ fn bench_reinsert(c: &mut Criterion) {
let trigrams = trigrams();
let mut group = c.benchmark_group("reinsert_1000_trigrams");

group.bench_function("hashbrown+Identity", |b| {
group.bench_function("std::HashMap+Identity", |b| {
b.iter_batched(
|| {
let mut map =
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
trigrams.len(),
Default::default(),
);
Expand Down Expand Up @@ -194,10 +184,10 @@ fn bench_grow(c: &mut Criterion) {
let trigrams = trigrams();
let mut group = c.benchmark_group("grow_from_128_insert_1000_trigrams");

group.bench_function("hashbrown+Identity", |b| {
group.bench_function("std::HashMap+Identity", |b| {
b.iter_batched(
|| {
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
128,
Default::default(),
)
Expand Down Expand Up @@ -237,10 +227,10 @@ fn bench_count(c: &mut Criterion) {

let mut group = c.benchmark_group("count_4000_trigrams_get_or_default");

group.bench_function("hashbrown+Identity entry()", |b| {
group.bench_function("std::HashMap+Identity entry()", |b| {
b.iter_batched(
|| {
hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
trigrams.len(),
Default::default(),
)
Expand Down Expand Up @@ -299,11 +289,11 @@ fn bench_iter(c: &mut Criterion) {

let mut group = c.benchmark_group("iter_1000_trigrams");

group.bench_function("hashbrown+Identity iter()", |b| {
group.bench_function("std::HashMap+Identity iter()", |b| {
b.iter_batched(
|| {
let mut map =
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
trigrams.len(),
Default::default(),
);
Expand Down Expand Up @@ -346,11 +336,11 @@ fn bench_iter(c: &mut Criterion) {
);
});

group.bench_function("hashbrown+Identity into_iter()", |b| {
group.bench_function("std::HashMap+Identity into_iter()", |b| {
b.iter_batched(
|| {
let mut map =
hashbrown::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, usize, IdentityBuildHasher>::with_capacity_and_hasher(
trigrams.len(),
Default::default(),
);
Expand Down Expand Up @@ -507,10 +497,10 @@ fn bench_merge_sort(c: &mut Criterion) {
});
});

// ── 3. hashbrown HashMap merge, then sort into Vec ──────────────
group.bench_function("hashbrown merge + Vec sort", |b| {
// ── 3. std::HashMap merge, then sort into Vec ──────────────────
group.bench_function("std::HashMap+Identity merge + Vec sort", |b| {
b.iter(|| {
let mut map = hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
let mut map = std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
IdentityBuildHasher::default(),
);
for container in &hash_maps {
Expand All @@ -528,10 +518,10 @@ fn bench_merge_sort(c: &mut Criterion) {
});
});

// ── 4. hashbrown HashMap merge only (no sort) ───────────────────
group.bench_function("hashbrown merge", |b| {
// ── 4. std::HashMap merge only (no sort) ───────────────────────
group.bench_function("std::HashMap+Identity merge", |b| {
b.iter(|| {
let mut map = hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
let mut map = std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_hasher(
IdentityBuildHasher::default(),
);
for container in &hash_maps {
Expand All @@ -557,11 +547,11 @@ fn bench_merge_sort(c: &mut Criterion) {
});
});

// ── 6. hashbrown presized merge only ────────────────────────────
group.bench_function("hashbrown merge presized", |b| {
// ── 6. std::HashMap presized merge only ────────────────────────
group.bench_function("std::HashMap+Identity merge presized", |b| {
b.iter(|| {
let mut map =
hashbrown::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
std::collections::HashMap::<u32, u32, IdentityBuildHasher>::with_capacity_and_hasher(
1_000_000,
IdentityBuildHasher::default(),
);
Expand Down