Skip to content

Add the Silesia Zstd corpus path and CPU denominator to the bench harness - #347

Merged
iderex merged 2 commits into
mainfrom
issue/227-zstd-corpus
Aug 11, 2026
Merged

Add the Silesia Zstd corpus path and CPU denominator to the bench harness#347
iderex merged 2 commits into
mainfrom
issue/227-zstd-corpus

Conversation

@iderex

@iderex iderex commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Closes #227.

bench_zstd could only build its hand-constructed worst case. The standard corpora had no path through it, so the M5 record carried no ratio and no CPU denominator, and a later device number would have had nothing to be read against.

What this adds

A standard corpus path that cuts a fetched corpus into independent frames at both endpoints of the 64 KiB to 512 KiB granularity range the batch model names, across levels 1, 3 and 19, and times single-thread ZSTD_decompress over each of the six cells. It consumes MakeZstdBatchFrames from tests/zstd_corpus.h rather than adding a second generator, which is what the issue asks for.

The forced-mode fixtures are run by bench_zstd --coverage. They report which decode surfaces were reached and state in their own output that they carry no number, because fixtures sized to reach one surface each cannot carry a throughput figure.

Recorded numbers

Measured in the digest-pinned nvidia/cuda:12.6.2-devel-ubuntu24.04 container on an AMD Ryzen 9 5950X, libzstd 1.5.7, over the twelve Silesia files at 211.94 MB. Reproduce with bench_zstd --warmup 3 --runs 30 bench/corpora/silesia/*.

granularity level compressed ratio p50 decode
64 KiB 1 76.70 MB 0.3619 1.281 GB/s
64 KiB 3 73.52 MB 0.3469 1.301 GB/s
64 KiB 19 65.04 MB 0.3069 1.086 GB/s
512 KiB 1 73.95 MB 0.3489 1.510 GB/s
512 KiB 3 68.23 MB 0.3219 1.404 GB/s
512 KiB 19 58.65 MB 0.2767 1.262 GB/s

docs/BENCHMARKS.md carries all six blocks with their full methodology rather than this table alone.

The reading worth carrying forward is that this disagrees with the Snappy entry above it. Cutting to 64 KiB frames costs libzstd 8 to 14 percent, where the same cut cost snappy 0.18 percent and sat inside its own run-to-run spread. So a device figure has to be quoted against the denominator at its own granularity; reading a 64 KiB batch decode against the 512 KiB row would credit the device with a difference that is the corpus shape.

The two guards, and the proof that each bites

A corpus can stop being the recorded one in two ways that both still round-trip, so there are two guards and both were checked by breaking them.

The frames are decoded back and their concatenation compared against the source before anything is timed. Dropping one frame from every built corpus:

the frames decode to 3080192 bytes but the source is 3145728 - the corpus is not the one the report would name
exit=1

Each cell's corpus digest is pinned in the selfcheck. Moving the selfcheck source from 3 MiB to 2 MiB:

6
the selfcheck corpus digest moved at 64 KiB frames, level 1: expected 24921f467ca2abc6, built 6bc8b18068e83092 - the corpus this harness constructs is not the one its numbers were recorded on
exit=1

All six cells red, not one, because the grid reports every digest before judging any of them. Reverting both mutations returns the selfcheck to exit=0.

The pin table and the run grid are indexed by the same two loop variables. A level added to one and not the other is now refused at compile time by a static_assert rather than read off the end of the array inside the selfcheck, which is the one place a wrong answer would look like a verdict.

Gate

Full build and ctest in the digest-pinned container against the local RTX 3080, driver 560.94, CUDA 12.6.2:

100% tests passed, 0 tests failed out of 40

40 rather than the previous 38: bench_zstd_selfcheck and bench_zstd_coverage_selfcheck. Both are CPU-only so they run on the GPU-less runner, and both carry the finite TIMEOUT that cudec_assert_test_timeouts() requires.

npx prettier@3 --check "**/*.{md,yml,yaml}" reports all matched files use Prettier code style.

What this does not claim

No second reader has looked at this change. What stands in place of one is the evidence above: the two guards were each mutated and watched to fail, the numbers carry the commands that produced them, and the gate output is quoted rather than summarised.

The compute-sanitizer net did not run over this change and nothing here asserts it is clean under those tools. That gate cannot attach to a device on this route, which is recorded on #258, and this change adds no device code.

No claim is made about why level 19 decodes slowest while compressing best. The per-phase split that would answer it is a separate entry.

iderex added 2 commits August 11, 2026 10:42
…ness [#227]

bench_zstd could only build its hand-constructed worst case. The standard
corpora had no path through it at all, so the M5 record carried no ratio and
no CPU denominator, and a later device number would have had nothing to be
read against.

The new path cuts a fetched corpus into independent frames at both endpoints
of the 64 KiB to 512 KiB granularity range the batch model names, across
levels 1, 3 and 19, and times single-thread ZSTD_decompress over each cell.
It consumes MakeZstdBatchFrames from tests/zstd_corpus.h rather than adding a
second generator, and the forced-mode fixtures beside it are run as a
coverage report that states in its own output that it carries no number.

Two guards, because the corpus can stop being the recorded one in two ways
that both still round-trip. The frames are decoded back and their
concatenation compared against the source, so a frame set that dropped or
reordered part of the corpus fails instead of reporting a plausible ratio
over different bytes. And each cell's corpus digest is pinned in the
selfcheck, so a moved compressor pin or a changed granularity reds CI. Both
were checked by breaking them: shrinking the selfcheck source reds all six
digest cells, and dropping one frame per corpus trips the reconstruction
check.

The pin table and the run grid are indexed by the same two loop variables, so
a level added to one and not the other is refused at compile time rather than
read off the end of the array inside the selfcheck.

docs/BENCHMARKS.md gains the M5 section with all six recorded cells and their
methodology. The measurement it reports and the Snappy entry above it
disagree: cutting to 64 KiB frames costs libzstd 8 to 14 percent where it
cost snappy nothing measurable, so a device figure has to be quoted against
the denominator at its own granularity.
Both sides added an entry to the bench selfcheck timeout list and a section
to the benchmark record. Kept both in each: the parse-bound Snappy corpus
from the mainline and the Zstd corpus path from this branch.

Verified after the merge in the pinned container against the local device:
41/41 ctest, 0 failed.
@iderex
iderex merged commit 1294267 into main Aug 11, 2026
7 checks passed
@iderex
iderex deleted the issue/227-zstd-corpus branch August 11, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M5 bench: the Zstd corpus path (batch-shaped frames, level set, ratio) and the libzstd CPU denominator

1 participant