Skip to content

Add the max-element-density Snappy corpus to the bench harness - #345

Merged
iderex merged 1 commit into
mainfrom
issue/166-worst-density
Aug 10, 2026
Merged

Add the max-element-density Snappy corpus to the bench harness#345
iderex merged 1 commit into
mainfrom
issue/166-worst-density

Conversation

@iderex

@iderex iderex commented Aug 10, 2026

Copy link
Copy Markdown
Owner

What & why

Closes #166.

The Snappy bench harness had only the Silesia corpus, which measures the
format's ordinary case. The regime a per-element decoder floors in is the
opposite one, and the reference compressor never emits it: it extends an
offset-1 run into a single long copy, which is the best case rather than the
worst. So the corpus is constructed, and the reference is the authority on
whether the construction is a real Snappy stream rather than the author of it.

Back-to-back minimum-cost copies at offset 1. One element per two compressed
bytes, one per four decoded bytes, which is the maximum element density the
format can carry. This is the Snappy analog of the LZ4 worst-4Bmatch block, at
the same 64 KiB unit and the same 200 MB scale, so the two adversarial numbers
will be comparable once a Snappy kernel exists.

The means is C++ in bench/bench_snappy.cpp beside the corpus it contrasts
with, because the two corpora share the report, the timing loop and the oracle
gate and differ only in how the streams are produced.

Type of change

  • Build / CI / supply chain
  • Decode kernel / device code
  • Format support (LZ4 / Snappy / GDeflate / Zstd)
  • API surface
  • Performance
  • Bug fix
  • Refactor / code quality
  • Docs

Engineering checklist

  • Fail-closed preserved. The construction refuses a block below 256 bytes
    and a tail the one-byte literal tag cannot encode, rather than emitting
    something that is not the shape it is named for. --worst refuses to be
    combined with corpus files or a shape flag.
  • Oracle coverage: the pinned snappy 1.2.2 decoder passes verdict on every
    constructed stream and the round trip is compared byte for byte, both
    before any timing.
  • Determinism preserved. Nothing in the library moves; the construction
    takes no randomness at all.
  • No CUDA call is added.
  • Adversarial review: one reader, four lenses. See the disclosure at the
    end.
  • Review record below.

Review record

CONFIRMED 0 / PLAUSIBLE 1.

  • Correctness. The tag byte is derived from the format rather than copied: kind
    01 is the one-byte-offset copy, a zero length field reads as 4, and zero in
    the three high bits leaves the offset as the following byte alone. The
    measured density says the derivation is right, 3.9998 decoded bytes per
    element against a designed 4. The tail arithmetic cannot underflow because
    the loop stops while at least the tail remains and the tail is checked
    against what the one-byte literal tag encodes. No finding.
  • Robustness. PLAUSIBLE: CheckDensity divides by compressed_bytes, which
    would be a division by zero on an empty corpus. Disposition DECLINE with the
    reason, rather than a guard nothing can reach: the only caller builds the
    corpus immediately above it from a block that is refused below 256 bytes,
    so a zero-byte corpus does not exist at that call site. Adding a branch no
    input reaches would be a guard that cannot be shown to bite.
  • Performance. The corpus is built once and replicated by copy, which is what
    the LZ4 worst-case path does for the same reason. Outside every timed
    region. No finding.
  • Integration. bench/ stays consumer-only. The new ctest entry carries the
    mandatory finite timeout and passes cudec_assert_test_timeouts(), and it
    carries no gpu label, so it runs in the CI selection rather than being
    deferred to the local gate.

The two locks, and the proof that each bites

Validity is not enough. A valid-but-easy stream round-trips and would leave the
report printing "max element density" over a corpus that no longer is one. So
there are two floors, because the corpus can stop being adversarial in two ways
that do not overlap:

  • A generator whose copies grew longer still emits one element per two
    compressed bytes, so an element-rate floor would not notice. What moves is
    the compressed share, from a half toward a fifth.
  • A generator that switched to a costlier element form keeps the compressed
    share and loses the element rate.

Both proven in the container on this branch, by breaking them one at a time and
restoring after each.

Copies lengthened from 4 to 11, which is the same one-byte-offset copy form
with its length field filled:

=== PROOF 1: lengthen the copies, the density floor must red ===
the max-density corpus compressed to 0.1820 of its output, below the 0.49 floor: its elements are producing more bytes each than the minimum copy, so this is no longer the worst case it reports
0% tests passed, 1 tests failed out of 1

One byte of the constructed stream flipped:

=== PROOF 2: corrupt one byte, the oracle gate must reject before timing ===
the oracle refuses stream 0 of max element density (constructed)
0% tests passed, 1 tests failed out of 1
=== restored, both selfchecks green again ===
100% tests passed, 0 tests failed out of 2

GPU sanitizer gate

  • Not applicable: this change touches no device code. bench_snappy is a
    plain C++ target with no CUDA in it, and no .cu file is modified.

So the block is not read as an answered one: #258 records that no route to a
device the sanitizer can attach to is available on this machine.

Performance checklist

  • Measured, not reasoned.
  • No regression against docs/BENCHMARKS.md. Nothing recorded there moves.

The corpus at its full scale, in the container on the local host. This is
evidence that the construction is adversarial, not a baseline entry: the issue
asks for the corpus, its oracle gate and its density lock, and recording a
number for a decoder that does not exist yet would be an entry nothing reads.

bench_snappy --worst --warmup 3 --runs 30

- corpus: max element density (constructed), 64 KiB-chunked streams, 3200 streams, 209.72 MB original, 104.88 MB compressed (ratio 0.500), hand-constructed in-harness as back-to-back minimum-cost copies at offset 1, which the reference compressor never emits; every stream validated by the pinned snappy oracle before timing
- wall per run: p50 1246.117 ms / p90 1368.566 ms / p99 1440.939 ms
- decode throughput: p50 0.168 GB/s / p90 0.153 GB/s / p99 0.146 GB/s
- element density: 52432000 elements, 0.4999 per compressed byte, 3.9998 decoded bytes each

0.168 GB/s against the 1.197 GB/s the same decoder reaches on Silesia in the M3
entry, on the same host and through the same timing loop. The corpus is 7.1
times harder for the reference than the ordinary case, which is what a maximum
element density is supposed to cost.

Quality checklist

  • Minimal. The tally moves out of CompressAll into its own function
    because this corpus is not compressed by the reference at all, and an
    accounting that only ran inside the compressor would report zeroes for
    it. Everything else is reused: the report, the timing loop, the oracle
    gate and the digest.
  • Self-documenting. The comments say why there are two floors rather than
    one, and where the tag byte's value comes from.
  • Conformance tests pass. The structural property this establishes, that
    the corpus is still adversarial, is locked by the selfcheck above.

Verification

Full gate in the pinned container against the local RTX 3080, at this branch's
head:

cmake -B build-cuda -DCUDEC_ENABLE_CUDA=ON && cmake --build build-cuda -j 12
ctest --test-dir build-cuda --output-on-failure

100% tests passed, 0 tests failed out of 38
Label Time Summary:
gpu    =   4.98 sec*proc (8 tests)
Total Test time (real) =   8.65 sec
  • Builds clean under -Wall -Wextra -Werror.
  • ctest green, 38 of 38. The count was 37 before this branch; the new
    entry is bench_snappy_worst_selfcheck.
  • Prettier clean. No .md, .yml or .yaml file is touched.
  • Docs synced. No behaviour, API or recorded number moves.

Notes

This change had no second reader. The review record above is one reader over
four lenses, and the two break-and-restore proofs and the pasted gate output
are the evidence in place of one.

The Snappy bench had only the Silesia corpus, which measures the format's
ordinary case. The regime a per-element decoder actually floors in is the
opposite one, and the reference compressor never produces it: it extends an
offset-1 run into a single long copy, which is the best case rather than the
worst.

The corpus is constructed instead. Back-to-back minimum-cost copies at offset
1, one element per two compressed bytes and one per four decoded bytes, which
is the maximum element density the format can carry. Measured on the built
corpus rather than claimed: 0.4999 elements per compressed byte, 3.9998
decoded bytes each, and the reference decodes it at 0.168 GB/s against 1.197
GB/s on Silesia.

Two things can go wrong here and neither stops the stream decoding, so both
are checked before any timing. The oracle passes verdict on validity, as it
does for every corpus in this harness. The density floors catch the other
half: a generator whose copies grew longer keeps its element rate and loses
its compressed share, and one that switched to a costlier element form does
the reverse, so there is a floor on each. A valid-but-easy corpus would
round-trip and leave the report claiming a worst case it no longer measures.

Both proven by breaking them. Copies lengthened from 4 to 11: the share floor
reds at 0.1820. One byte of the constructed stream flipped: the oracle refuses
it and nothing is timed.

The tally moves out of CompressAll, because this corpus is not compressed by
the reference at all and an accounting that only ran inside the compressor
would report zeroes for it.
@iderex iderex added enhancement New feature or request area:bench Benchmarks and baselines area:tests Harness, oracles, negative and conformance tests labels Aug 10, 2026
@iderex iderex self-assigned this Aug 10, 2026
@iderex iderex added the priority:p2 Next: current milestone, not blocking label Aug 10, 2026
@iderex iderex added this to the M3 - Snappy milestone Aug 10, 2026
@iderex
iderex merged commit d694a13 into main Aug 10, 2026
7 checks passed
@iderex
iderex deleted the issue/166-worst-density branch August 10, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:bench Benchmarks and baselines area:tests Harness, oracles, negative and conformance tests enhancement New feature or request priority:p2 Next: current milestone, not blocking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M3 bench: the Snappy max-element-density worst-case corpus, oracle-gated and density-locked

1 participant