Add the max-element-density Snappy corpus to the bench harness - #345
Merged
Conversation
The Snappy bench had only the Silesia corpus, which measures the format's ordinary case. The regime a per-element decoder actually floors in is the opposite one, and the reference compressor never produces it: it extends an offset-1 run into a single long copy, which is the best case rather than the worst. The corpus is constructed instead. Back-to-back minimum-cost copies at offset 1, one element per two compressed bytes and one per four decoded bytes, which is the maximum element density the format can carry. Measured on the built corpus rather than claimed: 0.4999 elements per compressed byte, 3.9998 decoded bytes each, and the reference decodes it at 0.168 GB/s against 1.197 GB/s on Silesia. Two things can go wrong here and neither stops the stream decoding, so both are checked before any timing. The oracle passes verdict on validity, as it does for every corpus in this harness. The density floors catch the other half: a generator whose copies grew longer keeps its element rate and loses its compressed share, and one that switched to a costlier element form does the reverse, so there is a floor on each. A valid-but-easy corpus would round-trip and leave the report claiming a worst case it no longer measures. Both proven by breaking them. Copies lengthened from 4 to 11: the share floor reds at 0.1820. One byte of the constructed stream flipped: the oracle refuses it and nothing is timed. The tally moves out of CompressAll, because this corpus is not compressed by the reference at all and an accounting that only ran inside the compressor would report zeroes for it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Closes #166.
The Snappy bench harness had only the Silesia corpus, which measures the
format's ordinary case. The regime a per-element decoder floors in is the
opposite one, and the reference compressor never emits it: it extends an
offset-1 run into a single long copy, which is the best case rather than the
worst. So the corpus is constructed, and the reference is the authority on
whether the construction is a real Snappy stream rather than the author of it.
Back-to-back minimum-cost copies at offset 1. One element per two compressed
bytes, one per four decoded bytes, which is the maximum element density the
format can carry. This is the Snappy analog of the LZ4 worst-4Bmatch block, at
the same 64 KiB unit and the same 200 MB scale, so the two adversarial numbers
will be comparable once a Snappy kernel exists.
The means is C++ in
bench/bench_snappy.cppbeside the corpus it contrastswith, because the two corpora share the report, the timing loop and the oracle
gate and differ only in how the streams are produced.
Type of change
Engineering checklist
and a tail the one-byte literal tag cannot encode, rather than emitting
something that is not the shape it is named for.
--worstrefuses to becombined with corpus files or a shape flag.
constructed stream and the round trip is compared byte for byte, both
before any timing.
takes no randomness at all.
end.
Review record
CONFIRMED 0 / PLAUSIBLE 1.
01 is the one-byte-offset copy, a zero length field reads as 4, and zero in
the three high bits leaves the offset as the following byte alone. The
measured density says the derivation is right, 3.9998 decoded bytes per
element against a designed 4. The tail arithmetic cannot underflow because
the loop stops while at least the tail remains and the tail is checked
against what the one-byte literal tag encodes. No finding.
CheckDensitydivides bycompressed_bytes, whichwould be a division by zero on an empty corpus. Disposition DECLINE with the
reason, rather than a guard nothing can reach: the only caller builds the
corpus immediately above it from a block that is refused below 256 bytes,
so a zero-byte corpus does not exist at that call site. Adding a branch no
input reaches would be a guard that cannot be shown to bite.
the LZ4 worst-case path does for the same reason. Outside every timed
region. No finding.
bench/stays consumer-only. The new ctest entry carries themandatory finite timeout and passes
cudec_assert_test_timeouts(), and itcarries no gpu label, so it runs in the CI selection rather than being
deferred to the local gate.
The two locks, and the proof that each bites
Validity is not enough. A valid-but-easy stream round-trips and would leave the
report printing "max element density" over a corpus that no longer is one. So
there are two floors, because the corpus can stop being adversarial in two ways
that do not overlap:
compressed bytes, so an element-rate floor would not notice. What moves is
the compressed share, from a half toward a fifth.
share and loses the element rate.
Both proven in the container on this branch, by breaking them one at a time and
restoring after each.
Copies lengthened from 4 to 11, which is the same one-byte-offset copy form
with its length field filled:
One byte of the constructed stream flipped:
GPU sanitizer gate
bench_snappyis aplain C++ target with no CUDA in it, and no
.cufile is modified.So the block is not read as an answered one: #258 records that no route to a
device the sanitizer can attach to is available on this machine.
Performance checklist
The corpus at its full scale, in the container on the local host. This is
evidence that the construction is adversarial, not a baseline entry: the issue
asks for the corpus, its oracle gate and its density lock, and recording a
number for a decoder that does not exist yet would be an entry nothing reads.
0.168 GB/s against the 1.197 GB/s the same decoder reaches on Silesia in the M3
entry, on the same host and through the same timing loop. The corpus is 7.1
times harder for the reference than the ordinary case, which is what a maximum
element density is supposed to cost.
Quality checklist
CompressAllinto its own functionbecause this corpus is not compressed by the reference at all, and an
accounting that only ran inside the compressor would report zeroes for
it. Everything else is reused: the report, the timing loop, the oracle
gate and the digest.
one, and where the tag byte's value comes from.
the corpus is still adversarial, is locked by the selfcheck above.
Verification
Full gate in the pinned container against the local RTX 3080, at this branch's
head:
-Wall -Wextra -Werror.ctestgreen, 38 of 38. The count was 37 before this branch; the newentry is
bench_snappy_worst_selfcheck..md,.ymlor.yamlfile is touched.Notes
This change had no second reader. The review record above is one reader over
four lenses, and the two break-and-restore proofs and the pasted gate output
are the evidence in place of one.