perf(decode): cut per-frame fixed overhead on the in-memory path

[polaz]

Summary

Cuts the per-frame fixed overhead on the in-memory decode path, where it
dominates small / high-entropy frames. On the small-1k-random level-4 frame
(a single RAW block — the shape random / already-compressed payloads always
take) the pure-Rust decode goes from ~86 ns to ~36 ns (−58 %) and now
beats the C reference (42.8 ns → 1.18×), up from 0.49× before. The wins are
byte-identical and generalize to other small frames (small-10k-random decode
−17 %); larger frames where the per-block work dominates see proportionally
less.

All four commits keep the wire output and decoded bytes identical (full decode +
fuzz_interop suites pass); the streaming Read decode path is untouched.

Profiling note

The decode path was profiled on the x86_64 bench host with callgrind +
perf (built -Csplit-debuginfo=off so valgrind resolves Rust source lines).
The per-frame machinery — not the block copy — was ~50 % of a tiny-frame decode;
__memcpy (the raw copy) is already at parity with C.

Changes

Skip the per-frame entropy-table reset when unbuilt

The per-frame scratch reset cleared all three sequence FSE tables and the
Huffman table (buffers + a [0; 13] rank-count memset) even on RAW / RLE /
raw-literal frames that never build an entropy table. Upstream zstd's
ZSTD_decompressBegin never clears entropy per frame — it marks tables invalid
by flag and rebuilds lazily. Gate each reset on whether the table holds built
state (FSE accuracy_log != 0, Huffman max_num_bits != 0); resetting an
already-empty table is a no-op on observable state, so this is byte-identical.

Direct copy for a single-RAW-block frame

A frame that is one RAW block spanning the whole declared content went through
the full direct-decode machinery (DirectScratch / DecodeBuffer /
UserSliceBackend wrappers + the general per-block loop) for what is one memcpy.
Mirror upstream's ZSTD_copyRawBlock: detect the single last RAW block whose
size equals the frame content size and copy it straight into the caller slice,
with the same checksum / counter bookkeeping. Falls through to the general path
for every other shape.

Parse the frame header straight from the input slice

The in-memory path holds the input as a &[u8], but the header parse went
through read_frame_header_with_format's generic Read interface: 3–5 per-field
read_exact calls dispatched through the io::impls Read-for-&[u8] shim,
plus per-byte little-endian assembly loops. This was the single biggest
tiny-frame cost (the non-inlined read_exact call overhead far exceeds its
instruction count). Add read_frame_header_from_slice — direct byte indexing +
from_le_bytes, advancing the slice exactly as the Read version does
(identical skippable-frame / truncation contracts) — and route decode_all /
the skippable-visitor path through a reset_from_slice that uses it, sharing the
parsed-header apply with the Read path. Mirrors upstream parsing the header
from a raw pointer. This single change is −43 % on the 1 KiB frame.

Drop the redundant no-dict arm from the compress-dict bench matrix

c_ffi_without_dict re-measured C compressing the same level/scenario without a
dictionary — already covered by the plain compress/{level}/{scenario}/matrix/ c_ffi group, and it built the CCtx per-iter while the two dictionary arms are
steady-state, so it was not even a clean baseline. Keep only c_ffi_with_dict
and pure_rust_with_dict.

Testing

• cargo nextest run -p structured-zstd --features hash,std,dict_builder — 837 pass
• cargo nextest run -p ffi-bench --features bench_internals,dict_builder — 59 pass
  (includes fuzz_interop round-trip + skippable-frame coverage)
• cargo clippy (default + --no-default-features no-std + --tests) clean; cargo fmt --check clean
• i9 (x86_64 BMI2/AVX2) before/after on compare_ffi decode with a flat c_ffi
  control arm; the three perf commits measured individually and cumulatively

Summary by CodeRabbit

• New Features
  • Enhanced decompression to parse and decode frames directly from in-memory slices, improving efficiency for non-stream input paths.
• Bug Fixes
  • Improved consistency in how frame content checksums and single-block/raw fast-path frames are processed.
• Performance Improvements
  • Reduced unnecessary decoder and entropy-table reset work by lazily clearing FSE/Huffman structures only when needed.
• Tests
  • Added coverage to ensure the FSE “populated” state is detected correctly for RLE builds.
• Benchmarks
  • Adjusted dictionary benchmark coverage so dictionary timings are isolated more accurately.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ce00eef2-5571-42ea-b14d-d4fe0cef33e4

📥 Commits

Reviewing files that changed from the base of the PR and between 7a52dbb and d4e0971.

📒 Files selected for processing (6)

• zstd/benches/compare_ffi.rs
• zstd/src/decoding/frame.rs
• zstd/src/decoding/frame_decoder.rs
• zstd/src/decoding/scratch.rs
• zstd/src/fse/fse_decoder.rs
• zstd/src/fse/fse_decoder/tests.rs

---

📝 Walkthrough

Walkthrough

Adds slice-based frame header parsing and slice-direct decoder reset paths, wires them into lsm decode-all flows, extends direct decoding with a single-RAW-block fast path, makes scratch resets conditional on populated tables, and removes a redundant benchmark arm.

Changes

Slice-direct decoding optimizations

Layer / File(s)	Summary
FSETableImpl::is_populated helper zstd/src/fse/fse_decoder.rs, zstd/src/fse/fse_decoder/tests.rs	Adds is_populated() based on decode_len != 0, and adds a regression test covering RLE-built table detection.
Conditional FSE and Huffman scratch reset zstd/src/decoding/scratch.rs	DecoderScratch::reset now skips FSE resets unless each table is populated and skips Huffman reset when max_num_bits is zero.
Slice-based frame header parser zstd/src/decoding/frame.rs	Adds read_frame_header_from_slice, which parses a frame header from &mut &[u8], advances the slice, and preserves the existing truncation errors.
FrameDecoderState parsed-header refactor zstd/src/decoding/frame_decoder.rs	new_with_format and reset_with_format now delegate to parsed-header helpers for shared decoder-state handling.
FrameDecoder slice reset and decode-all wiring zstd/src/decoding/frame_decoder.rs	Adds FrameDecoder::reset_from_slice and switches the lsm decode-all entry points to use it per frame.
Direct RAW-block fast path zstd/src/decoding/frame_decoder.rs	run_direct_decode now fast-copies a single RAW block when it matches content_size, then updates counters, checksum, and hash state.

Benchmark cleanup

Layer / File(s)	Summary
Dictionary benchmark arm removal zstd/benches/compare_ffi.rs	Removes the c_ffi_without_dict arm from the dictionary benchmark group and leaves comments noting the plain benchmark group covers the no-dictionary baseline.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• structured-world/structured-zstd#222: Adds related frame-header parsing and decoder initialization changes around the same magicless/header path.
• structured-world/structured-zstd#249: Touches frame_decoder.rs header/reset validation logic in the same decoder flow.
• structured-world/structured-zstd#394: Modifies the direct decode path in frame_decoder.rs around checksum and XXH64 handling.

Poem

🐇 I nibble bytes with slice and speed,
A RAW block zooms when that’s the need.
Populated tables stay awake,
My little paws take only what they take.
The benchmark trims a duplicate hop,
And this bunny grin won’t ever stop.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: reducing per-frame decode overhead on the in-memory path.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/bench-drop-redundant-nodict-dict-arm

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 88.12500% with 19 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
zstd/src/decoding/frame_decoder.rs	82.43%	13 Missing ⚠️
zstd/src/decoding/frame.rs	91.54%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR reduces fixed overhead in the in-memory decode path. The main changes are:

• Parse frame headers directly from input slices for decode_all and skippable-frame handling.
• Add a direct copy fast path for single RAW-block frames with declared content size.
• Skip redundant entropy-table resets when FSE and Huffman tables are already unbuilt.
• Remove the duplicate no-dictionary C FFI benchmark arm from the dictionary matrix.
• Add a test that confirms RLE-built FSE tables are detected as populated.

Confidence Score: 5/5

The changes appear merge-safe with no actionable correctness issues identified.

The implementation is narrowly scoped to the in-memory decode fast path and benchmark cleanup, with tests described for decode, interop, clippy, formatting, and the new FSE table-state behavior.

T-Rex Logs

What T-Rex did

• T-Rex ran the single-raw-fastpath fastpath test using the harness and compared before and after logs, confirming the head and single-frame digests remained the same.
• T-Rex executed the slice-header-parse test with the Rust harness and observed that the after results match the before in status and digests, while error messages are shown as UnexpectedEof variants.
• T-Rex ran the entropy-reset test suite and saw that decode_all_status, per-frame statuses, and digests match between before and after, with head RLE coverage indicating a built-RLE table was populated.
• T-Rex exercised the streaming-read harness and verified that all cases yield identical statuses and digests before and after, including skippable and truncation header handling; harness outputs were captured.
• T-Rex ran the bench-matrix tests and confirmed the head dictionary_compress_arms includes c_ffi_with_dict and pure_rust_with_dict, c_ffi_without_dict is false, and plain_compress_arms include c_ffi, with exit code 0.

_{Ran code and verified through T-Rex}

_{Reviews (2): Last reviewed commit: "fix(decode): detect RLE tables in is_pop..." | Re-trigger Greptile}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@zstd/src/decoding/frame_decoder.rs`:
- Around line 3053-3071: The fast path in `frame_decoder.rs` for single-block
terminal frames is incorrectly guarding checksum-byte consumption behind
`#[cfg(feature = "hash")]`, unlike the general `decode_blocks` and direct-loop
paths. Update the `content_checksum_flag()` handling in the
`state.frame_header.descriptor` branch so the 4 trailing checksum bytes are
always read, `bytes_read_counter` is always advanced, and `state.check_sum` is
always set; keep only the scratch hash seeding and `twox_hash::XxHash64` logic
behind the `hash` feature. This should mirror the existing checksum handling
used by `verify_content_checksum` / `get_calculated_checksum` and preserve
`is_finished()` behavior in no-`hash` builds.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d264ea14-7145-408e-a792-708dd7ed5f7f

📥 Commits

Reviewing files that changed from the base of the PR and between 01f5407 and 7a52dbb.

📒 Files selected for processing (5)

• zstd/benches/compare_ffi.rs
• zstd/src/decoding/frame.rs
• zstd/src/decoding/frame_decoder.rs
• zstd/src/decoding/scratch.rs
• zstd/src/fse/fse_decoder.rs