Upload the frame path's metadata in one transfer - #343
Merged
Conversation
DecodeAndAssemble allocated four device buffers for the per-block source pointers, source sizes, destination pointers and destination capacities, and filled them with four blocking H2D copies. The streaming path solved this already and its layout is the one adopted here: one region, [src_ptrs][src_sizes][dst_ptrs][dst_caps], four sections at a uniform stride, uploaded once. Counted inside the function rather than asserted: five H2D copies become two, of which exactly one is metadata and the other is the compressed-source staging that was already a single transfer. Seven ensure() calls become four, so three device allocations go with the three copies. Every section's stride is written in units of void*, including the two that hold size_t, which is what the streaming path does. A static_assert now says so out loud: on a host where size_t is narrower the two size sections would misalign silently, and nothing in the tree said that. The packed request is four strides rather than one, so it gets the overflow check the n * block_max product already gets, and rejects rather than wraps. No throughput claim. On a 3 MB frame at the 64 KB rung, the shortest wall this harness can put the change under, eight alternations of the two binaries give 13.46 ms against 13.80 ms on the medians with the packed build ahead in five of eight passes, inside a run-to-run spread of about 10%. The saving is a fixed few hundred microseconds per decode and this path cannot resolve it. What the change is for is the submission count, and that is counted above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Closes #134.
DecodeAndAssembleallocated four device metadata arrays and filled them withfour separate blocking
cudaMemcpycalls. They now live in one allocation asfour equal-stride sections and go up in one transfer, which is the layout
src/stream.cppalready uses (kMetaStride/kMetaBytesat lines 44-48, thesingle copy at 295-298). The frame path predates that work and never picked it
up.
No behavioural change. The same pointers, sizes and capacities reach
cudec_lz4_decompress_batchin the same order; only the number of submissionsmoves, from four to one.
The means is C++ inside the existing host translation unit, because the thing
being changed is the transfer shape of that function and the layout it copies
is three files away in the same language.
Type of change
Engineering checklist
n != 0guard and then * block_maxoverflow check are untouched. Packing multiplies the metadata request by
four, so that product gets the same treatment before the driver sees it.
frame_twindecodes against liblz4's frame API and isgreen, unchanged.
determinism_gpugreen.FRAME_CUDA, as before. No exceptions crossthe ABI.
disclosure at the end.
The new guard
n > SIZE_MAX / (4 * sizeof(void*))is stated as unproven ratherthan as proven. No input the frame parser admits can reach it:
ncountscompressed blocks in a frame already bounded by the caller's byte count, so a
value above
SIZE_MAX / 32cannot be constructed and no test makes it bite.It sits beside the identical unreachable guard on
n * block_maxthat wasalready there, for the same arithmetic reason, and both are here so the
multiplication is checked rather than assumed.
static_assert(sizeof(size_t) == sizeof(void*))carries the alignmentconstraint the issue asked for: every section gets a
void*-sized stride,including the two holding
size_t, so a host where the two differ is acompile error rather than a silent misalignment of the two size sections.
It is unproven for the same class of reason as the guard above and the reason
is worth being exact about, because it is not the reason it looks like.
Rewriting the stride expression does not trip it: the assertion compares two
sizeofresults and says nothing about how the stride is spelled. What tripsit is a host where the two types differ in width, and no compiler in this
gate is one. What it is worth is stated as what it is, a refusal that fires on
a port rather than a guard this suite can redden.
Review record
One reader, four lenses, over the whole diff. CONFIRMED 0 / PLAUSIBLE 2.
order the four casts read them back out, checked element by element against
the pre-image. Section offsets are multiples of 8 and the allocation base is
a
cudaMallocresult, so every section is naturally aligned. No finding.that cannot be shown to bite. Declined rather than removed because the
alternative is an unchecked multiplication.
memcpycalls into a stagingvector that it did not do before, and that vector is a fresh allocation per
decode. Disposition DECLINE, measured: the four host copies move at most
1248 bytes each at the largest block count in the sweep, against the 71 to 86
microseconds of submission cost removed. The numbers are below.
dd_resstays its own allocation because it is a device-to-host destinationand not metadata. No finding.
GPU sanitizer gate
.cufile, nokernel and no launch configuration is modified.
Separately, and so the empty block is not read as an answered one: #258 records
that no route to a device the sanitizer can attach to is available on this
machine. Under WSL2 the device is in WDDM mode, the debugger interface is
absent, and all four tools report the same two initialization errors on a
program with no fault in it.
Performance checklist
block path and the M2 batch path; it carries no frame-path baseline, so
there is no recorded number for this change to move.
Both sides were built and run in the pinned container against the local RTX
3080, one after the other:
RTX 3080 (sm_86), driver 12.6, runtime 12.6, host AMD Ryzen 9 5950X, liblz4
1.10.0, 3 warmup plus 30 measured runs, wall clock around the whole
synchronous
cudec_lz4f_decompresscall.p50 wall per run, before at
3fb2c55and after at this branch's head:Three rungs faster, three slower, and the signs do not follow the block count.
That is the honest reading: this sweep cannot resolve the change. Its own
within-run spread says so, p50 to p99 running from 2.0% to 14.8% across the
twelve report blocks, on one binary and one set of bytes, which is wider than
every delta in the table.
So the mechanism was measured where it is visible, on the same device, with a
program that does the two transfer shapes and nothing else. Source and
invocation, so the numbers can be rebuilt rather than trusted:
Three submissions, 71 to 86 microseconds, near enough flat in the block count
because it is per-call cost and not per-byte cost. Against the walls in the
first table that is 0.10% at the slowest rung and 0.71% at the fastest, which
is why the sweep swallows it.
One correction to the expectation written on the issue, which said the win is
largest as a fraction on few-block frames. On this harness it is the other way
round. Fewer blocks means less parallelism and a longer wall, so the 1 MB rungs
are the slowest and the fixed saving is the smallest fraction of them. The
fraction is largest on the many-block 64 KB rungs.
Quality checklist
void*and what the assertion is holding, not what the memcpy does.to lock.
Verification
Full gate in the pinned container against the local RTX 3080, at this
branch's head:
-Wall -Wextra -Werrorand the strict device flags.ctestgreen, 36 of 36, including the eight GPU-labelled entries(
frame_twin,stream_twin,determinism_gpu,termination_gpu,bench_frame_selfcheckamong them)..md,.ymlor.yamlfile is touched.The same gate was run at
3fb2c55for the before column, also 36 of 36.Notes
This change had no second reader. The review record above is one reader over
four lenses, and the pasted gate output and the two measurements are the
evidence in place of one.