Fix 32-bit byte accounting in Buffer - #105
Merged
Merged
Conversation
The byte counts in buffer.c were 32-bit, and the two I/O helpers accumulated
into int. Three consequences, none of which corrupt the heap -- the memcpy
bounds are computed per node and stay within it -- but all of which make the
buffer misreport or mishandle its own contents:
* struct buffer's size and node_size, and buffer_node's start and end, were
unsigned. A buffer holding more than 4 GiB wrapped its accounting, so
Buffer#size and the clamp in Buffer#read no longer described what the
buffer actually held.
* buffer_read_from() and buffer_write_to() accumulated into int. A single
call transferring more than 2 GiB overflowed a signed integer, which is
undefined behaviour, and returned a meaningless count.
* Buffer#read with no argument assigned buf->size to an int length. Past
INT_MAX that goes negative, and rb_str_new(0, length) is then handed a
negative length.
Carry the counts in size_t and the transfer totals in ssize_t throughout.
Buffer#size and Buffer.default_node_size now convert with SIZET2NUM, which is
the same Integer for every value they could return before.
Buffer#read reads its argument as long rather than int, so a length that does
not fit in an int is clamped to the buffer size like any other oversized
length instead of raising RangeError out of the conversion. A negative or zero
length still raises ArgumentError from the same check as before.
Not addressed here: the buffer still has no total-size limit, so it grows to
whatever the application feeds it. A cap belongs behind an opt-in setting
rather than a silent default, which is a separate change.
Removes three -Wsign-compare warnings in buffer.c.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The byte counts in
buffer.care 32-bit, and the two I/O helpers accumulate intoint. None of this corrupts the heap — thememcpybounds are computed per node and stay inside it — but each case makes the buffer misreport or mishandle its own contents.struct buffer'ssize/node_sizeandbuffer_node'sstart/endareunsigned. A buffer holding more than 4 GiB wraps its accounting, after whichBuffer#sizeand the clamp insideBuffer#readno longer describe what the buffer actually holds.buffer_read_from()andbuffer_write_to()accumulate intoint. A single call transferring more than 2 GiB overflows a signed integer — undefined behaviour — and returns a meaningless count.Buffer#readwith no argument assignsbuf->sizeto anint length. PastINT_MAXthat goes negative, andrb_str_new(0, length)is then handed a negative length.Fix
Carry the counts in
size_tand the transfer totals inssize_tthroughout.Buffer#sizeandBuffer.default_node_sizeconvert withSIZET2NUM, which yields the same Integer for every value they could return before.Buffer#readreads its argument withNUM2LONGrather thanNUM2INT, so the existinglength < 1check still raisesArgumentError— notRangeErrorout of the conversion — for zero and negative lengths.Behaviour change worth naming
A length that does not fit in a C
int(e.g.buffer.read(2**31)) used to raiseRangeErrorfromNUM2INT. It is now clamped to the buffer size, like any other oversized length. That is the point of the change, but it is a visible difference for a caller who relied on the exception.Everything else is unchanged:
MAX_SIZEstill reports the 1 GiB node-size cap,Buffer#read_fromstill returnsnilat EOF, and no method changes its return type.Not addressed here
The buffer still has no total-size limit — it grows to whatever the application feeds it, and an application that never drains a slow peer can be pushed into OOM. A cap belongs behind an opt-in setting rather than a silent default, so it is left for a separate change.
Tests
spec/iobuffer_spec.rbgains three examples:ArgumentErrorfor a length below one, the clamp for a length that does not fit in anint, and the resulting buffer state. The clamp example fails on the parent commit (RangeError) and passes with this change.rspec— 67 examples, 0 failures.buffer.cgo from 6 to 3; the three that disappear are-Wsign-compare. The remaining three (unsigned char *signedness inbuffer_read_frame, unusedtmp) are pre-existing and untouched here.🤖 Generated with Claude Code