Fix buffer offset corruption in runtime image decoders#95
Conversation
Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com>
Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes pixel corruption that occurred when runtime decoders (ImageDecoder/OffscreenCanvas) produced ImageData.data views with non-zero byteOffset, and the code incorrectly built RGBA arrays from imageData.data.buffer starting at offset 0.
Changes:
- Updated multiple runtime decoder paths to build RGBA buffers from
imageData.datarather thanimageData.data.buffer. - Added a new roundtrip regression test suite covering JPEG/PNG and repeated JPEG roundtrips.
- Documented the fix in the Unreleased changelog.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| test/roundtrip_corruption.test.ts | Adds regression tests intended to detect offset/scramble corruption via roundtrip encode/decode. |
| src/formats/jpeg.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| src/formats/webp.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| src/formats/gif.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| src/formats/tiff.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| src/formats/heic.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| src/formats/avif.ts | Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets. |
| CHANGELOG.md | Records the runtime decoder pixel-offset corruption fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
test/roundtrip_corruption.test.ts
Outdated
| // Helper to compare pixel data with tolerance for lossy formats | ||
| function pixelDataSimilar( | ||
| data1: Uint8Array, | ||
| data2: Uint8Array, | ||
| tolerance = 5, | ||
| ): boolean { | ||
| if (data1.length !== data2.length) return false; | ||
| for (let i = 0; i < data1.length; i++) { | ||
| if (Math.abs(data1[i] - data2[i]) > tolerance) { | ||
| return false; | ||
| } | ||
| } |
There was a problem hiding this comment.
pixelDataSimilar() requires every byte in the decoded JPEG to be within tolerance, which is much stricter than the repo’s existing JPEG roundtrip checks (e.g., allowing a percentage of pixels to differ). This is likely to be flaky across runtimes/encoders. Consider switching to a comparison that tolerates some outliers (percentage-based like imagesApproximatelyEqual in test/purejs_roundtrip.test.ts) or use an aggregate error metric (MAE/PSNR) instead of a per-byte all-or-nothing check.
| // Helper to compare pixel data with tolerance for lossy formats | |
| function pixelDataSimilar( | |
| data1: Uint8Array, | |
| data2: Uint8Array, | |
| tolerance = 5, | |
| ): boolean { | |
| if (data1.length !== data2.length) return false; | |
| for (let i = 0; i < data1.length; i++) { | |
| if (Math.abs(data1[i] - data2[i]) > tolerance) { | |
| return false; | |
| } | |
| } | |
| // Helper to compare pixel data with tolerance for lossy formats. | |
| // Instead of requiring every byte to be within tolerance (which is too strict | |
| // for JPEG and can be flaky across runtimes/encoders), we allow a small | |
| // fraction of bytes to exceed the tolerance while still considering the | |
| // images "similar". This is sufficient to catch structural corruption while | |
| // avoiding over-sensitivity to encoder differences. | |
| function pixelDataSimilar( | |
| data1: Uint8Array, | |
| data2: Uint8Array, | |
| tolerance = 5, | |
| ): boolean { | |
| if (data1.length !== data2.length) return false; | |
| // Allow up to 5% of bytes to exceed the tolerance. | |
| const maxOutlierRatio = 0.05; | |
| let outlierCount = 0; | |
| for (let i = 0; i < data1.length; i++) { | |
| if (Math.abs(data1[i] - data2[i]) > tolerance) { | |
| outlierCount++; | |
| if (outlierCount / data1.length > maxOutlierRatio) { | |
| return false; | |
| } | |
| } | |
| } |
| Math.abs(avgBlue - 192) < 50, | ||
| true, | ||
| `Average blue should be close to 192, got ${avgBlue}`, | ||
| ); |
There was a problem hiding this comment.
The "Multiple roundtrips" assertion uses only per-channel averages, which is largely insensitive to byte-order/offset corruption (a scrambled buffer can keep similar averages). To make this test actually catch offset/stride issues, consider also validating a handful of specific pixel positions (e.g., corners/center) or using a simple checksum/histogram that is sensitive to reordering, in addition to the averages.
| ); | |
| ); | |
| // Additionally verify a few specific pixel positions to catch offset/stride issues. | |
| // Because JPEG is lossy, allow a generous per-channel tolerance. | |
| const channelTolerance = 40; | |
| const sampleCoords: Array<[number, number]> = [ | |
| [0, 0], // top-left | |
| [width - 1, 0], // top-right | |
| [0, height - 1], // bottom-left | |
| [width - 1, height - 1], // bottom-right | |
| [Math.floor(width / 2), Math.floor(height / 2)], // center | |
| ]; | |
| for (const [x, y] of sampleCoords) { | |
| const idx = (y * width + x) * 4; | |
| const r = finalData[idx]; | |
| const g = finalData[idx + 1]; | |
| const b = finalData[idx + 2]; | |
| assertEquals( | |
| Math.abs(r - 128) <= channelTolerance, | |
| true, | |
| `Pixel at (${x}, ${y}) has unexpected red channel: ${r}`, | |
| ); | |
| assertEquals( | |
| Math.abs(g - 64) <= channelTolerance, | |
| true, | |
| `Pixel at (${x}, ${y}) has unexpected green channel: ${g}`, | |
| ); | |
| assertEquals( | |
| Math.abs(b - 192) <= channelTolerance, | |
| true, | |
| `Pixel at (${x}, ${y}) has unexpected blue channel: ${b}`, | |
| ); | |
| } |
src/formats/jpeg.ts
Outdated
| bitmap.close(); | ||
|
|
||
| return new Uint8Array(imageData.data.buffer); | ||
| return new Uint8Array(imageData.data); |
There was a problem hiding this comment.
new Uint8Array(imageData.data) fixes the byteOffset issue, but it also copies the entire pixel buffer. For large images this can be a significant extra allocation and CPU cost during decode. Consider returning a view that respects the offset/length instead (e.g., construct Uint8Array from imageData.data.buffer with byteOffset/byteLength) to avoid the copy while still fixing corruption.
| return new Uint8Array(imageData.data); | |
| return new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ); |
src/formats/webp.ts
Outdated
| bitmap.close(); | ||
|
|
||
| return new Uint8Array(imageData.data.buffer); | ||
| return new Uint8Array(imageData.data); |
There was a problem hiding this comment.
This runtime decode path now copies pixel data via new Uint8Array(imageData.data). That avoids offset corruption, but it introduces an O(n) copy and extra memory. Prefer creating a Uint8Array view over imageData.data.buffer using byteOffset and byteLength so the returned array is the correct window without copying.
| return new Uint8Array(imageData.data); | |
| return new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ); |
src/formats/gif.ts
Outdated
| bitmap.close(); | ||
|
|
||
| return new Uint8Array(imageData.data.buffer); | ||
| return new Uint8Array(imageData.data); |
There was a problem hiding this comment.
new Uint8Array(imageData.data) copies the pixel data. To avoid an extra allocation on every runtime decode, consider creating a Uint8Array view with imageData.data.buffer + byteOffset + byteLength instead; it still fixes the offset bug but is cheaper.
| return new Uint8Array(imageData.data); | |
| return new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ); |
src/formats/tiff.ts
Outdated
| bitmap.close(); | ||
|
|
||
| return new Uint8Array(imageData.data.buffer); | ||
| return new Uint8Array(imageData.data); |
There was a problem hiding this comment.
This change fixes the offset bug but now copies all decoded pixels (new Uint8Array(imageData.data)). For large TIFFs this may be expensive; you can keep correctness without copying by creating a Uint8Array view over imageData.data.buffer using byteOffset and byteLength.
| return new Uint8Array(imageData.data); | |
| return new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ); |
src/formats/heic.ts
Outdated
| width: canvas.width, | ||
| height: canvas.height, | ||
| rgba: new Uint8Array(imageData.data.buffer), | ||
| rgba: new Uint8Array(imageData.data), |
There was a problem hiding this comment.
The new rgba: new Uint8Array(imageData.data) copies the full pixel buffer. If the intent is only to respect non-zero byteOffset, consider returning a Uint8Array view using imageData.data.buffer + byteOffset/byteLength to avoid an extra allocation during HEIC runtime decode.
| rgba: new Uint8Array(imageData.data), | |
| rgba: new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ), |
src/formats/avif.ts
Outdated
| width: canvas.width, | ||
| height: canvas.height, | ||
| rgba: new Uint8Array(imageData.data.buffer), | ||
| rgba: new Uint8Array(imageData.data), |
There was a problem hiding this comment.
This now copies decoded pixels via new Uint8Array(imageData.data). If you don't specifically need a copy, a Uint8Array view over imageData.data.buffer with byteOffset/byteLength fixes the corruption issue while avoiding the extra allocation for large AVIF images.
| rgba: new Uint8Array(imageData.data), | |
| rgba: new Uint8Array( | |
| imageData.data.buffer, | |
| imageData.data.byteOffset, | |
| imageData.data.byteLength, | |
| ), |
|
@copilot apply changes based on the comments in this thread |
- Use Uint8Array view (with byteOffset/byteLength) instead of copy for performance - Update pixelDataSimilar to allow 5% outlier ratio for JPEG tolerance - Add specific pixel position checks to detect offset/stride corruption Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com> Agent-Logs-Url: https://github.com/cross-org/image/sessions/a90dded2-ed43-46e9-88c6-c8a04f8aa7e8
Applied all suggested changes in commit d332e4d:
All 96 format-specific tests and 3 roundtrip tests pass. |
Fix pixel-offset corruption after decoding/re-encoding
Root Cause: When decoding images using runtime APIs (ImageDecoder/OffscreenCanvas), the code incorrectly created
new Uint8Array(imageData.data.buffer)which assumed pixel data starts at offset 0 of the buffer. TheimageData.datais aUint8ClampedArraythat may have a non-zerobyteOffset, causing pixel data to be read from the wrong memory location, resulting in image corruption.Fix: Changed to
new Uint8Array(imageData.data.buffer, imageData.data.byteOffset, imageData.data.byteLength)which creates a proper view with correct offset handling without unnecessary copying.Test Improvements:
pixelDataSimilarto allow up to 5% of bytes to exceed tolerance (avoiding flakiness across different JPEG encoders)Testing: All 552+ existing tests pass plus new roundtrip corruption tests.
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.