Skip to content

Fix buffer offset corruption in runtime image decoders#95

Merged
Hexagon merged 5 commits intomainfrom
copilot/fix-pixel-offset-corruption
Mar 22, 2026
Merged

Fix buffer offset corruption in runtime image decoders#95
Hexagon merged 5 commits intomainfrom
copilot/fix-pixel-offset-corruption

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 17, 2026

Fix pixel-offset corruption after decoding/re-encoding

  • Identify root cause: improper buffer handling in runtime decoders
  • Fix JPEG format decoder
  • Fix WebP format decoder
  • Fix GIF format decoder
  • Fix TIFF format decoder
  • Fix HEIC format decoder
  • Fix AVIF format decoder
  • Create test case to validate the fix
  • Run precommit validation (all tests pass)
  • Request code review
  • Address review feedback: optimize buffer handling
  • Address review feedback: improve test robustness

Root Cause: When decoding images using runtime APIs (ImageDecoder/OffscreenCanvas), the code incorrectly created new Uint8Array(imageData.data.buffer) which assumed pixel data starts at offset 0 of the buffer. The imageData.data is a Uint8ClampedArray that may have a non-zero byteOffset, causing pixel data to be read from the wrong memory location, resulting in image corruption.

Fix: Changed to new Uint8Array(imageData.data.buffer, imageData.data.byteOffset, imageData.data.byteLength) which creates a proper view with correct offset handling without unnecessary copying.

Test Improvements:

  • Updated pixelDataSimilar to allow up to 5% of bytes to exceed tolerance (avoiding flakiness across different JPEG encoders)
  • Added specific pixel position checks (corners, center) to detect offset/stride issues

Testing: All 552+ existing tests pass plus new roundtrip corruption tests.

Original prompt

This section details on the original issue you should resolve

<issue_title>Instances of pixel-offset corruption after decoding/re-encoding</issue_title>
<issue_description>Hello! Thank you for this very useful tool.

I've discovered a couple of instances of image distortion when simply decoding and then re-encoding in the same format. I've only tested in Deno (v2.6.4).

To reproduce, simply drop the attached files into a directory and then run the following script with all permissions:

import { Image } from "@cross/image";
import * as path from "@std/path";

const aBytes = await Deno.readFile(path.join(Deno.cwd(), "A.jpg"));
const aImg = await Image.decode(aBytes, "jpeg");
const aImgPrime = await aImg.encode("jpeg");
await Deno.writeFile(path.join(Deno.cwd(), "aPrime.jpg"), aImgPrime);

const bBytes = await Deno.readFile(path.join(Deno.cwd(), "B.png"));
const bImg = await Image.decode(bBytes, "png");
const bImgPrime = await bImg.encode("png");
await Deno.writeFile(path.join(Deno.cwd(), "bPrime.png"), bImgPrime);
Image

Image

Really hoping you can help ...</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits January 17, 2026 00:25
Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com>
Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix instances of pixel-offset corruption after decoding and re-encoding Fix buffer offset corruption in runtime image decoders Jan 17, 2026
Copilot AI requested a review from Hexagon January 17, 2026 00:29
@Hexagon Hexagon marked this pull request as ready for review March 21, 2026 20:54
@Hexagon Hexagon requested a review from Copilot March 21, 2026 20:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes pixel corruption that occurred when runtime decoders (ImageDecoder/OffscreenCanvas) produced ImageData.data views with non-zero byteOffset, and the code incorrectly built RGBA arrays from imageData.data.buffer starting at offset 0.

Changes:

  • Updated multiple runtime decoder paths to build RGBA buffers from imageData.data rather than imageData.data.buffer.
  • Added a new roundtrip regression test suite covering JPEG/PNG and repeated JPEG roundtrips.
  • Documented the fix in the Unreleased changelog.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
test/roundtrip_corruption.test.ts Adds regression tests intended to detect offset/scramble corruption via roundtrip encode/decode.
src/formats/jpeg.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
src/formats/webp.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
src/formats/gif.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
src/formats/tiff.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
src/formats/heic.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
src/formats/avif.ts Fixes runtime decode RGBA extraction to respect non-zero TypedArray offsets.
CHANGELOG.md Records the runtime decoder pixel-offset corruption fix.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +22 to +33
// Helper to compare pixel data with tolerance for lossy formats
function pixelDataSimilar(
data1: Uint8Array,
data2: Uint8Array,
tolerance = 5,
): boolean {
if (data1.length !== data2.length) return false;
for (let i = 0; i < data1.length; i++) {
if (Math.abs(data1[i] - data2[i]) > tolerance) {
return false;
}
}
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pixelDataSimilar() requires every byte in the decoded JPEG to be within tolerance, which is much stricter than the repo’s existing JPEG roundtrip checks (e.g., allowing a percentage of pixels to differ). This is likely to be flaky across runtimes/encoders. Consider switching to a comparison that tolerates some outliers (percentage-based like imagesApproximatelyEqual in test/purejs_roundtrip.test.ts) or use an aggregate error metric (MAE/PSNR) instead of a per-byte all-or-nothing check.

Suggested change
// Helper to compare pixel data with tolerance for lossy formats
function pixelDataSimilar(
data1: Uint8Array,
data2: Uint8Array,
tolerance = 5,
): boolean {
if (data1.length !== data2.length) return false;
for (let i = 0; i < data1.length; i++) {
if (Math.abs(data1[i] - data2[i]) > tolerance) {
return false;
}
}
// Helper to compare pixel data with tolerance for lossy formats.
// Instead of requiring every byte to be within tolerance (which is too strict
// for JPEG and can be flaky across runtimes/encoders), we allow a small
// fraction of bytes to exceed the tolerance while still considering the
// images "similar". This is sufficient to catch structural corruption while
// avoiding over-sensitivity to encoder differences.
function pixelDataSimilar(
data1: Uint8Array,
data2: Uint8Array,
tolerance = 5,
): boolean {
if (data1.length !== data2.length) return false;
// Allow up to 5% of bytes to exceed the tolerance.
const maxOutlierRatio = 0.05;
let outlierCount = 0;
for (let i = 0; i < data1.length; i++) {
if (Math.abs(data1[i] - data2[i]) > tolerance) {
outlierCount++;
if (outlierCount / data1.length > maxOutlierRatio) {
return false;
}
}
}

Copilot uses AI. Check for mistakes.
Math.abs(avgBlue - 192) < 50,
true,
`Average blue should be close to 192, got ${avgBlue}`,
);
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Multiple roundtrips" assertion uses only per-channel averages, which is largely insensitive to byte-order/offset corruption (a scrambled buffer can keep similar averages). To make this test actually catch offset/stride issues, consider also validating a handful of specific pixel positions (e.g., corners/center) or using a simple checksum/histogram that is sensitive to reordering, in addition to the averages.

Suggested change
);
);
// Additionally verify a few specific pixel positions to catch offset/stride issues.
// Because JPEG is lossy, allow a generous per-channel tolerance.
const channelTolerance = 40;
const sampleCoords: Array<[number, number]> = [
[0, 0], // top-left
[width - 1, 0], // top-right
[0, height - 1], // bottom-left
[width - 1, height - 1], // bottom-right
[Math.floor(width / 2), Math.floor(height / 2)], // center
];
for (const [x, y] of sampleCoords) {
const idx = (y * width + x) * 4;
const r = finalData[idx];
const g = finalData[idx + 1];
const b = finalData[idx + 2];
assertEquals(
Math.abs(r - 128) <= channelTolerance,
true,
`Pixel at (${x}, ${y}) has unexpected red channel: ${r}`,
);
assertEquals(
Math.abs(g - 64) <= channelTolerance,
true,
`Pixel at (${x}, ${y}) has unexpected green channel: ${g}`,
);
assertEquals(
Math.abs(b - 192) <= channelTolerance,
true,
`Pixel at (${x}, ${y}) has unexpected blue channel: ${b}`,
);
}

Copilot uses AI. Check for mistakes.
bitmap.close();

return new Uint8Array(imageData.data.buffer);
return new Uint8Array(imageData.data);
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new Uint8Array(imageData.data) fixes the byteOffset issue, but it also copies the entire pixel buffer. For large images this can be a significant extra allocation and CPU cost during decode. Consider returning a view that respects the offset/length instead (e.g., construct Uint8Array from imageData.data.buffer with byteOffset/byteLength) to avoid the copy while still fixing corruption.

Suggested change
return new Uint8Array(imageData.data);
return new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
);

Copilot uses AI. Check for mistakes.
bitmap.close();

return new Uint8Array(imageData.data.buffer);
return new Uint8Array(imageData.data);
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runtime decode path now copies pixel data via new Uint8Array(imageData.data). That avoids offset corruption, but it introduces an O(n) copy and extra memory. Prefer creating a Uint8Array view over imageData.data.buffer using byteOffset and byteLength so the returned array is the correct window without copying.

Suggested change
return new Uint8Array(imageData.data);
return new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
);

Copilot uses AI. Check for mistakes.
bitmap.close();

return new Uint8Array(imageData.data.buffer);
return new Uint8Array(imageData.data);
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new Uint8Array(imageData.data) copies the pixel data. To avoid an extra allocation on every runtime decode, consider creating a Uint8Array view with imageData.data.buffer + byteOffset + byteLength instead; it still fixes the offset bug but is cheaper.

Suggested change
return new Uint8Array(imageData.data);
return new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
);

Copilot uses AI. Check for mistakes.
bitmap.close();

return new Uint8Array(imageData.data.buffer);
return new Uint8Array(imageData.data);
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change fixes the offset bug but now copies all decoded pixels (new Uint8Array(imageData.data)). For large TIFFs this may be expensive; you can keep correctness without copying by creating a Uint8Array view over imageData.data.buffer using byteOffset and byteLength.

Suggested change
return new Uint8Array(imageData.data);
return new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
);

Copilot uses AI. Check for mistakes.
width: canvas.width,
height: canvas.height,
rgba: new Uint8Array(imageData.data.buffer),
rgba: new Uint8Array(imageData.data),
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new rgba: new Uint8Array(imageData.data) copies the full pixel buffer. If the intent is only to respect non-zero byteOffset, consider returning a Uint8Array view using imageData.data.buffer + byteOffset/byteLength to avoid an extra allocation during HEIC runtime decode.

Suggested change
rgba: new Uint8Array(imageData.data),
rgba: new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
),

Copilot uses AI. Check for mistakes.
width: canvas.width,
height: canvas.height,
rgba: new Uint8Array(imageData.data.buffer),
rgba: new Uint8Array(imageData.data),
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now copies decoded pixels via new Uint8Array(imageData.data). If you don't specifically need a copy, a Uint8Array view over imageData.data.buffer with byteOffset/byteLength fixes the corruption issue while avoiding the extra allocation for large AVIF images.

Suggested change
rgba: new Uint8Array(imageData.data),
rgba: new Uint8Array(
imageData.data.buffer,
imageData.data.byteOffset,
imageData.data.byteLength,
),

Copilot uses AI. Check for mistakes.
@Hexagon
Copy link
Copy Markdown
Member

Hexagon commented Mar 21, 2026

@copilot apply changes based on the comments in this thread

- Use Uint8Array view (with byteOffset/byteLength) instead of copy for performance
- Update pixelDataSimilar to allow 5% outlier ratio for JPEG tolerance
- Add specific pixel position checks to detect offset/stride corruption

Co-authored-by: Hexagon <419737+Hexagon@users.noreply.github.com>
Agent-Logs-Url: https://github.com/cross-org/image/sessions/a90dded2-ed43-46e9-88c6-c8a04f8aa7e8
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 21, 2026

@copilot apply changes based on the comments in this thread

Applied all suggested changes in commit d332e4d:

  1. Performance optimization: All format decoders now use new Uint8Array(imageData.data.buffer, imageData.data.byteOffset, imageData.data.byteLength) to create views instead of copying pixel data
  2. Test robustness: Updated pixelDataSimilar to allow up to 5% outlier ratio, avoiding flakiness across different JPEG encoders
  3. Offset detection: Added specific pixel position checks (corners and center) to the multiple roundtrips test to catch offset/stride issues

All 96 format-specific tests and 3 roundtrip tests pass.

@Hexagon Hexagon merged commit d389d1d into main Mar 22, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Instances of pixel-offset corruption after decoding/re-encoding

3 participants