Skip to content

Add HEIF/HEIC and AVIF metadata support (EXIF, XMP, CONFIG) - #58

Merged
bep merged 3 commits into
bep:mainfrom
tonimelisma:feat/heif-avif
Feb 22, 2026
Merged

Add HEIF/HEIC and AVIF metadata support (EXIF, XMP, CONFIG)#58
bep merged 3 commits into
bep:mainfrom
tonimelisma:feat/heif-avif

Conversation

@tonimelisma

Copy link
Copy Markdown
Contributor

Closes #55.

Adds metadata decoding support for HEIF/HEIC and AVIF images. Both formats use the
ISO Base Media File Format (ISOBMFF) container. Metadata extraction only requires
parsing the container's box structure — no pixel decoding, no new dependencies.

Supported sources

  • EXIF — full EXIF/MakerNote decoding via the existing newMetaDecoderEXIF pipeline
  • XMP — raw XMP blob passed to the existing decodeXMP pipeline
  • CONFIG — image dimensions from ispe via pitm/ipma + rotation applied from irot
  • IPTC — not part of the HEIF/AVIF spec; not supported

Implementation

One new file imagedecoder_heif.go implementing the existing decoder interface.
ISOBMFF box types are defined as package-level variables, consistent with the fourCC
and []byte tag constants used in the WebP and PNG decoders. Item IDs are tracked as
uint32 throughout, supporting both infe version 2 (16-bit) and version 3 (32-bit)
without truncation.

The ISOBMFF parsing flow:

  1. Validate ftyp box (caller is responsible for the format hint — HEIF or AVIF)
  2. Scan top-level boxes for meta
  3. Parse meta children:
    • pitm → primary item ID (used for CONFIG dimension selection)
    • iinf → find item IDs for EXIF (itemType = "Exif") and XMP (itemType = "mime");
      both infe version 2 (16-bit item IDs) and version 3 (32-bit item IDs) are handled
    • iloc → map those item IDs to absolute file offsets (baseOffset + extent offset)
      and lengths; iloc versions 0, 1, and 2 with variable field sizes (0/2/4/8 bytes);
      items using non-file-offset construction methods (idat/item) are skipped
    • iprpipco collects all properties by index; ipma maps item IDs to
      property indices; ispe and irot are extracted for CONFIG
  4. Seek to iloc offsets, extract EXIF and XMP blobs, hand to existing decoders

EXIF blobs in HEIF have a 4-byte big-endian prefix indicating bytes to skip before
the TIFF header. The TIFF byte-order mark ("MM"/"II") is then auto-detected by
newMetaDecoderEXIF as usual. EXIF decoding is wrapped in a defer/recover handler
(matching the JPEG decoder's handleEXIF pattern) so that XMP decoding can proceed
even if EXIF parsing panics.

Image dimensions use a two-tier strategy. The primary path resolves the pitm
item ID through ipma associations to find the exact ispe and irot properties
belonging to the primary image — this is the spec-correct approach and avoids
selecting tile or thumbnail dimensions. When pitm or ipma data is absent (e.g.
older encoders), the decoder falls back to selecting the ispe entry with the
largest pixel area, which is reliable in practice since the main image is always
larger than tiles or thumbnails. irot angle 1 or 3 (90°/270° rotation) swaps
width and height to give display-oriented dimensions — tested against both an
iPhone 15 Pro (no rotation) and a Sony A6700 (irot angle=3, swaps 6192×4128 to
4128×6192).

Buffered readers for EXIF and XMP are scoped via anonymous functions / handleEXIF
to ensure prompt Close() calls that return pooled buffers, following the pattern
established by the WebP and PNG decoders.

source_string.go is regenerated — this also picks up the pre-existing CONFIG
source constant that was missing from the Source stringer.

Test images

Both images are provided under CC BY-SA 4.0 (see testdata/images/license.txt):

  • testdata/images/iphone.heic — iPhone 15 Pro, big-endian EXIF, ftyp brand heic,
    5712×4284 px, includes GPS with fractional seconds
  • testdata/images/sony.heif — Sony ILCE-6700, little-endian EXIF, ftyp brand heix,
    6192×4128 coded / 4128×6192 display (portrait via irot angle=3)

Golden files generated with ExifTool 13.50 (go generate ./gen). Both images are
included in the non-skipped golden tests (TestGoldenEXIF, TestGoldenConfig, etc.).

Tests

  • TestDecodeHEIF — EXIF tag assertions for both test images (Make, Model, Orientation)
  • TestDecodeHEIFConfig — image dimension assertions for both test images, including
    rotation swap verification for the Sony image
  • TestGoldenEXIF, TestGoldenConfig — full golden file comparison against ExifTool
  • FuzzDecodeHEIF, FuzzDecodeAVIF — fuzz tests seeded with the test images
  • go test -race ./... — all pass
  • go vet ./... — clean
  • gofmt -d . — no diff

Prerequisites

This PR includes the two bug fixes from the companion PRs
(fix/gps-timestamp-fractional-seconds and fix/test-infra-numeric-normalization).
If those land first, this PR can be rebased to drop the duplicate changes.

tonimelisma and others added 2 commits February 21, 2026 11:02
The convertToTimestampString converter used ratNum() (numerator only) for
all three time components. It had a follow-up hack that detected 10-char
results and inserted a decimal point:

    4279/100 → numerator 4279 → "13:03:4279" (10 chars) → "13:03:42.79"

This only works when the seconds numerator is exactly 4 digits. A rational
like 181/5 (= 36.2 seconds) has a 3-digit numerator and falls through:

    181/5 → numerator 181 → "22:22:181" → no decimal inserted

Fix: use toFloat64() and strconv.FormatFloat for the seconds component.
Integer seconds still produce "47"; fractional seconds produce "36.2".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two test infrastructure fixes:

1. The fuzzy float comparator regex for space-delimited fields (LensInfo,
PrimaryChromaticities, etc.) required both the first and second token to
contain a decimal point. Fields with integer tokens (e.g. LensInfo
"2.22 9 1.78 2.8" where 9 is the max focal length) fell through to exact
string comparison, failing on float64 precision differences.

Fix: allow subsequent tokens to be integers.

2. ExifTool with -n outputs numeric-looking software versions (e.g. iOS
"26.3") as JSON numbers rather than strings. After json.Unmarshal this
becomes float64(26.3) while imagemeta correctly returns string("26.3")
since EXIF Software is an ASCII field.

Fix: convert float64 back to string for Software in normalizeThem.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Feb 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.12544% with 80 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.19%. Comparing base (93d0b00) to head (b914d35).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
imagedecoder_heif.go 71.21% 49 Missing and 29 partials ⚠️
source_string.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #58      +/-   ##
==========================================
- Coverage   79.38%   78.19%   -1.20%     
==========================================
  Files          14       15       +1     
  Lines        1606     1885     +279     
==========================================
+ Hits         1275     1474     +199     
- Misses        228      279      +51     
- Partials      103      132      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread imagedecoder_heif.go Outdated

// ISOBMFF box types used in HEIF/AVIF containers.
var (
boxTypeFtyp = "ftyp"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a fourCC type used elsewhere for the same purpose. I would prefer to use this. That also avoid the string conversions when comparing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — switched to fourCC type with fcc prefix naming, matching the WebP decoder pattern. The readBox helper now returns fourCC directly (reads into the named return value), eliminating the intermediate [4]byte and string() conversion. Same for infe item type comparison.

Comment thread imagedecoder_heif.go Outdated
// than tiles or thumbnails in standard HEIF/AVIF output).
for _, p := range ipcoProps {
if p.isIspe && p.width > 0 && p.height > 0 {
if int(p.width)*int(p.height) > int(cfgWidth)*int(cfgHeight) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above looks like it could overflow; are those int conversions really needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — on 32-bit platforms int is 32 bits, so int(uint32) * int(uint32) can overflow. Changed to uint64 multiplication, which can hold the full range (uint32 × uint32 fits in uint64).

Closes #55.

Add metadata decoding for HEIF/HEIC and AVIF images (ISOBMFF containers).
Supports EXIF, XMP, and CONFIG sources. No pixel decoding, no new dependencies.
@bep
bep merged commit 0985245 into bep:main Feb 22, 2026
6 checks passed
@tonimelisma
tonimelisma deleted the feat/heif-avif branch February 22, 2026 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add HEIF/HEIC and AVIF metadata support

3 participants