Skip to content

8.2_Bit_Depth_Chroma_Compression.md

Latest commit

 

History

History
282 lines (217 loc) · 8.54 KB

8.2_Bit_Depth_Chroma_Compression.md

File metadata and controls

282 lines (217 loc) · 8.54 KB

8.2 Bit-Depth, Chroma Subsampling, and Compression

Theory

Bit Depth Fundamentals

[Bit depth] refers to the number of bits used to represent each color component (red, green, blue) of a pixel:

Mathematical Foundation:

  • 8-bit: 2^8 = 256 possible values per channel (16.7 million colors)
  • 10-bit: 2^10 = 1,024 possible values per channel (1.07 billion colors)
  • 12-bit: 2^12 = 4,096 possible values per channel (68.7 billion colors)
  • 16-bit: 2^16 = 65,536 possible values per channel (281 trillion colors)

Perceptual Impact:

  • Higher bit depth = smoother gradations
  • Reduces banding in gradients (skies, shadows, skin tones)
  • Increased color precision
  • More headroom for color grading
  • Better HDR performance

Dynamic Range Relationship:

  • Bit depth and dynamic range are related but distinct
  • Higher bit depth enables capture of wider dynamic range
  • Log encoding uses bit depth efficiently for wide dynamic range
  • HDR requires minimum 10-bit, preferably 12-bit

Chroma Subsampling Principles

[Chroma subsampling] reduces color information while preserving luminance, based on human visual system's greater sensitivity to brightness than color:

J:a:b Notation Explained:

  • J: Width of reference pixel block (typically 4)
  • a: Number of chroma samples in first row
  • b: Number of additional chroma samples in second row

Common Schemes:

4:4:4 (No Subsampling)

  • Full color resolution
  • Each pixel has complete RGB information
  • Highest quality, largest file size
  • Required for: VFX work, chroma keying, high-end grading

4:2:2 (Horizontal Subsampling)

  • Horizontal color resolution halved
  • Vertical color resolution maintained
  • Industry standard for professional production
  • Good balance of quality and file size
  • Used in: ProRes 422, DNxHR, XAVC-I

4:2:0 (Horizontal and Vertical Subsampling)

  • Both horizontal and vertical color resolution halved
  • For every 2x2 pixel block, only 1 chroma sample
  • Highest compression, lowest quality
  • Standard for: Web video, streaming, broadcast
  • Used in: H.264, H.265, AV1

4:1:1 (Legacy Format)

  • Older DV format
  • Severe subsampling, largely obsolete
  • Mentioned for historical context only

Compression Ratio Calculations

[Compression ratio] compares original uncompressed size to compressed size:

Calculations:

  • Uncompressed 1080p 10-bit 4:2:2: ~995 Mbps
  • ProRes 422 (147 Mbps): 995/147 = 6.8:1 compression
  • H.264 high quality (20 Mbps): 995/20 = 50:1 compression
  • H.265 high quality (10 Mbps): 995/10 = 100:1 compression

Quality Implications:

  • <5:1: Visually lossless for most content
  • 5:1 to 10:1: Excellent quality, suitable for mastering
  • 10:1 to 20:1: Good quality, suitable for distribution
  • 20:1 to 50:1: Acceptable quality for web streaming

  • 50:1: Noticeable artifacts likely

Practice

Bit Depth Selection

When to Use Each Bit Depth:

8-bit:

  • Consumer and web video delivery
  • Simple graphics and animation
  • Non-critical monitoring
  • Bandwidth-constrained streaming
  • Avoid for: Grading, VFX, HDR, fine gradients

10-bit:

  • Standard professional production
  • Color grading and color correction
  • HDR content (minimum requirement)
  • High-quality streaming
  • Good balance of quality and file size

12-bit:

  • High-end cinema production
  • Extensive color grading
  • VFX-heavy productions
  • Premium HDR content
  • When maximum quality required

16-bit:

  • RAW photography workflows
  • Extensive post-production manipulation
  • Archival masters
  • High-end visual effects
  • Maximum quality regardless of cost

Bit Depth Workflow Recommendations:

Camera RAW (12-16 bit)
        ↓
   Intermediate (10-bit ProRes/DNxHR)
        ↓
     Grading (10-12 bit)
        ↓
    Delivery (8-10 bit)

Chroma Subsampling Decisions

Selection Guide:

Use 4:4:4 for:

  • Green screen/blue screen work
  • VFX compositing
  • High-end color grading
  • Graphics and text overlays
  • Archival masters
  • When storage/bandwidth not limiting

Use 4:2:2 for:

  • Standard production work
  • Color grading (most cases)
  • Live production and broadcasting
  • Good balance of quality and efficiency
  • Industry standard for professional codecs

Use 4:2:0 for:

  • Web delivery and streaming
  • Consumer content
  • Final distribution format
  • Bandwidth-constrained environments
  • When quality requirements less critical

Chroma Key Considerations:

  • 4:4:4 essential for clean keys
  • 4:2:2 may work with simple backgrounds
  • 4:2:0 generally unsuitable for chroma keying
  • Test keying workflow before committing to subsampling level

Compression Settings

Codec-Specific Recommendations:

ProRes Family:

  • Proxy/LT: Offline editing, proxy workflows
  • Standard: Online editing, moderate grading
  • HQ/XQ: Extensive grading, VFX work, mastering
  • 4444: Alpha channel, maximum quality

DNxHR Family:

  • LB/SQ: Similar to ProRes LT/Standard
  • HQ/HQX: Similar to ProRes HQ
  • 444: Alpha channel, maximum quality

H.264/H.265:

  • Use constant quality (CRF) mode rather than constant bitrate
  • CRF 18-22 for high quality (H.264)
  • CRF 20-24 for high quality (H.265)
  • Use 2-pass encoding for VBR
  • Enable appropriate presets (medium, slow, slower)

Quality Testing:

  • Test with worst-case footage (fine detail, motion, gradients)
  • Examine gradients for banding
  • Check edges for compression artifacts
  • Verify skin tones look natural
  • Test on target viewing platforms

Failure Modes

Banding

Symptoms:

  • Visible steps in smooth gradients
  • Posterization effect in skies, shadows
  • Color appears "chunky" or banded
  • Especially visible in subtle gradients

Causes:

  • Insufficient bit depth (8-bit for gradients)
  • Excessive compression
  • Multiple generations of compression
  • Wrong color space (Rec.709 for wide gamut content)
  • Poor encoding settings

Mitigation:

  • Use minimum 10-bit for gradient-heavy content
  • Increase bitrate/quality for distribution codecs
  • Use dithering to mask banding
  • Apply subtle noise to gradients
  • Test worst-case gradients (sunset, dim lighting)

Color Fringing

Symptoms:

  • Color artifacts around sharp edges
  • Rainbow halos around high-contrast boundaries
  • Color smearing or bleeding
  • Especially visible with text and graphics

Causes:

  • Chroma subsampling (4:2:0 most problematic)
  • Excessive compression
  • Poor chroma filtering in codec
  • Multiple generations of encoding

Mitigation:

  • Use 4:2:2 or 4:4:4 for critical content
  • Reduce compression ratio
  • Use better quality presets
  • Avoid multiple re-encodings
  • Test with graphics and text overlays

Compression Artifacts

Symptoms:

  • Blocking (square artifacts in flat areas)
  • Mosquito noise around edges
  • Ringing or halos
  • Loss of fine texture detail

Causes:

  • Bitrate too low for content complexity
  • Wrong codec for content type
  • Excessive compression ratio
  • Single-pass encoding with VBR
  • Poor quality preset

Mitigation:

  • Increase bitrate/quality settings
  • Use appropriate codec for content (H.265 for 4K, H.264 for HD)
  • Use 2-pass VBR encoding
  • Use slower/better quality presets
  • Test with various content types

Mini Glossary

  • [Bit depth]: Number of bits used to represent each color component, determining number of possible color values (8-bit = 256 values, 10-bit = 1,024 values).

  • [Chroma subsampling]: Technique reducing color information while preserving luminance, based on human eye's greater sensitivity to brightness than color.

  • [4:4:4]: No chroma subsampling, full color information for each pixel, highest quality but largest file size.

  • [4:2:2]: Horizontal color resolution halved, vertical maintained, professional standard, good quality/size balance.

  • [4:2:0]: Both horizontal and vertical color resolution halved, standard for web and broadcast, highest compression.

  • [Compression ratio]: Ratio of original uncompressed size to compressed size (e.g., 10:1 means compressed file is 1/10th original size).

  • [Banding]: Visible steps in smooth gradients caused by insufficient bit depth or excessive compression, common in 8-bit content.

  • [Color fringing]: Chromatic aberration around edges caused by chroma subsampling or excessive compression.

  • [Dynamic range]: Ratio between brightest and darkest captureable values, related to but distinct from bit depth.

  • [Gradation]: Smooth transition between colors or tones, higher bit depth produces smoother gradations with less banding.

Further Reading

  • ITU-R BT.601 (Studio encoding parameters for digital TV)
  • ITU-R BT.709 (Parameter values for HDTV)
  • SMPTE ST 2084 (High Dynamic Range electro-optical transfer function)
  • Codec white papers: Apple ProRes, Avid DNxHR