Skip to content

feat: FFmpeg hardware-accelerated export pipeline (NVENC/AMF/QuickSync)#141

Open
lsj5031 wants to merge 1 commit intowebadderall:mainfrom
lsj5031:feat/ffmpeg-gpu-export-clean
Open

feat: FFmpeg hardware-accelerated export pipeline (NVENC/AMF/QuickSync)#141
lsj5031 wants to merge 1 commit intowebadderall:mainfrom
lsj5031:feat/ffmpeg-gpu-export-clean

Conversation

@lsj5031
Copy link
Copy Markdown

@lsj5031 lsj5031 commented Mar 30, 2026

Summary

This PR adds a new FFmpeg-based export path that can use hardware video encoders when available, while still falling back safely to CPU encoding.

It does not replace the existing WebCodecs/WebMuxer export flow. Instead, it introduces an additional pipeline that can be used for MP4 export in environments where FFmpeg-backed encoding is preferable.

What’s included

FFmpeg export pipeline

  • Adds a new FFmpegExporter
  • Streams raw RGBA frames to FFmpeg over IPC instead of writing large temporary raw video files
  • Supports hardware encoder probing in this order:
    1. NVENC (NVIDIA)
    2. AMF (AMD)
    3. Quick Sync (Intel)
    4. CPU fallback (libx265 / libx264)

Encoder selection and safety

  • Checks whether each encoder is actually available before using it
  • Runs a lightweight startup probe before beginning the real export
  • Falls through automatically if a hardware encoder is missing or unusable
  • Remains safe for non-NVIDIA Windows users because CPU fallback is always available

Export metadata

  • Returns encoding details back to the renderer so the UI can show which encoder/path was used
  • Adds ExportEncodingInfo to describe:
    • encoder name
    • codec family
    • acceleration type
    • whether the export was hardware-accelerated

Supporting changes

  • Adds
    eadCompositeRgbaFrame() to FrameRenderer so the composite canvas can be streamed into FFmpeg
  • Exposes new FFmpeg IPC methods through preload.ts and �lectron-env.d.ts
  • Cleans up WGC helper shutdown by replacing detached stdin thread handling + ExitProcess(0) with normal thread join + return

Why this helps

The current export pipeline already works well, but FFmpeg gives us another option that can be useful for:

  • leveraging vendor-specific hardware encoders
  • reducing memory/disk pressure by streaming frames directly
  • giving clearer visibility into which encoding path was used
  • broadening compatibility when WebCodecs support is limited or inconsistent

Notes on compatibility

This change is designed to degrade gracefully:

  • If NVENC is unavailable, it tries AMF
  • If AMF is unavailable, it tries Quick Sync
  • If no hardware encoder works, it falls back to CPU encoding

So this should still be safe on systems without NVIDIA GPUs.

Files changed

  • src/lib/exporter/ffmpegExporter.ts
  • src/lib/exporter/frameRenderer.ts
  • src/lib/exporter/types.ts
  • src/lib/exporter/index.ts
  • �lectron/ipc/handlers.ts
  • �lectron/preload.ts
  • �lectron/electron-env.d.ts
  • �lectron/native/wgc-capture/src/main.cpp

Validation

Tested locally on Windows:

  • native helper build succeeded
  • TypeScript build succeeded
  • Vite build succeeded
  • Electron packaging succeeded

Follow-up ideas

If this direction looks good, a follow-up could wire this exporter more explicitly into the editor UI so users can choose or inspect the export path more directly.

Add FFmpegExporter as an alternative to WebCodecs-based VideoExporter,
enabling GPU-accelerated encoding via NVENC (NVIDIA), AMF (AMD),
QuickSync (Intel), with automatic CPU fallback (libx265/libx264).

Key changes:
- New FFmpegExporter streams RGBA frames to FFmpeg stdin via IPC,
  avoiding multi-GB temp files
- Encoder probe system: checks availability and usability of each
  hardware encoder before use, falling back gracefully
- ExportEncodingInfo type reports encoder, codec family, and
  acceleration mode back to the renderer
- readCompositeRgbaFrame() added to FrameRenderer for raw pixel access
- Clean WGC shutdown: replace detached thread + ExitProcess(0) with
  joinable thread and normal return
- Add missing electron-updater dependency for TypeScript compilation
@lsj5031 lsj5031 force-pushed the feat/ffmpeg-gpu-export-clean branch from b415c24 to 42dd585 Compare March 30, 2026 08:48
@webadderall
Copy link
Copy Markdown
Owner

will take a look at this soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants