Skip to content

feat: add animated GIF export to record-video - #84

Merged
onevcat merged 2 commits into
lycorp-jp:mainfrom
kws0210:feature/gif-export
Jul 30, 2026
Merged

feat: add animated GIF export to record-video#84
onevcat merged 2 commits into
lycorp-jp:mainfrom
kws0210:feature/gif-export

Conversation

@kws0210

@kws0210 kws0210 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Tracking: LINECLIENT-1689 (LY internal)

What

record-video gains --format mp4|gif on all three surfaces (top-level, ios record-video, android record-video).
The format is also inferred from a .gif --output extension; an explicit --format wins, with a stderr note on mismatch.

sim-use record-video --output demo.gif            # inferred; 10 fps / 0.5 scale GIF defaults
sim-use record-video --format gif --fps 15 --scale 0.4

Why

Animated GIFs auto-play inline in PRs / issues / chat, which makes them the most convenient evidence format for short agent-driven verification flows — today producing one requires an out-of-band ffmpeg conversion.

Design

GIF is a post-stop transcode, not a fourth capture path: none of the capture pipelines (idb's in-process recorder on iOS, adb screenrecord passthrough on Android, the screenshot/screencap fallbacks) expose decoded frames in-process, and the signal-to-finalise path is latency-critical. Capture lands in an intermediate MP4; on stop, GIFTranscoder (in SimUseVideo) decodes it with AVAssetReader and encodes a looping GIF via CGImageDestination.

  • Two decoding passes bound decode memory to one frame: pass 1 collects display-timeline PTS (CGImageDestination needs the frame count up front; passthrough PTS carry B-frame reorder offsets and un-applied edit lists, so a non-decoding read can't drive the plan), pass 2 appends only the sampled frames.
  • Sampling walks a 1/fps target clock (capped at 50 fps — the GIF centisecond delay floor makes faster pacing dishonest), normalizing Android's VFR capture; per-frame delays follow source timestamp gaps.
  • Shared orchestration: RecordingOutputPlan owns resolve → warn → prepare → stale-intermediate cleanup → transcode for every surface, so the flag contract can't drift between platforms (same posture as VideoRecordingOptions).
  • Failure safety: a failed transcode preserves the intermediate MP4 (and says where), removes the partially written GIF, and a stale intermediate from a previous failed/killed run is cleared before capture. The capture signal observer is torn down before the transcode so Ctrl+C during a long encode stays effective. Encodes past ~300 frames print a memory warning (ImageIO holds all frames until finalize; documented in the README).

Base

Started from 50bc5a0 and rebased onto da63f16 (post-v0.12.0), relocating the code into the SimUseVideo module introduced by #79 — which is also what made the android record-video surface come along for free (as discussed on Slack).

Testing

  • make test fully green (834 unit tests), including 16 new GIFTranscoderTests — sampling-plan properties, format/option resolution, output-plan lifecycle, and a real MP4→GIF round-trip synthesized through H264StreamRecorder (no simulator needed).
  • Live-verified on an iPhone 17 simulator (iOS 26) against LINE Dev: multiple end-to-end repro GIFs recorded with a single command each (see Demo).

Demo

Each recorded against LINE Dev on an iPhone 17 simulator with a single record-video --output <name>.gif invocation (touch points enabled in the app):

Open the camera from the LINE AI screen
(83 frames, 905 KB)
Change the app font and apply
(68 frames, 626 KB)
line-ai-camera-repro line-font-change-repro

🤖 Generated with Claude Code

record-video gains --format mp4|gif (also inferred from a .gif
--output extension; explicit --format wins with a stderr note on
mismatch) on all three surfaces: the top-level cross-platform verb,
ios record-video, and android record-video.

GIF is a post-step rather than another capture path: none of the
capture pipelines (idb's in-process recorder on iOS, adb screenrecord
passthrough on Android, the screenshot/screencap fallbacks) expose
decoded frames in-process, and the signal-to-finalise path is
latency-critical. The recording lands in an intermediate MP4 first;
on stop, GIFTranscoder (SimUseVideo) decodes it with AVAssetReader
and encodes a looping GIF via CGImageDestination.

Two decoding passes bound peak memory to one frame: pass 1 collects
display-timeline timestamps (CGImageDestination needs the image count
at creation; passthrough PTS carry B-frame reorder offsets and
un-applied edit lists, so a non-decoding read can't drive the plan),
pass 2 appends only the sampled frames. Sampling walks a 1/fps target
clock, which also normalizes Android's variable-frame-rate capture;
per-frame delays follow the source timestamp gaps (2 cs floor).

Format/fps/scale resolution lives in SimUseVideo
(ResolvedRecordingOptions) so the surfaces cannot drift: GIF defaults
are fps 10 / scale 0.5 (mp4 keeps 30 / 1.0) because full-rate
full-scale GIFs are enormous; both remain overridable. A failed
transcode preserves the intermediate MP4 and reports its path, so
footage is never lost.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: wonseob <wonseob@linecorp.com>
@onevcat

onevcat commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

The current sampling plan can stretch GIF playback at the documented 50 fps cap.

For a 60 fps source with --fps 60 (capped to 50), pts >= nextTarget - epsilon admits frames that are only 16.7 ms apart. Each is then independently clamped to the GIF 20 ms floor. Over a 10-second source this yields about 400 GIF frames and about 10.65 seconds of playback; a 120 fps source produces about 10.80 seconds. This breaks the stated wall-clock pacing guarantee on a normal iOS 60 fps recording path.

Could we make frame selection and centisecond quantization share a cumulative timing model, so the 20 ms floor does not lose timing debt? Re-selecting frames or carrying the rounding error into subsequent delays would both avoid the stretch.

Please also add 60 fps and 120 fps source cases with --fps 60, asserting total GIF duration to centisecond-level tolerance. The current abs(duration - 1.0) < 0.1 check allows this regression through.

Review follow-up (lycorp-jp#84): with a 60/120 fps source, the target-clock
admission let frames closer than the 2 cs GIF floor through, and each
gap was then clamped up independently — accumulating ~6.5%/8% playback
stretch over a 10 s recording despite the 50 fps cap.

Two-part fix, per review: selection now refuses a frame closer than
minimumDelay to the previous kept frame (a frame that would need
clamping is never chosen), and delays quantize the cumulative timeline
to centiseconds so per-frame rounding carries forward instead of
accumulating. New parametrized tests pin 60 and 120 fps sources at
--fps 60 to centisecond-level total-duration tolerance, and the
existing cap test's tolerance is tightened from 0.1 to 0.02.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: wonseob <wonseob@linecorp.com>
@kws0210

kws0210 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Confirmed — a code bug, not a doc issue: the admission tolerance let sub-2cs gaps through, and independent clamping accumulated exactly the stretch you measured.

Fixed in 35848ce with your cumulative timing model:

  1. Selection skips any frame closer than minimumDelay to the last kept frame — no chosen gap ever needs clamping.
  2. Delays quantize the cumulative timeline to centiseconds, so rounding carries forward instead of accumulating.

Added the requested 60/120 fps source tests (10 s at --fps 60, abs(duration - 10.0) <= 0.02) and tightened the cap test to the same tolerance.
Suite green (835).

@onevcat

onevcat commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

@kws0210 Thank you! LGTM and this is really useful!

@onevcat
onevcat merged commit 15473f2 into lycorp-jp:main Jul 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants