Skip to content

Answer sixel handshake queries so apps emit and size sixel correctly#77

Merged
tig merged 2 commits into
mainfrom
claude/sixel-handshake
Jun 5, 2026
Merged

Answer sixel handshake queries so apps emit and size sixel correctly#77
tig merged 2 commits into
mainfrom
claude/sixel-handshake

Conversation

@tig

@tig tig commented Jun 4, 2026

Copy link
Copy Markdown
Member

Summary

A sixel-capable TUI only emits sixel after a terminal handshake, and the rendered raster is sized from the cell metrics the terminal reports. tuirec answered DA1 but in a form many apps reject, and never answered the window/cell geometry queries ? so apps either never enabled sixel, drew nothing (no screen size), or emitted a raster that overflowed its on-screen cells.

Verified end-to-end: real Terminal.Gui UICatalog Mandelbrot scenario -> tuirec (these fixes) -> sixel-aware agg -> the Mandelbrot renders in the GIF, fitting its view.

What was wrong

  • DA1 form. Response was ESC [ ? 62 ; 4 c. Apps that detect sixel by splitting the reply on ; and checking for the token 4 (e.g. Terminal.Gui's SixelSupportDetector: response.Split(";").Contains("4")) saw the token 4c and never enabled sixel.
  • No geometry replies. tuirec did not answer CSI 14/16/18 t. Without 18 t (text-area size) Terminal.Gui's ANSI driver could not determine the screen size and drew nothing; without 16 t (cell size) sixel fell back to 1px/cell.
  • Raster overflow. When a fixed cell size was reported it did not match agg's, so the app's raster (cells x reported-cell-px) did not map to agg's cells and overflowed the view.

Changes

  • DA1 response is now ESC [ ? 62 ; 4 ; 6 ; 22 c (sixel attribute mid-list, as real terminals report it).
  • The interceptor answers CSI 16 t (cell size), CSI 18 t (text area in chars), and CSI 14 t (window in pixels).
  • The reported cell size is derived from the agg font-size/line-height (row = fontSize*lineHeight, column ~= 0.6*fontSize) so the app's raster matches the cells agg renders.
  • Agent guidance (RECORDING-AGENT.md, agent-guide.md) and README.md updated: sixel support, the Linux/macOS-only constraint, --trim=false for alternate-screen apps, and how to record a Terminal.Gui sixel scenario.

Test plan

  • go test ./pkg/record/ ? new unit tests cover the DA1 token (TestDA1ResponseAdvertisesSixelAsDelimitedToken) and the geometry replies (TestInterceptorAnswersCellSizeQuery, TestInterceptorAnswersTextAreaQuery); existing sixel pipeline tests still pass.
  • End-to-end on Linux: recorded UICatalog Mandelbrot (--args Mandelbrot --trim=false, quit with Esc); cast holds 67+ sixel DCS payloads with Terminal.Gui's palette; the sixel-aware agg renders the Mandelbrot fitting its view.

Known follow-ups (not in this PR)

  • --trim discards alternate-screen recordings. With trim enabled (default), a recording whose content is entirely inside the alternate screen is trimmed to nothing; --trim=false is the current workaround. Worth a dedicated fix.
  • Windows ConPTY strips sixel DCS (and answers the app's DA1 itself), so sixel cannot be captured on Windows. This is a conhost limitation, not fixable in tuirec; sixel recording is Linux/macOS-only.
  • Rendering requires a sixel-aware agg (asciinema/agg sixel support).

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

A sixel-capable app (e.g. Terminal.Gui's UICatalog Mandelbrot scenario) only
emits sixel after a terminal handshake. tuirec answered DA1 but with the form
ESC [ ? 62 ; 4 c. Apps that detect support by splitting the response on ";"
and looking for the token "4" (Terminal.Gui's SixelSupportDetector) saw "4c"
and never enabled sixel. tuirec also did not answer the window/cell geometry
queries (CSI 14/16/18 t), so the app could not determine the screen size (and
drew nothing) or the cell size (sixel fell back to 1px per cell), and the
rendered raster overflowed its on-screen cells.

Changes:
- DA1 response is now ESC [ ? 62 ; 4 ; 6 ; 22 c (sixel attribute mid-list, as
  real terminals report it).
- The interceptor answers CSI 16 t (cell size), 18 t (text-area in chars), and
  14 t (window in pixels).
- The reported cell size is derived from the agg font-size/line-height
  (row = fontSize*lineHeight, column ~= 0.6*fontSize) so the app's sixel raster
  matches the cells agg renders.
- Unit tests in pkg/record cover the DA1 token and the geometry replies.

Docs: update agent guidance (RECORDING-AGENT.md, agent-guide.md) and README to
describe sixel support, the Linux/macOS-only constraint (Windows ConPTY strips
sixel DCS), the --trim=false requirement for alternate-screen apps, and how to
record a Terminal.Gui sixel scenario.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80d9cb3388

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/record/record.go Outdated
record.Run accepts a zero-valued Config (pty.Start and gif.Render fill in
defaults later), but the sixel interceptor was built from the raw config, so a
zero Config produced cellW/cellH and rows/cols of 0. Apps that sent CSI 16/14/18
t then got reports like CSI 6;0;0t or CSI 8;0;0t, defeating the handshake (no
layout, or a 0x0 raster) even though the PTY used default dimensions.

Add sixelGeometry, which normalizes the terminal size and GIF font config with
the same defaults pty.Start and gif.Render apply, and derive the reports from
it. Exports pty.NormalizeSize and gif.NormalizeConfig so the geometry mirrors
the size/metrics actually used. Covered by a unit test asserting a zero Config
yields 120x30 cells and an 8x18 px cell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tig tig merged commit 5352e99 into main Jun 5, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant