The gap
Our unit coverage is strong (~340 tests over model, persistence, palette, hotkey, and config logic), and not unit-testing SwiftUI/AppKit/libghostty is the right call. But that leaves a whole category untested: we have no integration or end-to-end tests at all, and CI runs only format, lint, and unit tests. Nothing checks that the pieces work once assembled, and nothing checks that a real terminal session behaves correctly from the first keypress. That's exactly where our worst regressions came from — the broken TERM=xterm-ghostty and dead keyboard input in #39/#40 shipped because no test could catch them.
Two tiers would close this. Integration tests check that the assembled app is wired correctly; end-to-end tests check that a real session actually works, with nothing stubbed (real shells, PTYs, libghostty, and relaunch cycles).
Integration tier
Drive the built app with XCUITest, seed known state before launch, and assert on the result. Seeding is easy: the app already reads -macterm.activeProjectID and persists to a per-build data dir (Macterm Debug/), so a test can drop in a known projects.json/workspaces_v3.json/.macterm/layout.yaml, launch, assert, and clean up. The high-value flows are workspace restore on launch, declarative layout auto-apply on first open, the config pipeline actually applying (our overrides winning over the user's Ghostty config), single-window enforcement, and focus landing correctly after tab/split reshapes — all things unit tests prove in isolation but never prove assembled.
End-to-end tier
This runs real terminal sessions and asserts on terminal behavior, not app state. The one wrinkle: the surface is Metal-rendered, so its text isn't readable via accessibility. The fix is to assert through side channels the shell can write — have the shell run echo OK > "$OUT" (or printf '%s' "$TERM", infocmp xterm-ghostty, etc.) and poll the file. That's actually a better test of "did the command really run" than reading pixels, and it needs no special privileges.
The scenarios that matter most are a keystroke-to-execution round-trip (the basic "the terminal works," and a guard against #40), TERM/terminfo correctness with a real TUI launching and exiting cleanly (the #39/#40 breakage, ideally run once on a machine with no Ghostty.app so bundled resources are truly exercised), shell integration propagating cwd/title, and a full session surviving quit-and-relaunch. Clipboard, scrollback (#102), and resize/reflow are worth adding later.
CI
Both tiers need a GUI login session. Hosted macos runners can run basic XCUITest and most of the side-channel E2E tests; anything needing Accessibility or dismissing system dialogs can't run there. So keep both out of the required PR check and tier execution: local always, a nightly hosted job for the grant-free subset, and a self-hosted runner with Accessibility pre-granted for the rest if it earns its keep.
Scope
Additive only — this doesn't change the unit suite or the "don't unit-test the UI" rule, and it's not about snapshot/visual tests or libghostty internals. A reasonable start is the harness plus two smoke tests (workspace restore + keystroke round-trip), expanding from there.
The gap
Our unit coverage is strong (~340 tests over model, persistence, palette, hotkey, and config logic), and not unit-testing SwiftUI/AppKit/libghostty is the right call. But that leaves a whole category untested: we have no integration or end-to-end tests at all, and CI runs only format, lint, and unit tests. Nothing checks that the pieces work once assembled, and nothing checks that a real terminal session behaves correctly from the first keypress. That's exactly where our worst regressions came from — the broken
TERM=xterm-ghosttyand dead keyboard input in #39/#40 shipped because no test could catch them.Two tiers would close this. Integration tests check that the assembled app is wired correctly; end-to-end tests check that a real session actually works, with nothing stubbed (real shells, PTYs, libghostty, and relaunch cycles).
Integration tier
Drive the built app with XCUITest, seed known state before launch, and assert on the result. Seeding is easy: the app already reads
-macterm.activeProjectIDand persists to a per-build data dir (Macterm Debug/), so a test can drop in a knownprojects.json/workspaces_v3.json/.macterm/layout.yaml, launch, assert, and clean up. The high-value flows are workspace restore on launch, declarative layout auto-apply on first open, the config pipeline actually applying (our overrides winning over the user's Ghostty config), single-window enforcement, and focus landing correctly after tab/split reshapes — all things unit tests prove in isolation but never prove assembled.End-to-end tier
This runs real terminal sessions and asserts on terminal behavior, not app state. The one wrinkle: the surface is Metal-rendered, so its text isn't readable via accessibility. The fix is to assert through side channels the shell can write — have the shell run
echo OK > "$OUT"(orprintf '%s' "$TERM",infocmp xterm-ghostty, etc.) and poll the file. That's actually a better test of "did the command really run" than reading pixels, and it needs no special privileges.The scenarios that matter most are a keystroke-to-execution round-trip (the basic "the terminal works," and a guard against #40),
TERM/terminfo correctness with a real TUI launching and exiting cleanly (the #39/#40 breakage, ideally run once on a machine with no Ghostty.app so bundled resources are truly exercised), shell integration propagating cwd/title, and a full session surviving quit-and-relaunch. Clipboard, scrollback (#102), and resize/reflow are worth adding later.CI
Both tiers need a GUI login session. Hosted
macosrunners can run basic XCUITest and most of the side-channel E2E tests; anything needing Accessibility or dismissing system dialogs can't run there. So keep both out of the required PR check and tier execution: local always, a nightly hosted job for the grant-free subset, and a self-hosted runner with Accessibility pre-granted for the rest if it earns its keep.Scope
Additive only — this doesn't change the unit suite or the "don't unit-test the UI" rule, and it's not about snapshot/visual tests or libghostty internals. A reasonable start is the harness plus two smoke tests (workspace restore + keystroke round-trip), expanding from there.