From 3a183344ca7a1523f3204845a669d972af27b54e Mon Sep 17 00:00:00 2001 From: "Jiwei,Yuan" Date: Wed, 5 Aug 2026 14:11:58 +0100 Subject: [PATCH] docs: establish the device architecture from primary sources MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add termiod-device-architecture.md: there is no "remote" — there are devices, each running one termiod, and every UI including the Mac app is a client that attaches to one. A device's identity is its host_id, not an SSH alias; SSH is one route among several to the same machine, so the same box reachable over LAN, WAN and Tailscale stops forking into three session lists. Correct the state-authority section against primary sources. An earlier draft, written from press paraphrase ("the server maintaining all state"), concluded we should move to server-side state synchronisation. Mitchell's architecture video and his reply to tmux maintainer Jonathan Slenders say the opposite: Superlogical tees raw PTY bytes to clients "like SSH", and while the server does parse, "the teeing happens ahead of the server". That is termiod's existing anti-100x invariant, arrived at independently — the architecture needs no course change. Adopt their better argument against screen diffs: the problem is less performance than that a diff-fed client owns no real scrollback and cannot select across history. That is a capability limit no amount of encoding work removes, so G stays opt-in even once the wire cell shrinks. Bring the protocol spec up to the branch's state: §C.10 resumable subscriptions, and the VT-sequence snapshot format with the measurements behind it. Add the hot-path and client-class analysis, and CLAUDE.md, which records the positioning and the architectural invariants that follow from it. Refs: #164, #177 --- CLAUDE.md | 54 +++ docs/README.md | 2 + docs/design/termiod-device-architecture.md | 410 ++++++++++++++++++ .../termiod-hot-path-and-client-classes.md | 371 ++++++++++++++++ docs/design/termiod-session-protocol.md | 269 +++++++++++- 5 files changed, 1088 insertions(+), 18 deletions(-) create mode 100644 CLAUDE.md create mode 100644 docs/design/termiod-device-architecture.md create mode 100644 docs/design/termiod-hot-path-and-client-classes.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..42b4fab8 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,54 @@ +# termio + +## What termio is + +**A terminal-first agentic development environment, including its own remote server.** + +Coding agents live in terminals. termio is the place those agents run, on machines +the user already owns — a laptop, a VPS, a devbox — supervised from a native Mac +app and an iPhone. `termiod` is the durable session host that makes "on any box" +one code path instead of four. + +The category is crowded (Zed remote, VS Code Remote, Codespaces, Warp). That is a +known and accepted cost of this position, not an oversight. **Do not re-argue the +positioning.** Build it. + +## What makes it different (aim here, not at editors) + +- **The session lives on the box, not in the connection.** Detach ≠ kill. An agent + keeps working while the laptop is shut; reattach restores the exact screen. + Zed and VS Code tie remote work to a live connection; termio does not. +- **The user's machines, not our cloud.** termio never provisions compute and never + runs a hosted control plane. The code never leaves hardware the user controls — + the only trust story that survives giving an agent shell access to a private repo. +- **Agent-native at the protocol level.** Workstream status (`working` / `idle` / + `needs-you` / `done`) is a first-class protocol object, not a heuristic scraped + off the screen. +- **Terminal-first is a commitment, not a stage.** The terminal *is* the interface. + Chat UIs have been built and fully reverted twice. Don't rebuild them. + +## Architectural non-negotiables + +These come from `docs/design/termiod-session-protocol.md` §A and §H. Violating one +is a design regression, not a tradeoff: + +1. **Anti-100× invariant** — byte delivery never blocks on host-side VT parse. The + authoritative VT is a sidecar for snapshots; any per-frame grid encoder between + the PTY and the pipe rebuilds the tmux tax and is rejected. +2. **State sync only at boundaries** — snapshots on attach / resize / resync, never + per frame. `grid_diff` is an opt-in bad-network degrade, never the default. +3. **Never embed SSH or crypto.** System OpenSSH, tailnets, and OS keychains are the + security team we didn't hire. The user's `~/.ssh/config` is authoritative — read + it, never override it. +4. **One protocol, versioned and transport-agnostic.** Unix socket, SSH stdio, and + later QUIC carry the *same* framed messages. No second protocol for the phone. +5. **No nested window manager in the host.** One PTY per session; panes and layout + are client concerns. +6. **Single writer, many readers.** Observers never claim the write token. + +## Working style + +- Elegance is a small surface area. No grand architecture, no cut-rate MVP. +- Verify before claiming done; report failures with the actual output. +- Name mechanisms, not agents. +- Don't re-pitch ideas that were built and dropped — the design docs record why. diff --git a/docs/README.md b/docs/README.md index ca50a167..ffed073f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -76,6 +76,8 @@ from the real front matter). | draft | design | [移动端 Agent UI 协议 —— PTY 之上的旁路结构面（ACP 词汇）](design/mobile-agent-ui-protocol.md) | | draft | design | [调研：下一批 AgentAdapter 的落盘格式（OpenCode / Pi / Amp / Cursor / Kimi）](design/agent-transcript-survey.md) | | draft | design | ["Markdown preview — Apple-grade reading typography"](design/markdown-preview-reading-typography.md) | +| draft | design | [Device Architecture — one server per device, every UI a client](design/termiod-device-architecture.md) | +| draft | design | [Hot path, attach join point, and client classes](design/termiod-hot-path-and-client-classes.md) | | draft | design | [Issue Triage → 本地 agent（GitHub / Linear 事件驱动 termio session）](design/issue-triage-local-agent.md) | | draft | design | [Mac retention analytics (no app changes)](design/mac-retention-analytics.md) | | draft | design | [Remote Projects (open an SSH/VPS box like a local project)](design/remote-projects.md) | diff --git a/docs/design/termiod-device-architecture.md b/docs/design/termiod-device-architecture.md new file mode 100644 index 00000000..d2ec850a --- /dev/null +++ b/docs/design/termiod-device-architecture.md @@ -0,0 +1,410 @@ +--- +title: Device Architecture — one server per device, every UI a client +status: draft +type: design +created: 2026-08-05 +updated: 2026-08-05 +related: + - termiod-session-protocol.md + - termiod-session-mux.md + - session-daemon-architecture.md + - remote-projects.md +--- + +# Design: Device Architecture + +> There is no "remote". There are **devices**, each running one `termiod`, and every UI — the Mac app included — is a **client** that attaches to one of them. + +**Evidence policy** (inherited from [termiod-session-protocol.md](termiod-session-protocol.md)): +Superlogical statements are labeled **Announced** (superlogical.com / Mitchell's +post / press coverage), **Inferred**, or **Unknown**. They have published no wire +protocol; nothing here guesses at one. + +--- + +## 0. Conclusion first + +1. **Client/server is the model, including locally.** `termio.app` owns no PTY. + tmux has shipped this for twenty years; Superlogical chose the same + (**Announced**: *"sessions run server-side, clients send input as with SSH"*). + Local is not a special case — it is the device whose route is a Unix socket. +2. **A device's identity is its `termiod` `host_id`**, not an SSH alias. SSH is + one *route* to a device, not the device itself. +3. **Raw PTY bytes are teed to clients; the host parses in parallel, never in + between.** This is termio's existing anti-100× invariant, and primary + sources confirm Superlogical converged on the identical design + (**Announced**: *"we tee them off to all the clients, and we send them raw + like SSH"*; *"the teeing happens ahead of the server"*). §3 corrects an + earlier revision of this document that had it backwards. +4. **The server never decides presentation.** It describes what is on the + screen; the client's libghostty decides how it looks. This is termio's own, + learned the expensive way (§4). +5. **Four planes, one connection, one recovery rule.** Terminal, resource + subscription, request, device transfer — all resumable by cursor. + +What this retires: the local/remote fork in the app, `TERMIO_TERMIOD` as a +flag, per-session remote host, and every menu verb with "Remote" in its name. + +--- + +## 1. Why client/server, even locally + +termio today has two ways to run a session: `PTYProcess` in-process, or the +daemon when `TERMIO_TERMIOD=1`. Two paths means every feature is written twice +and the second one rots. The bugs found while testing this branch were all +symptoms: a local terminal silently became remote because of an environment +variable; a remote terminal landed in the loose-terminals bucket because there +was no device to hang it on. + +The fix is not better branching. It is deleting the branch: + +| | Before | After | +| --- | --- | --- | +| PTY owner | app *or* daemon | **always** the device's `termiod` | +| Local session | in-process | attach over Unix socket | +| Remote session | attach over SSH | attach over SSH | +| Code paths | two | **one** | + +**Cost, measured** (this Mac, 2026-08-05): connect+hello 0.2 ms, attach→first +frame 2.2 ms, echo ~1 ms above in-process. Against a 16 ms frame budget this is +not perceptible. The real cost is a new failure mode (§6), not latency. + +--- + +## 2. Device and route + +``` +Device (host_id — the machine's identity) + └──< Route unix socket · ssh · later QUIC / relay +``` + +| Concept | What it is | Source of truth | +| --- | --- | --- | +| **Device** | A machine running `termiod` | `host_id`, minted on the daemon's first run, persisted beside its socket | +| **Route** | A way to reach that device | `~/.ssh/config` for SSH routes; the socket path for local | +| **Readiness** | Reachable? `termiod` installed? version? | Runtime probe, never configuration | + +**Devices are discovered, not configured.** You cannot know which device a +route leads to until you connect: open the route, read `host_id` from +`hello_ok`, and record `host_id ← route`. The same `host_id` arriving over a +second route joins that device's route list rather than creating a twin. + +This matters concretely: one VPS commonly appears in `~/.ssh/config` as +`vps-lan`, `vps-wan`, and a Tailscale name. Keyed by alias they are three +devices with three session lists. Keyed by `host_id` they are one machine with +three roads, and switching networks does not fork your state. + +`~/.ssh/config` remains the single source of SSH hosts — termio keeps no host +database, per §H #8 (never embed SSH). The parser already handles wildcards, +negation, `Include`, and `%h` (`Sources/termio/Settings/SSHConfig.swift`). + +**Open question (§9.1):** `host_id` is not intrinsic — a cloned VM or a reused +container image carries the same one, and a reinstall mints a new one. See §9. + +--- + +## 3. State authority: independent convergence, not a course change + +An earlier revision of this section said we should adopt "server maintains all +state" from Superlogical. **That was based on press paraphrase and is wrong.** +Primary sources — Mitchell's 10:40 architecture video and his replies — say the +opposite of what the coverage implied. + +**Announced, from the video:** + +> instead of sending down **screen diffs** … we take the PTY bytes, we **tee +> them off to all the clients, and we send them raw like SSH** … basically we +> assume you're running libghostty everywhere + +**Announced, answering tmux maintainer Jonathan Slenders** — who asked the +sharp question: if the server sends visible screen state on attach, doesn't it +have to parse too, and isn't that still double parsing? + +> **Yes, the server parses too. But the teeing happens ahead of the server**, so +> the clients can parse **simultaneously** to the servers (also if anyone is +> slow there's some queues). + +That is termiod's anti-100× invariant, stated by someone who arrived at it +independently. **We are already on this architecture; nothing here needs to +change.** The invariant, restated with the extra precision his answer supplies: + +> The authoritative VT is fed **in parallel with**, never **between**, the PTY +> and the clients. The host parsing slowly must never slow a client down; a +> backed-up consumer is absorbed by a queue, not by stalling delivery. + +**Why the host still keeps authoritative state.** Both designs parse +server-side, because attach needs it. Ours additionally wants it for peek +without attach (`what is on that agent's screen right now?` — today termio's +CLI scrapes screens for this), and for catch-up. So §C.6's "sidecar consulted +only to build snapshots" understates its role — but the *hot path* framing was +right all along. + +**Attach handshake, theirs and ours, are the same shape** (**Announced**): +server pauses PTY processing → sends just enough screen state over a custom +binary protocol → a **ready frame** → then raw teed bytes; scrollback streams +in behind, newest-first. termiod already ships `S` → `ready` → `D`, with `H` +scrollback newest-first. + +**Why screen diffs are rejected — the reason we had wrong.** We justified +keeping diffs off the default path with a bandwidth measurement (8.6× worse for +scrolling output on a real VPS, 2026-08-05). That number is real but indicts +our 16-byte-per-cell encoding, not diffing as an idea. His reason is better and +we should adopt it: + +> The issue with the screen diffing is **less performance and more making it +> very difficult to allow native scrollback, selection**, etc. + +A client that receives diffs owns no real scrollback and cannot select text +across history — it only has the rows the server chose to send. That is a +*capability* argument, and it holds no matter how well the diff is encoded. +Consequence for us: **`G` should not be positioned as the bad-network default +even after the wire cell is compressed.** Theirs stays an opt-in extra — +synced viewports are sent as *additional* frames when a user asks for shared +scrolling, not as the transport. + +**Their degrade path** (**Announced**): queues absorb slow clients and a +reconnect catches up by PTY replay; if the queue fills or the disconnect runs +long, a **tombstone** record tells the client to do a full resync. termiod's +ring + 4 MiB backlog + snapshot-on-gap is the same mechanism under different +names — and §C.10's `gap: true` *is* the tombstone. + +--- + +## 4. The presentation boundary (termio's own rule) + +> **The host describes state. It never decides how that state looks.** + +This was learned by shipping the violation: the host resolved every cell to RGB +against *its own* palette, so a remote session ignored the viewer's theme and +rendered on a black background, while bold and underline vanished entirely +(the wire cell's `attributes` field was reserved-zero). + +| Plane | Host sends | Host must not send | +| --- | --- | --- | +| Screen | VT sequences (`38;5;N` indices) | resolved RGB, OSC 4 palette | +| Files | structured entries | a rendered tree | +| Git | porcelain output | formatted diffs | +| Status | enum values | human-facing copy | + +The screen case is fixed: `S` payload v2 is libghostty's own formatter output +with `palette: false`, so colour indices arrive and the viewer's theme resolves +them. Measured 559 B vs 6,504 B of cells for the same 10×40 screen — more +faithful *and* 11.6× smaller. + +The rule is testable: **feed one snapshot to a light-theme and a dark-theme +client; they must look different.** If they look the same, the host is +overreaching. + +--- + +## 5. Four planes, one connection + +| Plane | Carries | Recovery | Status | +| --- | --- | --- | --- | +| **Terminal** | raw bytes (default) · `S` VT snapshot (bootstrap) · `G` diffs (opt-in only — §3) | snapshot + seq | shipped | +| **Resource** | subscriptions with `seq` + bounded ring + linger (`fs:` first) | re-subscribe at cursor | `fs:` shipped | +| **Request** | `exec`, `list`, `read`, `write` | idempotent, request id | **not built** | +| **Transfer** | bytes *between devices*: clipboard, images, files | resume at offset | **not built** | + +All four ride one connection per device, multiplexed by channel id, over one +SSH ControlMaster. Reconnect is not a feature: open a pipe, re-subscribe every +resource at the last applied `seq`. No retry ceiling — a retry loses nothing — +which is strictly stronger than Zed's bounded `MAX_RECONNECT_ATTEMPTS` and than +VS Code's reconnection tokens, which die with the server PID. + +**Transfer is its own plane, not a request verb.** Under the device model, +moving a screenshot to the box an agent runs on is not "uploading to a server"; +it is Universal Clipboard between two of your machines, and the direction is +symmetric. This is what unblocks image paste — today it silently fails, because +the local mechanism is *the agent reading the Mac's pasteboard itself*, which +cannot work when the agent runs on a VPS. + +--- + +## 6. Lifecycle and failure + +Making the daemon the only PTY owner makes it a single point of failure: today +an in-process PTY dies *with* the app, which is at least a shared fate. + +- **Start:** launchd user agent on macOS, systemd `--user` with + `loginctl enable-linger` on Linux. Without linger, a Linux user daemon is + killed at logout — sessions would silently die between SSH connections. +- **Crash:** sessions cannot be resurrected — the PTYs are gone. They must not + vanish silently either: keep a **tombstone** (last snapshot, exit reason, + timestamp) so the UI can say what died instead of showing an empty list. +- **Version skew:** negotiate, never lockstep. `hello` capabilities already + express this; a daemon one version behind should serve what it can. Install + content-addressed (`~/.termio/termiod/-/`) with an atomically + flipped `current`, so versions coexist and a redeploy that matches is a no-op + — this also removes the `ETXTBSY` failure when overwriting a running binary. + +--- + +## 7. Where we agree with Superlogical, and where we do not + +All rows **Announced** unless marked; sources are the architecture video, the +reply threads, and superlogical.com. + +| | Superlogical | termio | +| --- | --- | --- | +| Sessions server-side, client/server even locally | yes — *"for local … its still IPC"* | same | +| Raw PTY bytes teed to clients | yes — *"raw like SSH"* | same | +| Server also parses, tee happens **ahead** of it | yes | same (anti-100×) | +| Attach = paused screen state → ready frame → live bytes | yes | same (`S` → `ready` → `D`) | +| Screen diffs as the default path | **rejected** | same, and for a better reason now (§3) | +| Synced viewport across clients | opt-in extra frames | not built | +| Scrollback streamed newest-first | yes | same (`H`) | +| Overflow degrade | queue → **tombstone** → full resync | ring → `gap: true` → snapshot | +| Splits | native windows/tabs, **one connection per PTY** | native panes; §H #7 forbids a nested WM | +| Legacy terminals | **compat mode**: a libghostty in the middle | not built | +| Wire protocol | custom binary, *"predominantly part of libghostty"*, called an **open protocol** | our own framed protocol | +| Remote transport | **WebSockets over HTTP** (browser is first-class); "not at all guaranteed to be final" | **SSH only** — a trust choice (§H #8) | +| Live human sharing | day one | not a v1 goal | +| Where compute lives | **Unknown** | **the user's own machines, never ours** | +| Agent status in the protocol | **Unknown** | first-class workstream object | + +**The architecture is not the differentiation.** Two teams converged on the +same design independently, which is the strongest evidence available that it is +correct — and it means nothing here is defensible as a moat. The bottom three +rows are. + +**Two things worth acting on:** + +1. Their wire protocol is described as *"predominantly part of libghostty"* and + an **open protocol**. If it ships that way, the cheapest path to a shared + ecosystem may be speaking theirs rather than ours. Worth watching before + investing further in framing details. +2. Their transport being WebSocket-over-HTTP is a **browser-first** decision, + explicitly not final. Ours being SSH-only is a **trust** decision. These are + not competing on the same axis, and ours should be stated as a choice rather + than defended as a feature. + +--- + +## 8. Migration: delete before adding + +Each step is independently shippable and mostly removes code. + +1. **Record device identity.** Keep `host_id` from `hello_ok`; build the + `host_id ↔ routes` map. Pure addition, no UI change. **Done** — + `TermiodDeviceRegistry`, persisted to `devices.json`; every handshake + (attach, `list`, `kill`) records the device it reached, so a second alias for + one machine joins its route list instead of forking it. +2. **Re-key state by device.** Sessions belong to a device; `remoteCheckouts` + moves from alias-keyed to `host_id`-keyed (alias-keyed state silently splits + the day the user changes networks). **Done, minus the container merge** — + `Project.deviceID` / `Session.deviceID` are backfilled on first handshake + (`TermioStore.adoptDevice`), and legacy alias-keyed checkouts are promoted in + place on the way past, so old state files keep working. Containers are still + created and matched by alias, because alias is the only identity that exists + before connecting; merging two of them into one device is §9.5. +3. **Daemon lifecycle on macOS.** launchd agent, restart-on-crash, tombstones. + **Done** — `termiod service install|uninstall|status` writes a + `sh.termio.termiod` launchd user agent (`RunAtLoad` + `KeepAlive`); never + installed automatically, since it makes the daemon outlive every termio + process. Tombstones (`termiod/src/tombstone.rs`) ride the `list` reply and + record `exited` / `killed` / `daemon_lost` with the session's identity, + status, and timestamps. The crash case is inferred rather than supervised: a + session still on the on-disk roster when a daemon starts was never buried, so + the previous daemon died under it. **Not done:** the last screen (§6 asks for + it; capturing it needs a snapshot request threaded through the sidecar's + shutdown path), and the Linux systemd `--user` + `enable-linger` unit. +4. **Delete the fork.** Remove the `TERMIO_TERMIOD` flag, the in-process + `PTYProcess` path, per-session remote host, and every "Remote" menu verb. + `New Terminal` opens on the current device. +5. **Device switcher** in window chrome, with readiness state. Only meaningful + after step 4 — before it, local is still a special case. +6. **Request plane**, then file tree and git move to the current device. +7. **Transfer plane**, then clipboard and image paste work across devices. + +Steps 1–4 are net deletions. The features people are waiting for (files, git, +image paste) are 6–7, and they are last **on purpose**: built before the device +model exists, each would grow its own local/remote fork. + +--- + +## 9. Open questions + +1. **`host_id` is not intrinsic.** A cloned VM or reused container image + carries the same id; a reinstall mints a new one. Options: treat a duplicate + as a conflict and prompt; mix in a machine fingerprint; or accept that the + id names *an installation*, not hardware, and let the user merge/split + devices. Not decided — needs a real collision to reason about honestly. +2. **Route selection.** With several routes reachable, choose by latency — + but probing costs, and it is unresolved whether a live session migrates + across routes or simply reconnects. Baseline measured: SSH cold 216–292 ms, + warm with ControlMaster 26–33 ms. +3. **Global device vs cross-device roster.** A single "current device" is + clean, but termio's value is seeing every agent at once. Current position: + the device scopes *new work and the panels*; the session list stays + cross-device with a device column. Unproven in use. +4. **Clipboard semantics.** Push on copy (eager, leaks everything you copy to + whichever device is current) or pull on paste (lazy, one round trip). Pull + is the safer default; not yet measured for feel. + +5. **Merging two containers that turn out to be one device.** Specified here, + deliberately **not implemented** — the identities are recorded first + (`Project.deviceID`, `Session.deviceID`, shipped), and merging becomes a step + to add rather than a rewrite. + + **Why it cannot simply be "key containers by `host_id`".** `host_id` is + *a posteriori* and an alias is *a priori*. A container must exist the instant + a session is created — `hostContainer(for:)` is called synchronously, before + any handshake has run — and most aliases in a `~/.ssh/config` have never been + connected to at all. So the model is two identities coexisting, not one + replacing the other: + + | | Bootstrap identity | Stable identity | + | --- | --- | --- | + | What | SSH alias (`Project.sshHost`) | device (`Project.deviceID`) | + | Known | before connecting, from `~/.ssh/config` | after the first `hello_ok` | + | Role | the container is born from it and matched by it | the container *is* it, once known | + + A container is created and matched by alias, exactly as today. `deviceID` is + backfilled on first handshake (`TermioStore.adoptDevice`). Only then can two + containers be known to be one machine — and only then may a merge run. + + **The rules a merge must satisfy** (each is a question the naive version gets + wrong): + + - **Sessions.** Concatenate, oldest container first, ordered by `createdAt` + within each. Session ids are already globally unique, so nothing is + renumbered — but auto-generated `Terminal N` titles collide across the two + blocks and must renumber on collision only, exactly as + `liftingRemoteSessionsToHosts` already does. A title the user or a clone + chose is never touched. + - **The container name.** Whichever the user renamed wins; if both were + renamed, the one with the most recent session activity wins and the other + name is kept as a secondary label rather than discarded — a name the user + typed is data, not decoration. If neither was renamed, the alias of the + most recently used route wins, since that is the road they are currently + on. Note this is a *client* decision: the host never supplies a display + name (§4). + - **`remoteCheckouts`.** Already device-keyed, so the two maps merge by key. + A genuine conflict — two different paths for one device — means one entry + is stale; keep the one whose directory the more recent session used and + surface the other rather than silently dropping it. + - **Conflicting `host_id`s must not merge blindly** (§9.1). A cloned VM or a + reused container image carries a duplicate id, so identical `host_id` is + *evidence* of one machine, not proof. Before merging, require corroboration + that the two routes reach the same box — matching boot id or machine-id, + or the same `termiod` process start time — and when it fails, ask rather + than merge. **A wrong merge is destructive** (two machines' sessions + collapse into one list); a missed merge is merely untidy. Asymmetric cost, + asymmetric default: never merge on doubt. + - **Reversibility.** A merge must be undoable, which means the surviving + container has to record the aliases it absorbed. A user who splits a + wrongly merged container should get their two blocks back, not a manual + rebuild. + +--- + +## 10. Relationship to existing docs + +- **Supersedes** §C.6's "sidecar only" framing of the host VT (§3), and the + local/remote asymmetry in [remote-projects.md](remote-projects.md). +- **Extends** [termiod-session-protocol.md](termiod-session-protocol.md) with + the device layer above the host noun; §C.10's resumable subscription becomes + the recovery rule for all four planes. +- **Realises** [session-daemon-architecture.md](session-daemon-architecture.md)'s + "local is the degenerate remote" by removing the last in-process PTY path. diff --git a/docs/design/termiod-hot-path-and-client-classes.md b/docs/design/termiod-hot-path-and-client-classes.md new file mode 100644 index 00000000..de3ef87e --- /dev/null +++ b/docs/design/termiod-hot-path-and-client-classes.md @@ -0,0 +1,371 @@ +--- +title: Hot path, attach join point, and client classes +status: draft +type: design +created: 2026-08-05 +updated: 2026-08-05 +related: + - termiod-session-protocol.md + - termiod-device-architecture.md + - termiod-vt-sidecar-spike.md +--- + +# Design: Hot path, attach join point, and client classes + +> A review of the Grok refinements against the shipped POC. The architecture is +> settled; what is missing is three *unwritten invariants* the code currently +> gets right by accident, and one prologue the client invents for itself. + +**Evidence policy** (inherited from +[termiod-session-protocol.md](termiod-session-protocol.md)): Superlogical +statements are labeled **Announced** (Mitchell's ~10:40 architecture video, +his replies, superlogical.com), **Inferred**, or **Unknown**. No wire spec is +published; nothing here guesses at one. Positioning is settled in +`CLAUDE.md` and is not re-argued. + +--- + +## A. Executive recommendation + +Of the ten Grok refinements: **four accept as written, four modify, two reject.** +None of them changes the architecture, because the architecture is right and +partly shipped. The value in this round is elsewhere — three things are load +bearing, undocumented, and currently correct only because of implementation +details nobody wrote down: + +1. **The attach join point has no spec.** Grok's #1 asks for "attach without + pausing the PTY". That is already true: `begin_snapshot_barrier` + (`termiod/src/session.rs`) never touches the PTY read loop — it flips the + *attaching client* to `SnapshotPending`, buffers that client's bytes, and + lets everyone else keep flowing. Correctness rests entirely on the sidecar + command channel being a **lossless FIFO shared by `Write` and `Snapshot`**: + the snapshot boundary is "after every `Write` enqueued before this + `Snapshot`", and the per-client buffer starts at the same instant. **The FIFO + *is* the sequence number.** Write that down as a normative invariant with a + regression test (§E, D1); do **not** put a `seq` on the wire (§C.2). + +2. **The snapshot prologue is invented by the client.** `render_snapshot` + (`termiod/src/client.rs:693`) emits `ESC[2J ESC[H` before the host's VT + payload. That prologue is a client-local guess: `2J` does not reset the + scrolling region, alt-screen, DECAWM, charsets, or pending SGR. Two clients + will diverge on reattach-over-dirty-screen, which breaks the + synchronized-state-machine guarantee §C.5 spends a paragraph defending. + **The prologue belongs to the host, inside the `S` payload** (D2). + +3. **The anti-100× invariant has a second shadow cost.** Risk #10 bounded the + *per-client* backlog (4 MiB, `CLIENT_BACKLOG_CAP`) — shipped. But the + **sidecar command queue is unbounded** (`std_mpsc::channel::`, + `session.rs:1064`). Refusing to block byte delivery on the VT parse means a + slow parse now accumulates *the whole PTY output* in a queue instead. It is + the same bug as risk #10, one hop upstream, and it is not written anywhere + (D4). + +Everything else in this doc is naming, policy wording, and PR sequencing. + +--- + +## B. Keep — settled, do not reopen + +| Kept | Where it lives | Why it is closed | +| --- | --- | --- | +| **Anti-100× invariant** — byte delivery never blocks on host VT parse | §A, CLAUDE.md #1 | Measured 4.4–6.0× tmux; independently converged on by Superlogical (**Announced**: *"the teeing happens ahead of the server"*) | +| **`S` → `ready` → `H`/`D` attach shape** | §C.6, shipped | Same shape Superlogical describes (**Announced**); already implemented including newest-first `H` | +| **Presentation boundary** — host describes state, client's libghostty decides looks | device doc §4 | Learned by shipping the violation; `S` v2 is formatter VT with `palette:false` | +| **SSH-only trust plane; never embed SSH or crypto** | CLAUDE.md #3, §H #8 | A trust choice, not a performance one. Their WebSocket-over-HTTP is a browser-first choice (**Announced**, *"not at all guaranteed to be final"*) — different axis, not a rebuttal | +| **Single writer, many readers; observers never claim** | CLAUDE.md #6, §C.5 | Newest-claim-wins is deterministic (`recompute_writer`, max `seq`) and observable | +| **No nested window manager; layout is a client concern** | CLAUDE.md #5, §H #7 | This is what Grok #4 is reaching for, and it is already law | +| **Device identity = `host_id`, routes are plural** | device doc §2 | Alias-keyed state forks the day the user changes networks | +| **Resource cursor + bounded ring + `gap:true`** | §C.10, `resource.rs` | Generalised reconnect; the terminal plane should adopt the *mechanism*, not rename itself after it | +| **Agent workstream events on the wire** | §C.4, device doc §7 | One of the three rows in device doc §7 that is actually defensible. Untouched by this round | + +--- + +## C. Rejected + +### C.1 The `Block` / `Session` rename (Grok #4) + +Grok proposes `Block` = 1:1 PTY, `Session` = roster of blocks + meta, `Layout` = +client-only. The third clause is already law (§H #7). The first two cost more +than they buy: + +- **`session_id` is on the wire, on disk, and in the CLI.** Renaming touches + `protocol.rs` (`Attach`/`Attached`/`Exited`/every `Event`), `tombstone.rs` + (roster + graveyard files), `resource.rs` scoping, `devices.json`, + `Session.deviceID` on the Swift side, `termio sessions` verbs, and + `session-deep-link.md`'s URL scheme. That is a `proto:2` break for a synonym. +- **"Block" is already taken, twice.** Superlogical's site uses *terminal + blocks* for prompt/output units (**Announced**), and Warp popularised the same + meaning. Adopting it for "one PTY" guarantees a permanent explanation tax in + every conversation with a user who has seen either product. +- **"Session = roster of blocks" is a noun we already have, twice.** A roster + scoped to a machine is the **Device**; a roster scoped to a directory root is + the **Workspace**. Adding a third container is how a host acquires a window + manager one field at a time. + +**Rejected.** The underlying rule — layout never crosses the wire — is kept and +restated in §D.1. + +### C.2 Per-`D`-frame sequence numbers on the wire + +Grok #1's mechanism ("`S@seq`, then `D` where `seq > at_seq`") needs every `D` +frame to carry a cursor, or the client cannot do the filtering the rule +describes. That is 8 bytes and a serialise step per chunk on the one path the +entire design exists to keep free — and it buys nothing, because **the host +already does the filtering** by buffering the attaching client. A cursor the +client cannot act on is a cursor that does not need to be sent. + +**Rejected as a wire field. Accepted as an internal invariant** (D1). If a +future QUIC binding needs a resume cursor (§D.1 `attach {target, last_seen}`), +it is a *stream-scoped* value negotiated at attach, not a per-frame tax. + +### C.3 Sticky writer as a policy (Grok #8) + +"Agent sessions keep their writer" sounds protective and introduces three races +the current rule does not have: + +- **Zombie owner.** A half-open TCP connection (phone through a NAT that stopped + answering) holds the token. `remove_dead` only fires when a send *fails*, + which for a wedged socket can be minutes. The Mac sitting in front of the user + cannot type into their own agent, and no event explains why. +- **Recovery is undefined.** Sticky implies release; release implies a lease; + a lease implies a TTL, a clock, and a steal verb. That is three new protocol + concepts to solve a problem no bug report has yet described. +- **It fights failover.** `recompute_writer` promotes by highest attach `seq`; + sticky needs a second ordering that survives disconnect, so two orderings now + disagree after a crash. + +**Rejected as a default policy. Accepted as an explicit, stateless modifier** +(D5): `attach {mode:"interact", claim:"polite"}` fails with `busy` when a live +writer exists, instead of silently demoting them. No lease, no TTL, no clock — +the caller decides, and the default stays newest-claim-wins. + +### C.4 Compat sink in v1 (part of Grok #2) + +A libghostty in the middle rendering for a non-libghostty terminal (**Announced** +as Superlogical's *compat mode*) is a coherent third client class and a direct +violation of the presentation boundary (device doc §4): the host must resolve +colour for a client that cannot. It also reintroduces the tmux tax *for that +client*, which is tolerable only because it is per-client and off the shared +tee — a nuance worth exactly zero engineering hours until a user asks. + +**Deferred, not refuted.** Named in §D.3 so it has a home; out of scope for v1. + +### C.5 Reusing the word "tombstone" for terminal overflow (part of Grok #5) + +`termiod/src/tombstone.rs` already owns that word: *what a session was when it +died, and why* — a durability record, written to disk, read by the next daemon, +shown in the UI. Superlogical's tombstone (**Announced**) is an overflow marker +meaning "resync, your baseline is gone", which in termio is spelled `gap`. +**Two different words for two different things is correct; unifying the +vocabulary here would collide a shipped concept with a wire signal.** + +--- + +## D. Proposed model + +### D.1 Nouns (unchanged; stated once so the next round does not relitigate) + +``` +Device (host_id) ──< Workspace ──< Session ──? Workstream + │ │ └──< Attachment >── Client + └──< Route └──< Resource (fs: …, cursor + ring + gap) +``` + +One rule, restated because it is what Grok #4 was correctly reaching for: +**anything that describes arrangement — panes, tabs, splits, focus, viewport — +is client state and never crosses the wire.** Superlogical reaches the same +place from the other side (**Announced**: native splits, *one connection per +PTY*); a split there is N attachments, not a host-side layout tree. + +### D.2 Attach and resync — the join point, normatively + +**Invariant (JOIN).** For each attaching client there exists exactly one +boundary B in the session's output byte stream such that: the `S` payload +reflects the terminal state after applying every byte before B, and the client +receives every byte from B onward, in order, exactly once. B is established by +enqueueing `Snapshot` on the sidecar FIFO in the same critical section that +flips the client to `SnapshotPending`. **The PTY is never paused, and no other +client's delivery is affected.** + +Three corollaries, all of which are load-bearing and none of which is currently +written down: + +1. **The sidecar channel must be lossless and ordered with respect to + `Write`.** If bytes destined for the VT can ever be dropped or reordered + (the obvious fix for D4's unbounded queue), B stops existing and `S` silently + describes a screen that never occurred. The permitted degrade is therefore + *never* "drop bytes to the VT" — it is "mark the VT stale and refuse + snapshots" (D4). +2. **Snapshot failure has a defined fallback**, already implemented + (`fallback_snapshot` → ring replay) and unspecified in the protocol doc. + Ring replay is a *lossy* fallback: `RING_CAP` is 128 KiB, so a client + resynced from the ring alone can land mid-escape. It must be reported, not + silent. +3. **Resize is the same barrier, not a second mechanism.** Shipped: + `SessionMsg::Resize` surfaces `TIOCSWINSZ` failure via `reject_resize` before + mutating stored dims, then `Resize` + `Snapshot` land adjacently on the FIFO. + Risks #10 and #11 in §F are stale — both are closed in the POC; their + residuals are D4 and this corollary. + +**The snapshot prologue is the host's.** `S` must be self-contained: applying it +to a client screen in *any* prior state — alt-screen active, scrolling region +set, charset shifted, SGR pending — must produce the same result as applying it +to a fresh terminal. Today the reference client prepends `ESC[2J ESC[H`, which +does none of that, and the Mac and iOS clients are free to prepend something +else. Where exactly the prologue comes from (a libghostty formatter option, or +termiod prepending a scoped reset) is §F.1. + +### D.3 Client classes (Grok #2, accepted as capability profiles) + +Not new nouns — three profiles over caps that already exist. A class is what a +client *negotiates*, and it is renegotiated per attach, not per client identity. + +| Class | Negotiates | Receives | Owns | Loses | Status | +| --- | --- | --- | --- | --- | --- | +| **Replica** (default) | `snapshot`, `scrollback` | `S` → `ready` → `H`* → raw `D` | Its own libghostty; full native scrollback, selection, reflow | Nothing | Shipped | +| **Mirror** | `grid_diff` (requires `snapshot`) | `S` → `ready` → `G`, no downstream `D` | Nothing; the host's grid is the truth | Native scrollback and selection beyond the rows the host chose to send | Shipped, unrecommended (§D.4) | +| **Compat sink** | — | Host-rendered output for a non-libghostty terminal | Nothing | The presentation boundary | **Not built, deferred** (§C.4) | + +The Mac and iOS clients are both **Replicas**. Mirror is a *state a Replica +enters under pressure*, not a device category — which is the substance of +Grok #3. + +### D.4 `G` policy — a pressure valve, never a transport choice + +**Normative: a client MUST NOT select `grid_diff` on the basis of transport +class.** "Remote ⇒ prefer `G`" is wrong twice over: + +1. **Capability.** A Mirror has no real scrollback and cannot select across + history — it holds only the rows the host chose to send. This is the better + argument and it survives any encoding improvement (**Announced**, Mitchell on + why he rejects screen diffs: *"less performance and more making it very + difficult to allow native scrollback, selection"*). It is also why compressing + the wire cell does not promote `G`. +2. **Bandwidth.** Measured 2026-08-05 over SSH to `ukvps`, identical 300-line + scroll: raw `D` 50,423 B / 16 frames against `G` 435,573 B / 18 frames — + **8.6× worse**. Scrolling dirties every row, so dirty-row filtering filters + nothing, and a 16-byte wire cell against ~1 byte of source text is a ~16× + inflation. Secondary evidence; cite it second. + +The two things `G` does buy: a **bound** (cost capped at frame-rate × screen no +matter what the PTY emits, so a `yes` flood cannot melt a metered link) and +**catch-up** (a client behind the window skips intermediate states instead of +replaying them). Legal selection signals: sustained backlog pressure, measured +loss, an explicit user "bounded bandwidth" mode, or a forced resync that would +otherwise drop the client. Illegal: "this connection is remote", "this client is +a phone". + +Superlogical's synced-viewport is the mirror image of the same judgement +(**Announced**): shared scrolling ships as *additional opt-in frames*, never as +the transport. + +### D.5 Writer and resize + +- **Writer: newest claim wins, observable.** Unchanged. Plus `claim:"polite"` + (D5) so a client can ask for the token without stealing it, and `busy` so it + learns why it did not get it. +- **Resize: the writer owns the one PTY size; a resize is a barrier.** Shipped + as described in D.2 corollary 3. +- **Observers letterbox at authoritative dims** (Grok #7). `Attached` already + carries `rows`/`cols` (`protocol.rs:332`, with `serde(default)` for v0 skew) — + the §C.5 note calling this a POC gap is stale. The **residual gap is client + conformance**, not the wire: the reference client still ignores `Resized`, and + a client that parses at its own window size diverges on wrap. That is a + conformance-suite item (PR 7), not a protocol change. + +--- + +## E. Protocol deltas vs `termiod-session-protocol.md` + +| # | Section | Delta | Kind | +| --- | --- | --- | --- | +| **D1** | §C.5 | Add invariant **JOIN** (§D.2) with its three corollaries, replacing the prose "resync: … one `S` snapshot". Explicitly: the PTY is never paused; the sidecar FIFO is the ordering authority; no wire `seq` | Spec only, no code | +| **D2** | §C.6 | The `S` payload **includes its own prologue** and must be state-independent on apply. Clients MUST NOT prepend their own reset. Frame order after `attached` is fixed: `S` → `ready` → `H`* interleavable with `D` | Wire semantics (payload grows) | +| **D3** | §C.6 | Replace the stage table's "who parses VT" column with the **client-class profiles** of §D.3. Stages describe the host's capability; classes describe what a client negotiates | Doc restructure | +| **D4** | §F #10 | Mark risk #10 **closed** (4 MiB `CLIENT_BACKLOG_CAP`, shipped) and open its two residuals: (a) the degrade should be **forced resync** (`S` + `ready` + `E{ev:"resynced", reason:"backlog"}`), dropping only on a second strike; (b) the **sidecar command queue is unbounded** — give it its own budget whose only legal degrade is `vt_stale` (refuse snapshots, fall back to ring replay, emit an event), never dropping bytes to the VT | New risk + wire event | +| **D5** | §C.4, §C.5 | `attach` gains `claim:"newest"|"polite"` (default `newest`, i.e. today's behaviour). `polite` returns `error{code:"busy"}` when a live writer exists. Supersedes the sticky-writer idea | Additive control field | +| **D6** | §C.6 | Normative: `grid_diff` MUST NOT be selected by transport class; legal selection signals enumerated (§D.4). Lead with the capability argument, cite the 8.6× second | Wording, normative | +| **D7** | §C.10 + §C.6 | State the shared mechanism once — *durable object + monotonic cursor + bounded ring + explicit gap signal* — and keep the two words distinct: **`gap`** = your baseline is unusable, resync; **tombstone** = this session is dead and here is why (`tombstone.rs`). The terminal plane adopts `gap`, not the word "tombstone" | Vocabulary | +| **D8** | §C.5 | Delete the stale "POC gap: `Attached` omits `rows`/`cols`" note; move the requirement into a **client conformance list** (parse at authoritative dims; honour `Resized`; honour `WriterChanged`) | Doc correction | +| **D9** | §F #11 | Mark closed — `reject_resize` surfaces `TIOCSWINSZ` failure before mutating dims, and the barrier is implemented | Doc correction | +| **D10** | §C.6 | Specify what a **forced resync owes**: `S` restores the screen, but `H` scrollback is staged at attach only, so a resynced client silently loses history. Either restage `H` on resync or say plainly that it is lost | Open semantics (see §F.4) | + +Not changed by this round: framing, `hello`/caps, the error vocabulary, the +transport table, §D.1's QUIC binding, §H's rejection list. + +--- + +## F. Open questions for a human + +1. **What is the snapshot prologue, exactly?** Does libghostty's formatter have a + "self-contained repaint" mode (emitting the reset it needs), or must termiod + prepend a scoped reset — and which one, given that `RIS` would also clear + things the client legitimately owns? The test is cheap and definitive: apply + one `S` to a client sitting in alt-screen with a scrolling region set, and + diff against the same `S` applied to a fresh terminal. **Blocks D2.** +2. **What should a wedged client cost a healthy one?** Today a slow client is + dropped at 4 MiB. Forced resync is friendlier and unbounded in the pathological + case (a client that cannot keep up will fail resync repeatedly). Drop, resync, + or freeze-and-notify is a UX call: a phone in a tunnel should not be able to + degrade the Mac the user is typing on, and it also should not silently vanish. +3. **Do we speak Superlogical's protocol if it ships as open?** Described as + *"predominantly part of libghostty"* and an *open protocol* (**Announced**). + If both ends are libghostty anyway, a shared wire is plausible — and would + retire most of §C. This is an ecosystem bet with a deadline attached to + someone else's release, not an engineering preference. +4. **Does a resynced client get its scrollback back?** `H` is attach-only today. + After a backlog resync the screen is right and the history is a hole the user + cannot see. Restaging is bounded work (1 MiB cap already exists); saying "lost" + is honest but surprising. +5. **Are legacy terminals a supported client at all?** Compat sink is the only + way `ssh box && termiod attach` works in a non-libghostty terminal. Building + it means the host renders — a deliberate, scoped exception to the presentation + boundary. Not building it means the CLI is a debugging tool, not a product + surface. +6. **Should `polite` be the default for the Mac app?** Newest-claim-wins means + glancing at a session on the phone silently demotes the Mac. That is correct + for a single user with two devices *if* the demotion is visible; it is wrong + the moment sharing exists (§F #3 of the protocol doc). + +--- + +## G. Implementation PR order + +Each step is small, independently shippable, and ordered so the spec lands +before the behaviour it constrains. PRs 1–3 add no features; they make the +current correctness *provable*. + +| # | PR | Contents | Depends on | +| --- | --- | --- | --- | +| 1 | **Spec: join point and vocabulary** | Protocol doc D1, D7, D8, D9. No code | — | +| 2 | **Test: attach during a flood** | Attach a second client mid-`yes`-flood; assert the second client's byte stream, replayed through a VT, is identical to a control capture of the first client's screen at the same boundary. Locks invariant JOIN before anyone optimises the sidecar | 1 | +| 3 | **Test: snapshot applied to a dirty screen** | Golden test for §F.1 — alt-screen, scrolling region, charset, pending SGR. Expected to **fail**; that failure is the spec for PR 4 | 1 | +| 4 | **Host-owned snapshot prologue** | D2. `S` carries its own prologue; `render_snapshot` stops emitting `ESC[2J ESC[H`; Mac/iOS apply raw. Old clients that still prepend a reset stay correct (idempotent) | 3 | +| 5 | **Backlog degrade: resync before drop** | D4(a). Reuse `begin_snapshot_barrier` for one client; add `E{ev:"resynced", reason}`; drop on second strike. Decide D10 here or defer explicitly | 2 | +| 6 | **Sidecar queue budget** | D4(b). Bound the `SidecarCommand` channel by bytes; on exhaustion mark the VT stale, refuse snapshots (existing `fallback_snapshot` path), emit an event. Never drop bytes to the VT | 2 | +| 7 | **Client conformance suite** | D3, D8. Replica and Mirror profiles; skew matrix; assert clients parse at authoritative dims and honour `Resized`/`WriterChanged` | 1 | +| 8 | **`claim:"polite"`** | D5. Additive `attach` field, `busy` error, Mac wiring (attach as observer with a visible badge instead of stealing) | 7 | +| 9 | **`G` policy wording + selection signals** | D6. Wording, plus removing any client-side "remote ⇒ `grid_diff`" heuristic if one has crept in | 7 | +| 10 | **(Conditional) wire-cell compression** | RLE spans, style separated from text. **Only if a Mirror client actually ships** — it does not promote `G`, it only makes the pressure valve cheaper | 9 | + +--- + +## H. Consensus table — Grok refinement → Claude position + +| # | Grok refinement | Position | Why (one line) | +| --- | --- | --- | --- | +| 1 | Attach without pausing the PTY: `S@seq`, then `D` where `seq > at_seq` | **Modify** | Already true and stronger in the POC (per-client buffering, FIFO-ordered snapshot); make it invariant JOIN + a test, not a wire `seq` the client cannot act on | +| 2 | Client classes: Replica / Mirror / Compat sink | **Accept, modified** | Good names — bind them to existing caps as *profiles*, not nouns; Compat sink is deferred because it breaks the presentation boundary | +| 3 | Demote `G`: never the remote/bad-net default | **Accept, strengthened** | Right conclusion, better reason available: a Mirror loses native scrollback and selection, which no encoding fixes; the 8.6× is corroboration, not the argument | +| 4 | Nouns: Block = PTY, Session = roster, Layout = client-only | **Reject (rename); accept (layout)** | `proto:2` break plus a name Warp and Superlogical already spent; the roster nouns exist as Device and Workspace, and layout-is-client-only is already law | +| 5 | Unify tombstone / gap / cursor across terminal + resource planes | **Modify** | Unify the *mechanism* (durable object + cursor + bounded ring + explicit gap); keep the words apart — `tombstone.rs` already means "this session died and here is why" | +| 6 | Per-client byte budget → tombstone (risk #10) | **Modify** | Budget shipped (4 MiB); change the degrade from *drop* to *forced resync*, and add the missing second budget on the unbounded sidecar queue | +| 7 | Wire `rows`/`cols` on `attached`; observers letterbox | **Accept — already shipped** | `protocol.rs:332`; the §C.5 "POC gap" note is stale. Residual is client conformance, not the wire | +| 8 | Optional sticky writer for agent sessions | **Reject (as policy); accept (as claim mode)** | Sticky needs a lease, a TTL, and a steal verb to survive a zombie owner; `claim:"polite"` gets the intent statelessly | +| 9 | Keep SSH trust default; do not copy WebSocket as product default | **Accept, no change** | CLAUDE.md #3. Theirs is a browser-first choice, explicitly not final (**Announced**); different axis | +| 10 | Agent workstream events stay the first-class differentiator | **Accept, no change** | One of the three rows in device doc §7 that the architecture convergence does not touch | + +**Net:** 4 accept, 4 modify, 2 reject — and three findings neither side raised +(the unspecified snapshot prologue, the unbounded sidecar queue, and the +scrollback hole after a forced resync), which are what this round should +actually ship. diff --git a/docs/design/termiod-session-protocol.md b/docs/design/termiod-session-protocol.md index 8a4bbe04..4fe6b0a3 100644 --- a/docs/design/termiod-session-protocol.md +++ b/docs/design/termiod-session-protocol.md @@ -3,13 +3,26 @@ title: termiod Session Protocol status: draft type: design created: 2026-07-30 -updated: 2026-07-30 +updated: 2026-08-04 related: - termiod-session-mux.md - session-daemon-architecture.md - _research-session-protocol-brief.md --- + + + # Design: termiod Session Protocol > One transport-agnostic protocol between attach clients and the `termiod` @@ -36,6 +49,30 @@ roles (control, attach), and three payload planes (control JSON, terminal bytes/diffs, agent events). Every transport — Unix socket, SSH stdio, QUIC, WSS — supplies channels; none of them ever changes a message. +**The core mental model — input replication, not state synchronization.** +A VT parser is a deterministic state machine: the same bytes in produce the +same screen out. So the host keeps every viewer consistent by **shipping the +log** (raw PTY bytes to every client, which each replay through their own +`libghostty`) — *not* by shipping the state (a server-maintained grid diffed +to clients). This is state-machine replication in the database sense: replicate +the input, replay deterministically, and snapshot only to bootstrap a replica +that missed the start. tmux does the opposite — it parses every byte into an +authoritative grid and diffs *that* to clients — which is precisely why a slow +middle emulator throttles the whole pipe. Our load-bearing invariant follows: + +> **Anti-100× invariant: byte delivery MUST NOT block on host-side VT parse.** +> The host tees raw bytes to clients and to its ring the instant it reads them; +> the authoritative VT (v1) is a **sidecar** consulted only to build snapshots, +> off the hot path. Any design that puts a per-frame grid encoder between the +> PTY and the pipe rebuilds the tmux tax and is rejected. + +Empirically confirmed: `termiod/bench/bench_100x.py` measures termiod at +**4.4–6.0× tmux's throughput** on the same byte stream, and — the real tell — +tmux's throughput falls ~50% from plain to ANSI-heavy payloads (a parser is +content-sensitive) while termiod's barely moves (a tee is not). State sync is +not the engine; it is two edge cases (§C.6): a one-shot bootstrap and an +opt-in bad-network degrade. + - **Now (v0.1):** freeze the POC's 5-byte framing; add `hello`/capabilities, request ids, a typed error model, and an event frame kind. Transports: Unix socket (local default), system SSH stdio (remote default). @@ -57,9 +94,10 @@ user already trusts.** ## B. Domain model ``` -Host 1 ──< Session 1 ──? Workstream (workstream = agent overlay, optional) - │ - └──< Attachment >── Client (N viewers per session, over channels) +Host 1 ──< Workspace 1 ──< Session 1 ──? Workstream (workstream = agent overlay, optional) + │ │ + │ └──< Attachment >── Client (N viewers per session) + └──< Resource >── Client (§C.10, resumable, workspace-scoped) ``` | Noun | What it is | Identity | Lifetime | @@ -67,6 +105,8 @@ Host 1 ──< Session 1 ──? Workstream (workstream = agent overlay, | **Host** | One `termiod` process on one machine; owns every PTY and the session table | `host_id`: stable random 128-bit, minted on first run, stored beside the socket | Daemon process (launchd/systemd `--user`); survives all clients | | **Session** | Durable runtime: PTY + process + (v1) vt state + ring/scrollback | `session_id`: host-scoped ULID; `name` is a mutable human alias, never an identity | From `create` until process exit or `kill`. **Detach never ends it** | | **Workstream** | Agent metadata over a session: `agent_id`, project/worktree, `status` (`working·idle·needs_you·done·failed·unknown`), pending approval, title | Same `session_id`; workstream fields are session attributes, not a second object to address | Attached at create or promoted later (agent detection); removed on demotion to plain shell | +| **Workspace** | A directory root on the host that sessions and non-terminal state are scoped to (a project or worktree). Not a container for processes — a *scope* for filesystem, git, and watch state | Canonicalised absolute path; two spellings of one root are one workspace | Implicit: exists while anything references it. Never created or destroyed explicitly | +| **Resource** | Durable host-side state a client subscribes to with a replayable cursor (§C.10): `fs:` in v1, git/status later | `:`; `seq` is its monotonic cursor | Independent of any connection **or** client; lingers past its last subscriber | | **Client** | One viewer/controller endpoint (Mac app, iOS, CLI, tool) | `client_id` per *connection*, assigned by host at `hello`; clients may also send a stable `client_name` for display | Connection lifetime; nothing a client holds is load-bearing | | **Transport endpoint** | Where a host is reachable: `unix:` · `ssh:` · later `quic:` · `wss:` | Resolved by discovery (`HostRef → Endpoint`); opaque to the protocol | Config lifetime | @@ -113,7 +153,8 @@ stay valid. | Resize | `R` | rows u16 BE · cols u16 BE | terminal | v0 | | Event | `E` | JSON object (`ev`-tagged) | events | v0.1 | | Snapshot | `S` | binary: header + packed vt cells (§C.6) | terminal | v1 | -| Diff | `G` | binary: dirty-row grid update (§C.6) | terminal | v1.1 | +| History | `H` | binary: newest-first scrollback rows (§C.6) | terminal | v1 | +| Diff (POC shipped) | `G` | binary: dirty-row grid update (§C.6) | terminal | v1.1 | Rules: unknown *control ops* and *event types* are ignored (additive evolution); unknown *frame kinds* are a protocol error → `proto_error` + @@ -170,6 +211,8 @@ makes one control channel safely multiplexable. Ops marked ✦ exist in POC v0. | `send` ✦ | c→h | Inject bytes without attaching — the `termio sessions send` path, now first-class | | `wait` | c→h | `{target, until:["needs_you","idle","done","exited"], timeout_ms}` → `wait_result`; gives `send --wait` real semantics instead of transcript polling | | `subscribe` | c→h | `{events:["roster","status"]}` on a control channel → stream of `E` frames for all sessions | +| `subscribe_resource` / `subscribed` | c→h / h→c | §C.10. `{resource, since?}` → `{resource, seq, gap}`; replayed `E` frames follow the reply. Gated on the `resources` capability | +| `unsubscribe_resource` | c→h | Release interest; the resource lingers for other subscribers and for this client's own return | | `resize_claim` | h→c | Informs a demoted client who owns size now (§C.5) | | `ok` ✦ / `error` ✦ | h→c | §C.7 | @@ -179,12 +222,28 @@ makes one control channel safely multiplexable. Ops marked ✦ exist in POC v0. 1. channel opens → `hello` exchange (role `attach`) 2. `attach {target, mode, rows, cols}` -3. `attached {session_id, writer:true|false}` +3. `attached {session_id, writer:true|false, rows, cols}` — **must carry the + authoritative PTY dimensions** (see below) 4. resync: v0 = ring-buffer replay as `D` frames; v1 = one `S` snapshot (viewport + cursor + title + a scrollback slice), then live `D` (or `G`) 5. steady state: `D`/`R` up, `D`/`S`/`G`/`E` down 6. `detach` (or channel death — equivalent) → session unaffected +**Every client parses at the authoritative PTY dimensions — this is a +correctness requirement, not a preference.** Input replication is only +deterministic if the replicas run the same state machine on the same input, +and a VT parser's output depends on its *width*: wrap points, `\r\n` handling, +and DECAWM autowrap all key off the column count. An observer whose terminal is +a different size than the PTY will wrap the identical byte stream differently +and diverge — the synchronized-state-machine guarantee silently breaks. So: +`attached` (and the v1 `S` snapshot) **carry `rows`/`cols`**, and a smart client +maintains an internal grid at *authoritative PTY dimensions* with its own +*local* viewport layered on top (letterbox / scale / scroll) — never by parsing +at its own window size. *(POC gap: `Attached` omits `rows`/`cols` +(`protocol.rs`), and the reference client ignores `Resized`/`WriterChanged` +(`client.rs`) — acceptable for a single same-size CLI, incorrect the moment a +second differently-sized viewer attaches.)* + **Writer policy — single writer, newest claim, observable.** Any `mode:"interact"` attach takes the write token; the previous writer stays attached but demoted, and *everyone* on the session gets @@ -203,19 +262,121 @@ viewer per session, which is a nested-window-manager tax in disguise. ### C.6 Terminal plane staging -| Stage | Steady state | Resync on attach/resize | Who parses VT | +The default steady state is **raw `D` bytes** — input replication (§A). A v1.1 +client may instead opt into `G` after bootstrap; state transfer appears in two +roles: a one-shot **bootstrap** for a replica that missed the log, and an opt-in +**bad-network degrade**. + +| Stage | Steady state | State transfer (when) | Who parses VT | | --- | --- | --- | --- | -| v0 | raw `D` bytes | ring replay (`D`) | every client | -| v1 | raw `D` bytes | one `S` snapshot | host authoritatively; clients still parse live bytes | -| v1.1 | `G` dirty-row diffs @ ≤120 fps (per-client cap negotiable) | `S` | host only, for `grid_diff` clients | - -`S`/`G` encodings copy the `libghostty-vt` render-state read-out -(`ghostty-web`'s model): packed 16-byte cells, per-row damage, one snapshot -call — the C struct doubles as the wire cell. The hot path stays cheap: -raw bytes today, frame-capped diffs tomorrow; the vt is a **host-side -authority/sidecar for resync**, never a per-keystroke re-encoder in the -middle of every pipe. v1.1 is where the phone stops needing a full VT engine -and flicker-free reattach becomes structural. +| v0 | raw `D` bytes | bootstrap = ring replay (`D`) on attach | every client | +| v1 | raw `D` bytes | bootstrap = one `S` snapshot on attach/resize/resync | host (sidecar) + every client on live bytes | +| v1.1 | raw `D` bytes, **or** `G` dirty-row diffs for clients that negotiate `grid_diff` | `S` keyframe + `G` deltas | host, only for `grid_diff` clients | + +**When state transfer fires (`S` snapshot triggers) — boundaries only, never +per frame:** (1) **attach** — a new viewer missed the byte log, so bootstrap it +with one state frame, then it follows raw `D`; (2) **resize** — the writer +changes PTY size, a barrier quiesces, resizes, emits a fresh `S`, resumes +deltas; (3) **desync / host restart recovery**; (4) in v1.1 diff mode only, a +**periodic keyframe** to bound drift. Steady-state typing and output never +snapshot — that would be the tmux tax. + +Clients that negotiate both `snapshot` and `scrollback` receive staged history +on attach only. The sidecar captures it at the same in-band boundary as `S`, +keeps at most 1 MiB of encoded rows with the newest rows winning, then emits +small newest-first `H` chunks after `ready` so live `D` can interleave. Resize +snapshots do not restage history; reflow semantics remain a later decision. + +`S`/`G` carry packed cells + per-row damage. **The wire cell is defined by +termiod and is engine-independent — it is NOT the VT engine's in-memory cell** +(corrected 2026-07-31 by the #181 de-risk spike, `termiod-vt-sidecar-spike.md`). +The earlier assumption that "the C struct doubles as the wire cell" is false: +libghostty-vt 1.3.2 exposes render-state cell iteration + per-row dirty tracking +(enough to *build* `S`/`G`) but its cells are **opaque**, with no wire-ready +16-byte packed cell and no one-call viewport snapshot — a conversion step is +required regardless of engine. **v1 engine DECISION (2026-07-31): `libghostty-vt`**, +FFI'd into the Rust host. The spike's build-convenience pick was +`alacritty_terminal`, but that was overridden on a **correctness** ground: every +termio client *is* libghostty (Mac embeds it, iOS mirrors it), and the +"synchronized distributed state machines" model only holds if the host authority +runs the **same** VT — a different engine (alacritty) can diverge on grapheme / +width / autowrap / obscure-escape handling, so its `S` snapshot would not match +what a libghostty client renders. Fidelity parity with the clients is the whole +point (Mitchell's "assume libghostty everywhere"), so we accept the Zig 0.15.2 + +FFI cost. Keep the opaque-cell → wire-cell conversion behind an engine-neutral +boundary anyway. The vt stays a **host-side authority/sidecar for resync**, never +a per-keystroke re-encoder in the middle of every pipe (the anti-100× invariant, +§A). Build path (from the #181 spike): vendor libghostty-vt 1.3.2 + a `build.rs` +that invokes Zig (herdr's pattern), bindgen `ghostty/vt.h`, link the static lib, +cross-compile to aarch64-musl. Phase 0 = an FFI build proof before daemon +integration. + +**The `S` snapshot carries VT sequences, not resolved cells (format v2).** The +host serialises the screen with libghostty-vt's own formatter +(`ghostty_formatter_*`, `emit = VT`) and the client feeds those bytes straight +into its terminal. This is not an encoding preference — it is the boundary the +whole architecture rests on: **the client's libghostty is the style authority, +never the host.** Packed 16-byte cells (v1) forced the host to resolve every +colour against *its* palette, which overrode the viewer's theme, silently +dropped bold/underline (the `attributes` field was reserved-zero) and lost +OSC 8. Concretely the formatter emits `38;5;N` palette indices, so the viewer's +own ANSI colours apply; `palette` (OSC 4) is deliberately **off**, because +emitting it would push the host's colours onto every client. Measured on a +10×40 screen: 559 bytes of VT against 6,504 bytes of cells — 11.6× smaller and +strictly more faithful. + +One caveat worth keeping: the formatter emits the cursor's CUP *before* state +extras, and some extras move the cursor as a side effect (`tabstops` walks the +row with CHA/HTS; DECSTBM homes it). The host re-asserts the true position with +a trailing CUP. + +Format v1 survives **only** for `grid_diff` clients, whose model is explicitly +server-side state and which need cells to seed their grid. + +**v1.1 `G` diffs are the bad-network degrade, not a faster default.** They are +capability-gated (`grid_diff` requires `snapshot`), phone-first, and are the +*same mechanism* as the QUIC state-sync layer in §D.1 — supersedable dirty rows +where a newer row version obsoletes an older one. After `S` + `ready`, a +grid-diff client receives no downstream `D`: each version-1 `G` carries a +monotonic per-session `frame_seq`, authoritative rows/cols and cursor/screen +state, then full 16-byte wire cells for each dirty row. Every 256 damage +flushes by default (test-overridable with `TERMIOD_KEYFRAME_EVERY`), the host +substitutes an ordered `S` + `ready` keyframe and then resumes increasing-seq +`G`. This is mosh's SSP rebuilt on standard transport. +On a good link (LAN, good Wi-Fi) raw byte replication wins outright and no +client should negotiate `grid_diff`; the diff path exists because a *reliable +ordered* byte stream must deliver every intermediate byte in order, which a +lossy high-RTT phone link cannot do cheaply — there, shipping "the latest row +state" is the win. + +**Measured, so nobody re-derives it the hard way** (2026-08-05, framed protocol +over SSH to the `ukvps` aarch64 host, identical 300-line scrolling burst): + +| Plane | Bytes on the wire | Frames | +| --- | --- | --- | +| raw `D` | **50,423** | 16 | +| `G` dirty rows | **435,573** | 18 | + +`G` cost **8.6× more**, not less. Two compounding reasons, both structural: +scrolling output dirties *every* row, so dirty-row filtering filters nothing; +and each cell is 16 wire bytes against roughly one byte of source text, so what +remains is a ~16× encoding inflation. Terminal output is dominated by +scrolling, so this is the common case, not a corner. + +The consequence for transport policy: **"remote ⇒ prefer `G`" is wrong.** `G` +is not a bandwidth optimisation at all. It buys two other things: a *bound* +(cost is capped at frame-rate × screen regardless of how much the PTY emits, so +a `yes` flood cannot melt a metered link) and *catch-up* (a client that has +fallen behind skips intermediate states instead of replaying them). Select it +on backlog pressure or measured loss — never on "this connection is remote". +A worthwhile future change is compressing the wire cell (run-length spans, +style separated from text); at 16 bytes per cell the format, not the idea, is +what makes `G` expensive. The current reliable transport delivers every emitted `G`, +but each `G` already coalesces source bytes into current full-row state; a QUIC +binding may later discard superseded row versions (§D.1). It coexists with the +byte path; it never replaces it. Paired with client-side predictive echo +(§D.1), this is what makes a 100 ms-RTT link *feel* local — the piece no +transport choice (SSH or QUIC) can deliver alone. ### C.7 Error model @@ -271,6 +432,70 @@ The transport rows differ; every frame after them is byte-identical. That is the acceptance test for "transport-agnostic": **a recorded local session transcript must replay verbatim against an SSH-piped host.** +### C.10 Resumable subscriptions (the generalised reconnect) + +The terminal plane already solved reconnect once: a session is a durable object +with an id, a monotonic cursor, a bounded ring, and `S` to bootstrap a replica +that missed the start. **§C.10 makes that one mechanism instead of one +terminal feature**, so every later plane — files, git, agent state — inherits +the same reconnect story rather than inventing its own. Without it, the file +plane grows a bespoke "re-sync on reconnect" path and we have rebuilt the +four-code-paths disease *inside* the host. + +**A resource** is durable host-side state a client observes. It has: + +| Property | Rule | +| --- | --- | +| **Id** | `:`, host-unique and stable across connections. v1 kind: `fs:` | +| **Cursor** | `seq`, monotonic per resource, starting at 1. Never reused, never rewound | +| **Ring** | A bounded replay buffer of recent batches. Overflow is *reported*, never silent | +| **Lifetime** | Independent of any connection **or client**. A watch outlives its last subscriber by a linger window — detach ≠ kill, applied to the resource plane | + +**The one verb:** + +``` +→ subscribe_resource {resource:"/work/termio", since?:41} +← subscribed {resource:"fs:/work/termio", seq:44, gap:false} +← E {ev:"fs_changed", resource:"fs:/work/termio", seq:42, paths:["/work/termio/src"]} +← E {ev:"fs_changed", …, seq:43, git_meta:true} +← E {ev:"fs_changed", …, seq:44, paths:["/work/termio/docs"]} + … then live batches continue from 45 +``` + +`gap:true` means the client's baseline is unusable and it must do a full scan +before applying anything further. It is returned for a first subscribe, for a +`since` that has aged out of the ring, and for a `since` ahead of the host. +**The reply always precedes replayed events**, so a client knows whether to +rescan before it starts applying them. + +**Reconnect is therefore not a feature.** It is: open a pipe, re-subscribe each +resource at the last `seq` you applied. There is no retry ceiling, because a +retry loses nothing — which is the substantive difference from Zed's bounded +`MAX_RECONNECT_ATTEMPTS` and from VS Code's reconnection tokens, which die with +the server PID. Cursors survive the *client*, not just the connection: quit the +Mac app, open the phone, resume the same cursor. + +**Workspace scope.** `fs:` resources are keyed by **canonicalised** root, so two +clients naming one repo differently share a single watcher. This is what keeps +five sessions in one repo at one OS watch rather than five — the failure mode +that exhausts Linux `max_user_watches`. + +**`fs_changed` semantics** (chosen to match the Mac client's existing +`FileTreeWatcher`, so the consumer needs no new model): + +| Field | Meaning | +| --- | --- | +| `paths` | Directories whose listing changed. Re-read only realized ones | +| `full_rescan` | The path set is **not** authoritative — re-walk. Set on watcher overflow (the wire form of FSEvents `MustScanSubDirs`), on watch-limit exhaustion, and on a change storm exceeding the per-batch path cap | +| `git_meta` | Index / HEAD / refs moved → re-read git status. Object-store and packfile churn is dropped host-side and never appears at all | + +Batches are debounced host-side (300 ms quiet window, matching the client's own +FSEvents coalescing) so a `git checkout` publishes one batch, not one per file. + +**Invariant:** resources live on the control plane and are **never** on the +terminal hot path (§A). A resource flush must never delay `fan_out`. Capability +`resources` gates the verb; `fs_watch` gates the `fs:` kind. + ## D. Transport bindings | | Unix socket | SSH stdio | QUIC (later) | WSS + relay (later) | @@ -413,6 +638,9 @@ announced has `needs_you` on the wire. | 6 | **Linux host + libghostty-vt** | v1 bets on `libghostty-vt` building cleanly into the Rust host on Linux (zig cross-compile). De-risk with a spike before committing v1 dates; fallback is any correct VT with damage tracking, at the cost of cell-format alignment. | | 7 | **Event flood vs UI** | Protocol allows high-rate `E`; client discipline (per-session `SessionRuntime`, no roster replace per tick) is already law — see sidebar-scroll-performance. Host also coalesces status transitions (≤ ~10/s per session). | | 8 | **Superlogical ships first and defines expectations** | Their step 1 is an incredible mux (**Announced**). Our counter is not feature racing; it's landing #170–#172 so agents *survive the app* this quarter, with agent events they haven't announced. | +| 9 | **No pipe-mode (non-tty) attach client** | The CLI `attach` assumes an interactive tty; driven non-interactively over a bare SSH channel it delivers **0 bytes** (confirmed 2026-07-31 on `ukvps`), which blocks scripting, piping, and honest WAN-throughput measurement. The protocol already has `mode:"observe"`; the fix is a CLI surface for it (`attach --observe`/`pipe` → raw `D` to stdout, no raw-mode, no stdin capture). Small, and needed before the Mac/iOS clients rely on the same read path. Note the sharper framing (Codex review 2026-07-31): today `remote attach` runs `ssh -t host termiod attach`, so the framed protocol lives *between the remote CLI and the remote socket* and never crosses SSH from a native client — the "same bytes over every transport" claim (§C.9) is **not yet exercised end-to-end**; a non-tty `termiod stdio` bridge is what makes it real. | +| 10 | **Unbounded per-client backlog (the non-blocking hot path's shadow cost)** | The anti-100× invariant makes `fan_out` never block on a slow consumer — but per-client and outbound channels are **unbounded** (`daemon.rs`), so a stalled socket (a wedged phone, a paused SSH client) accumulates raw output without limit until it threatens the daemon. The fix pairs with the `bytes::Bytes` fan-out (single shared chunk, refcounted): give each client a **byte budget / sequence cursor**; when a client falls behind the retained window, **drop it (v0) or resnapshot it (v1)** rather than grow forever. One change closes both the (C+2)×n copy cost and this memory risk. | +| 11 | **Resize is not a barrier; `pty.resize` errors ignored** | `handle_msg(Resize)` (`session.rs`) updates stored dims + emits `Resized` even if the `TIOCSWINSZ` ioctl failed, and a promoted writer after failover keeps stale dims and is not told to reclaim size. v1 must make resize the quiesce → resize → fresh `S` → resume barrier (§C.5) and surface ioctl failure. | ## G. Phased roadmap @@ -437,6 +665,11 @@ impossible retroactively. are not sessions; this was the founding rule. 3. **Freezing raw-PTY forever** — v0 raw is a stage, not the contract; `snapshot`/`grid_diff` capabilities are on the roadmap with dates. +3a. **State sync on the hot path** — a server-maintained grid diffed to clients + as the *default* steady state (the tmux/VNC model). That reintroduces the + middle-emulator parse tax the whole design exists to avoid (measured 4–6× in + `termiod/bench`); `grid_diff` stays an opt-in bad-network degrade (§C.6), + never the default, and never blocks byte delivery (§A invariant). 4. **Public `0.0.0.0` bind / raw TCP + DIY TLS** — Unix socket and SSH only until QUIC arrives with borrowed identity. 5. **CRDT multiplayer typing** — single writer with an observable claim;