diff --git a/.gitignore b/.gitignore index ba8f36d..0e4dc20 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,7 @@ htmlcov/ # Local agent settings .claude/ + +# Local harness / session scratch (never commit) +# Feature mockups / design prototypes — local-only ideas, never tracked +prototype/ diff --git a/README.md b/README.md index bb8b1e7..ae02264 100644 --- a/README.md +++ b/README.md @@ -1,300 +1,356 @@ # ShellPilot -A local-first AI shell harness: one terminal conversation that can answer, inspect your code, plan multi-step work, and run commands under deterministic, risk-based approval. +**A local-first AI shell harness for your terminal.** ShellPilot turns a small local model — served by [Ollama](https://ollama.com), running entirely on your machine — into a careful terminal agent: it reads code, edits files, runs commands, plans multi-step work, and searches the web when you allow it, with every risky action classified deterministically and approved by you before it happens. [![CI](https://github.com/lavindeep/ShellPilot/actions/workflows/ci.yml/badge.svg)](https://github.com/lavindeep/ShellPilot/actions/workflows/ci.yml) [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/downloads/) [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE) -ShellPilot runs a small local model through [Ollama](https://ollama.com) and gives it a tight set of structured tools — file reads, anchored edits, command execution, planning, memory, and optional web grounding. The model proposes; you approve. Command risk is classified by a deterministic policy engine, not by asking the model whether something is safe. By default, nothing leaves your machine: local Ollama only, no accounts, no API keys, no telemetry. Cloud models are opt-in and off by default, gated behind an explicit config switch and a per-session consent prompt. Web access is off by default, and when enabled, every request is approved one at a time. +No accounts. No API keys. No telemetry. Nothing leaves your machine unless you explicitly opt in — and when you do, the UI says so. -It is built for and tested against `gemma4:e4b` — a model that fits on modest hardware — so the design assumes a capable-but-fallible local model and treats recovery as the main loop, not an edge case. +```text +╭──────────────────────────────────────────────────────────────────────╮ +│ scrolling transcript — responses, tool calls, diffs, plan cards │ +│ │ +│ ⏺ run_command pytest tests/test_parser.py │ +│ ✓ 14 passed │ +│ │ +╰──────────────────────────────────────────────────────────────────────╯ +╭─ approve? ───────────────────────────────────────────────────────────╮ +│ ▌ │ +╰───────────────────────────────────────────────────────────────────────╯ + ~/project · gemma4:e4b · balanced · ⎇ main · ● local 11% ctx +``` -## Highlights +*Illustrative sketch of the full-screen app: transcript pane, modal input dock (the border label and color change with what it is asking — amber for a pending approval, red when a high-risk action needs the word `run` typed), and the persistent status bar.* -**Planning you can see and approve.** For multi-step work the model writes a structured plan to a file, shows it to you, and waits. You approve, reject, or ask for a revision before any work runs. As steps complete, the plan file updates, and a finished plan ends with a single clean summary — not the repeated, half-narrated summaries small models tend to emit. +--- -**Deterministic safety and command approval.** Every command is classified by a policy engine that inspects the executable, arguments, shell metacharacters, file targets, the workspace boundary, and known destructive patterns. The classification — and the plain-language explanation of *why* a command is risky — is produced by code, not the model, so the model can never talk a `rm -rf` down to "low risk." Agent commands always run with `shell=False`; the model never gets a raw shell. High-risk commands require you to type `run`. +## Table of contents -**Local-first by default and private.** The default model backend is local Ollama — sessions, memory, logs, plans, and audit trails stay on disk where you can read them. There is no telemetry. Cloud models are opt-in, off by default, and require an explicit config change and per-session consent before anything leaves the device. The single optional source of network egress in a local session — web grounding — is off until you set it in config, and every search and page fetch is individually approved in every profile. +- [Why ShellPilot](#why-shellpilot) +- [Feature tour](#feature-tour) + - [The full-screen terminal app](#the-full-screen-terminal-app) + - [Safety and approvals](#safety-and-approvals) + - [Planning](#planning) + - [Skills](#skills) + - [Web grounding](#web-grounding) + - [Memory, sessions, and context](#memory-sessions-and-context) + - [Cloud models (opt-in)](#cloud-models-opt-in) +- [Getting started](#getting-started) +- [Everyday usage](#everyday-usage) +- [Configuration](#configuration) +- [The security model](#the-security-model) +- [Architecture](#architecture) +- [Development](#development) +- [Platform support](#platform-support) +- [Roadmap](#roadmap) +- [License](#license) -Beyond the spotlight: +--- -- **One conversation loop.** No chat/agent mode switch. Ask a question and it answers; ask for work and it inspects, plans, edits, and runs commands. -- **Read before write.** Edits are anchored to content the model actually read, validated against a content hash, and shown as a diff before they land. -- **Skills with progressive disclosure (new in v0.9.0).** Trigger-driven markdown skills inject only the guidance relevant to the current state; deeper docs are read on demand via a `skill_read` tool instead of bloating every prompt. -- **Web grounding for small models.** When enabled, the model is taught to treat search snippets as leads, fetch the source before asserting a fact, and re-search rather than guess a URL when a fetch fails. -- **Memory with consent.** The model can propose memories; every update shows a preview and needs approval. Stored as plain JSON. -- **Manual shell.** `/shell` opens a clearly-bannered raw shell the model never touches. -- **Resumable sessions and a local audit log.** Conversations journal to disk; `--resume` restores one (active plan included). Approvals, commands, edits, and config changes are recorded as redacted JSONL. -- **Model-free CI.** A fake model client exercises the runtime in tests, so CI needs no GPU or Ollama. +## Why ShellPilot -## How a turn works +Most terminal agents assume a frontier model and a cloud connection. ShellPilot starts from the opposite premise: a **small model running on a laptop** can do genuinely useful shell work — *if* the harness around it is honest about what the model can and cannot be trusted with. -```mermaid -flowchart TD - User([You]) -->|message| Loop[Conversation runtime] - Prompt[System prompt:
base + skills + plan state] --> Loop - Loop -->|prompt + tools| Model[Local model via Ollama] - Model -->|answer| User - Model -->|tool call| Broker[Tool broker] - Broker --> Policy{Policy engine:
risk classification} - Policy -->|needs approval| Gate[Approval gate] - Gate -->|you approve| Exec[Tool / command
shell=False] - Policy -->|auto under profile| Exec - Exec -->|result| Loop - Exec --> Audit[(Audit log)] - Gate --> Audit - - click Loop "https://github.com/lavindeep/ShellPilot/tree/main/shellpilot/runtime" "Conversation runtime" - click Model "https://github.com/lavindeep/ShellPilot/tree/main/shellpilot/llm" "Ollama client" - click Broker "https://github.com/lavindeep/ShellPilot/tree/main/shellpilot/tools" "Tools" - click Policy "https://github.com/lavindeep/ShellPilot/tree/main/shellpilot/policy" "Policy engine" - click Prompt "https://github.com/lavindeep/ShellPilot/tree/main/shellpilot/skills" "Skills" -``` +That premise produces three design rules that shape everything here: + +1. **Determinism where it matters.** Safety, correctness, and control flow are never delegated to the model. The command classifier that decides what needs your approval is pure deterministic code; the explanation you see for a risky command is generated from the classifier's own reasons, not model prose; an edited command always re-enters the classifier. You cannot prompt-inject your way past a policy that never asks the model's opinion. +2. **Local-first as a guarantee, not a default.** The only network endpoint a stock session touches is your own Ollama server on localhost. Web tools are off until you enable them; cloud models are off until you enable them *and* consent per session; both states are visible in the UI at all times. +3. **Finished beats featureful.** Small releases, every behavior specified in [`docs/DESIGN.md`](docs/DESIGN.md) (the spec of record), and a phase gate — lint, format, strict typing, 1,600+ tests — that must be green for every commit. + +The tested baseline is `gemma4:e4b`, a model that fits on modest hardware — so the design assumes a capable-but-fallible model and treats recovery as the main loop, not an edge case. It is also, plainly, a side project and a portfolio piece: a place to work out what careful engineering looks like when the "user" of half your APIs is a small language model. + +--- + +## Feature tour + +### The full-screen terminal app + +Since v0.11.0 ShellPilot opens as a **persistent full-screen app** (prompt_toolkit on the alternate screen) rather than a line-based prompt loop. The layout is a scrolling transcript pane, a framed multi-line **input dock**, and a **status bar** pinned to the bottom (`workspace · model · profile · git branch · locality`, with context usage right-aligned). + +The dock is modal — its border tells you what it is asking: + +| Dock state | Border | +|---|---| +| Normal input | faint rounded frame (accent chevron) | +| Approval pending | amber, labeled `╭─ approve? ─╮` | +| High-risk approval | red, labeled `╭─ type "run" to execute ─╮` | + +Details that make it pleasant to live in: + +- **Type-ahead queue** — keep typing while a turn runs; your next message stages in the dock and fires when the turn ends. Up-arrow recalls a staged message. +- **Model turns run on a worker thread**, so the UI never freezes: streaming, scrolling, and Ctrl-C all stay live mid-generation. +- **Ctrl-C cancels cleanly** — mid-stream it aborts the generation; mid-command it kills the child process group immediately (no waiting out a timeout); during an approval it declines just that action. Conversation history and the on-disk session are both rolled back consistently. +- **Thinking trails** — reasoning models stream their thinking into a live, collapsible trail (collapsed to the first few lines; click to expand). Thinking is display-only; it is never fed back to the model. +- **Slash-command menu** above the dock, **Tab path completion** for `/cwd set`, `/attach`, and `/export`, and **model-name completion** for `/model use` backed by a background-refreshed cache (typing never blocks on Ollama). +- **Click or `Ctrl-O` to expand diffs**; approval previews show the *entire* diff, uncapped — only the copy sent back to the model is truncated to protect the context budget. +- **`!`** runs a one-shot manual shell command with its output captured into the transcript; `/shell` enters a full manual-shell mode. + +The classic line-based REPL is still there behind `--legacy-ui` (or `SHELLPILOT_UI=legacy`) and shares the same renderers, and non-TTY sessions (pipes, scripts) fall back to plain streaming output. + +### Safety and approvals + +Every tool call passes through a **deterministic policy layer** before anything touches your system: + +- **Command classification.** A pure-code classifier assigns each shell command a risk level — LOW, MEDIUM, or HIGH — from its actual structure: the executable, its flags, redirections, path arguments, whether it mutates git state, whether it reads sensitive files. There is no model in this loop. +- **Approval prompts that carry the evidence.** An approval shows the exact command, the *resolved* working directory and paths (anti-spoofing), a deterministic explanation derived from the classifier's reasons, and the full diff for file writes. +- **`[y]es / [e]dit / [n]o`.** `y` approves. `n` declines — a plain decline ends the turn (and pauses an active plan) instead of letting the model immediately try again. `e` is **reject-and-steer**: you describe what should change, and the model's corrected command re-enters the classifier from scratch — an edit can never smuggle a riskier command past the badge it was approved under. +- **HIGH risk means typing `run`.** A destructive command cannot be approved by a reflexive keystroke. +- **Two profiles.** `supervised` asks before every side-effecting action; `balanced` (the default) auto-runs low-risk commands and asks for the rest. Reads of sensitive files (keys, credentials, dotfiles) prompt separately (`allow_sensitive_reads = "ask"`). +- **Workspace boundary.** Tools operate inside the workspace you started in (or moved with `/cwd set`); escaping it is blocked by validation, not convention. + +The wider hardening — output sanitization, secret redaction, egress control, audit — is covered in [The security model](#the-security-model). + +### Planning + +For multi-step work the model proposes a plan through a dedicated `propose_plan` tool: a goal and concrete steps, rendered as a card and approved (or revised, or declined) by you before execution begins. During execution the model records progress with `update_plan`; the transcript tracks step state, `/plan` shows the live plan on demand, and the plan itself persists as an artifact on disk — it survives a `--resume`. + +The plan loop is harness-enforced where it counts: budgets are bounded (`max_plan_steps`, `max_tool_turns`), a stuck plan is finalized deterministically, re-proposing an identical plan is a no-op instead of a double-approval, and exactly one summary ends a completed plan. Declining an action mid-plan pauses the plan at that step rather than silently skipping work. + +### Skills + +Skills are markdown-defined behavior packs with **deterministic triggers** — a skill is active because a concrete condition holds (`ALWAYS_ON`, `ENABLED`, `PLAN_PROPOSED`, `PLAN_ACTIVE`, `PLAN_BLOCKED`, `WEB_ENABLED`), never because the model felt like it. The built-ins: + +| Skill | Trigger | +|---|---| +| `planning` | plan proposed / active / blocked (mode-specific guidance) | +| `context-management` | always on | +| `web-grounding` | web tools enabled | +| `debugging` · `verification` · `code-review` · `git-workflow` · `skill-authoring` | opt-in via `[skills] enabled` | + +Skills support **progressive disclosure**: a lean body is injected into the prompt, and deeper reference docs are exposed through a `skill_read` tool the model calls on demand — validated, name-addressed lookups, never filesystem paths. You can write your own; the `skill-authoring` skill documents the format, and `/skills` shows everything discovered with its triggers, resources, and why it is or isn't active. + +### Web grounding + +Off by default. With `[tools] web = true`, the model gets `web_search` (backed by DuckDuckGo — keyless, no account) and `web_fetch`, plus standing guidance that makes a small model a surprisingly honest researcher: snippets are leads, not evidence — fetch the source before asserting facts; decompose multi-entity questions into separate searches; never invent URLs; if a fetch is blocked, search again for another authoritative source rather than guess; cite what you fetched; admit what you couldn't verify. + +Fetches are hardened: redirects are re-validated hop-by-hop against private and internal addresses (DNS-rebinding resistant), responses are size-bounded, and web requests ride the same approval flow as everything else. + +### Memory, sessions, and context + +- **Sessions** are append-only JSONL transcripts (secrets redacted, file mode `0600`) written incrementally under `.shellpilot/sessions/`. `shellpilot --resume` restores the latest session for the workspace — including an in-flight plan; `/export` renders a session to markdown. Mid-turn corrections (a declined action, a cancelled tool call) are reconciled on disk too, so a resumed transcript never replays half-finished work. +- **Memory** is two explicit stores: global behavior preferences, and per-project preferences and facts (`.shellpilot/memory.json`) — inspected and edited with `/memory show|add|forget|compact`, with model-proposed updates approved before saving. Nothing is memorized silently. +- **`AGENTS.md`** project instructions load on a trust-on-first-use basis: you approve the file once, and again only if its content changes. +- **Context is budgeted, not vibes.** Token budgets derive from the model's real context window; `/context` breaks down usage per block, and selective compaction (`/compact`, or automatic at 70% of budget by default) trims conversation memory while never touching the on-disk transcript. + +### Cloud models (opt-in) -The model talks to a small set of flat-schema tools. Small local models make mistakes, so recovery is designed in: a malformed call gets one schema-reminder retry, repeated failures trigger a roadblock-and-replan protocol, and per-turn tool/turn budgets stop runaways. The model never reaches execution without passing the same deterministic gate every time. +ShellPilot can drive Ollama's `-cloud` models, but treats that as a boundary crossing with a fail-closed gate: `[model] allow_cloud = true` must be set in the config file, **and** each session asks for explicit consent before the first byte leaves your machine — refusing means no HTTP at all. While a cloud model is active, an amber **`☁ CLOUD`** indicator sits in the status bar, driven by the harness's own egress check (never by model output), and `/status` reports locality. Consent and model requests are audit-logged. A default session never egresses, period. -## Quick start +--- -You need Python 3.11+ and [Ollama](https://ollama.com) running locally. Pull the default model: +## Getting started + +**Prerequisites** + +- Python **3.11+** +- [Ollama](https://ollama.com) running locally +- A local model — the tested baseline is **`gemma4:e4b`**: ```bash ollama pull gemma4:e4b ``` -Then clone and install: +**Install** (from source; the runtime dependencies are just `rich`, `httpx`, `platformdirs`, and `prompt-toolkit`): ```bash git clone https://github.com/lavindeep/ShellPilot.git cd ShellPilot -python3 -m venv .venv -source .venv/bin/activate -python -m pip install -e ".[dev]" -shellpilot doctor # check Python, Ollama, installed models, and writable paths -shellpilot # start the conversation in the current directory +python -m venv .venv && source .venv/bin/activate +pip install . ``` -`gemma4:e4b` is the default and primary tested model; the `qwen3.5` family is also recognized as tested. Any installed Ollama model can be selected with `/model use ` — untested models work but print a qualification note. - -## Platforms +**Run** -ShellPilot is developed and tested on **macOS** (Apple Silicon) and is **continuously tested on Linux** — the full test suite runs on `ubuntu-latest` in CI on every commit. Intel Macs are expected to work: it is pure Python and Ollama ships for x86 macOS. - -**Windows is not supported yet.** The deterministic command-safety policy and process control are built on POSIX shell semantics, so a correct Windows port needs a Windows-aware risk classifier and process handling — it is deliberately deferred rather than shipped half-safe. +```bash +shellpilot doctor # checks Python, Ollama, models, and paths +shellpilot # opens the full-screen app in the current directory +``` -## A session +On boot you get a sectioned banner (commands, tips, active workflow skills, recent sessions) and a one-key model picker; `--model NAME` skips the picker. -A representative session in the default `balanced` profile (risk badges are colored chips in the terminal, shown here as text). On launch, ShellPilot v0.10.1 prints a panel banner — a block-art logo alongside Commands, Tips, Workflow-skills, and Recent-sessions sections — then drops to the prompt; the transcript below picks up there: +--- -```text -~/my-project · gemma4:e4b · balanced · ● local 6% ctx -❯ what does this repo do? - - This is a CLI that counts lines in source files. The entry point is - src/cli.py, which calls count_lines() in src/counter.py ... - -~/my-project · gemma4:e4b · balanced · ● local 11% ctx -❯ fix the off-by-one in count_lines and run the tests - - ╭─ Plan · 20260620-141502-fix-off-by-one ───────────╮ - │ Goal: Fix the off-by-one in count_lines and verify │ - │ │ - │ ○ 1 Read src/counter.py to locate count_lines │ - │ ○ 2 Patch the off-by-one in the loop bound │ - │ ○ 3 Run the test suite to confirm the fix │ - ╰────────────────────────────────────────────────────╯ - .shellpilot/tasks/20260620-141502-fix-off-by-one.md - Approve plan? [y]es / [e]dit / [n]o y - - ⏺ read_file(path='src/counter.py') - ⎿ ✓ read src/counter.py - - ⏺ patch_file(path='src/counter.py', …) - - @@ -12,1 +12,1 @@ - - for i in range(1, len(lines)): - + for i in range(0, len(lines)): - MEDIUM tool - CWD: ~/my-project - Approve? [y]es / [e]dit / [n]o y - ⎿ ✓ applied 1 change - - ⏺ run_command(argv=['python', '-m', 'pytest']) - - MEDIUM command · runs arbitrary python code - CWD: ~/my-project - Approve? [y]es / [e]dit / [n]o y - ⎿ ✓ 14 passed in 0.31s - - All steps complete. Fixed the off-by-one (the loop skipped the first line) - and the suite passes. -``` +## Everyday usage -Read-only tools and low-risk commands (`ls`, `git status`, an in-workspace `cat`) run automatically under `balanced`; anything that runs code or writes — including a test run — asks first, and a write shows the diff. A high-risk command is different again — it carries the deterministic purpose explanation and refuses a plain `y`: +**CLI** ```text - ⏺ run_command(argv=['rm', '-rf', 'build/']) - - HIGH command · recursive delete · "Recursively and permanently - deletes the target and everything inside it; this cannot be undone." - CWD: ~/my-project - Type "run" to execute, or press Enter to cancel: +shellpilot [--cwd PATH] [--resume [ID]] [--model NAME] [--legacy-ui] +shellpilot doctor +shellpilot config show|edit ``` -(Glyphs degrade to ASCII automatically under `NO_COLOR`, pipes, or a non-Unicode terminal.) +**Slash commands** (the in-app menu completes these as you type): -## Capabilities - -| Area | What it does | +| Group | Commands | |---|---| -| **Tools** | `read_file`, `list_dir`, `search_text`, `write_file`, `patch_file` (anchored edits), `run_command` (`shell=False`), `env_info`, plus planning (`propose_plan`, `update_plan`), memory (`memory_read`, `memory_propose_update`), and `view_image`. Flat schemas, kept few on purpose — small models degrade as tool count and schema complexity grow. | -| **Security profiles** | `supervised` asks before every side-effecting tool and command; `balanced` (default) auto-runs read-only tools and low-risk commands and asks for writes, installs, deletes, network, and anything risky. | -| **Planning** | Tasks needing three or more steps produce a visible plan file under `.shellpilot/tasks/`, approved before execution, updated as work progresses, with a single end-of-plan summary. | -| **Skills** | Trigger-driven markdown guidance injected only when relevant (a plan is live, web is enabled, always-on, or opted in). Built-ins cover planning, context management, web grounding, and skill authoring. Opt-in workflow skills: `debugging`, `verification`, `code-review`, `git-workflow` — each with a lean injected body routing to deeper on-demand docs via `skill_read`. | -| **Progressive disclosure** | Deeper skill docs are read on demand through the `skill_read` tool rather than injected into every prompt; active skills advertise their readable docs in a one-line menu. New in v0.9.0. | -| **Web grounding** | Opt-in `web_search` and `web_fetch`, off by default, network-approved per request in every profile. The model is guided to fetch sources before asserting facts and to re-search rather than invent URLs. | -| **Memory** | Global and project memory as plain JSON; the model proposes, you approve each change. Secrets are redacted before disk. | -| **Manual shell** | `/shell` opens a raw `shell=True` session the model is not part of; `/exit-shell` returns. `!` runs one command through the same audited path without entering the loop; bare `!` opens the loop. | -| **Image input** | `/attach ` stages a PNG/JPG/GIF/WebP image for the next message when the active model supports vision. | -| **Sessions & audit** | Conversations journal to `.shellpilot/sessions/`; `--resume` restores them (active plan included); `/export` writes markdown. Approvals, commands, edits, and config changes log as redacted JSONL. | - -### Command-line interface - -| Command | Purpose | +| Session | `/help` · `/exit` · `/clear` · `/status` · `/export ` | +| Model | `/model` · `/model list` · `/model use ` | +| Config | `/config show` · `/config edit` · `/config reload` · `/config set` · `/config unset` · `/config reset` | +| Context | `/context` · `/compact` · `/compact status` · `/compact auto ` | +| Planning | `/plan` · `/plan path` · `/plan cancel` · `/plan revise ` | +| Workspace | `/cwd` · `/cwd set ` · `/diff` · `/tools` | +| Safety | `/profile` · `/profile use ` · `/logs` · `/logs all` | +| Memory | `/memory show` · `/memory add ` · `/memory forget ` · `/memory compact` | +| Skills | `/skills` | +| Escape hatches | `/shell` (manual shell mode) · `!` (one-shot command) · `/attach ` (stage an image for vision models) | +| Health | `/doctor` | + +**Keys in the app** + +| Key | Action | |---|---| -| `shellpilot` | Interactive session in the current directory (`--cwd ` to point elsewhere). | -| `shellpilot --resume [id]` | Resume the latest, or a specific, saved session in this workspace. | -| `shellpilot --model ` | Start with a specific model, skipping the boot picker. | -| `shellpilot doctor` | Check Python, Ollama reachability, installed models, and writable paths. | -| `shellpilot config show` | Print the resolved config with the source layer of every key. | -| `shellpilot config edit` | Print the user and project config file paths for hand-editing. | -| `shellpilot --version` | Print the version and exit. | - -### Slash commands +| `Enter` / `Alt+Enter` | submit / insert a newline | +| `Tab` | complete slash commands and paths | +| `↑` | recall the staged (queued) message | +| `PageUp` / `PageDown` / mouse wheel | scroll the transcript | +| click / `Ctrl-O` | expand or collapse a diff (click also toggles thinking trails) | +| `Ctrl-C` | cancel the turn / kill the running command / decline the pending approval | -Inside a session, plain language is the primary interface; slash commands control the harness itself. - -| Command | Purpose | -|---|---| -| `/help` | List all slash commands. | -| `/status` | Model, profile, workspace, and context usage. | -| `/clear` | Clear the visible conversation (with confirmation); also cancels the active plan. | -| `/plan`, `/plan path`, `/plan cancel`, `/plan revise ` | Inspect, locate, cancel, or steer the active plan. | -| `/diff` | Diffs from this session's agent edits. | -| `/model`, `/model list`, `/model use ` | Show the active model; list installed models with tested/untested tags; switch model. | -| `/profile`, `/profile use ` | Show or switch the security profile for this session. | -| `/tools` | List tools available under the active profile. | -| `/skills` | List discovered skills with their triggers, status, active state, resources, and reasons. | -| `/context` | Per-block context breakdown: each system-prompt block with source, token estimate, and injection state. | -| `/compact`, `/compact status`, `/compact auto on\|off` | Compact context now; show usage; toggle auto-compaction. | -| `/memory show`, `/memory add `, `/memory forget `, `/memory compact` | Inspect and curate stored memory. | -| `/config show`, `/config edit`, `/config reload` | Print resolved config; show the config path; reload from disk. | -| `/config set `, `/config unset `, `/config reset` | Set, remove, or clear runtime overrides (persisted in `overrides.json`). | -| `/cwd`, `/cwd set ` | Show or change the workspace boundary. | -| `/logs`, `/logs all` | Recent audit events for this session, or across all sessions. | -| `/export ` | Export this session's transcript to markdown. | -| `/attach ` | Stage an image for your next message; bare `/attach` lists staged images. | -| `/shell`, `/exit-shell` | Enter and leave Manual Shell. | -| `!`, `!` | Run one command through the audited manual-shell path (raw shell, same as `/shell`) without entering the loop; bare `!` opens the Manual Shell loop. | -| `/doctor` | Run the doctor checks from within a session. | -| `/exit` | Exit ShellPilot. | +--- ## Configuration -Config is a user-owned TOML file. ShellPilot never rewrites it — only the program-managed `overrides.json` (set via `/config set`) is self-healing. The file lives in the platform-native config directory via `platformdirs`: on macOS `~/Library/Application Support/shellpilot/`, on Linux `~/.config/shellpilot/`. `shellpilot config edit` prints the exact paths. A `/.shellpilot/config.toml` can override the user file per project. +Configuration is layered, with strict precedence: **env/CLI → `/config set` overrides → project config → user config → defaults**. The user config lives in your platform's config directory (`shellpilot config edit` prints the exact path); a project can add its own `.shellpilot/config.toml`. Your `config.toml` is yours — ShellPilot never writes it, and errors in it are fatal rather than silently patched. Runtime overrides go to a separate program-managed `overrides.json` that self-heals with visible warnings. -Settings resolve highest-wins: CLI flags → a fixed set of `SHELLPILOT_*` env vars (model, color, glyphs) → `overrides.json` → project config → user config → defaults. Egress and safety keys (`tools.web`, `model.base_url`, `model.allow_cloud`, `runtime.security_profile`) have no env override by design — an ambient env var must never enable network egress or downgrade the safety posture. +A representative `config.toml`: ```toml [model] default = "gemma4:e4b" -keep_alive = "5m" # how long Ollama keeps the model warm between prompts - -[model.options] # verbatim Ollama options, passed through untouched -# repeat_penalty = 1.3 # num_ctx is reserved to the context budget and ignored here +reasoning = true # stream model thinking when available +allow_cloud = false # -cloud models refuse to run until this is true [runtime] -security_profile = "balanced" # or "supervised" - -[tools] -web = false # set true to register web_search + web_fetch (always asks) - -[skills] -enabled = ["my-skill"] # user skill folders (and the built-in skill-authoring) to activate +security_profile = "balanced" # or "supervised" +auto_compact = true [privacy] -allow_sensitive_reads = "ask" # ask | never | always — gates reads of .env, .ssh, etc. +redact_secrets = true +allow_sensitive_reads = "ask" [ui] theme = "default" -glyphs = "auto" # auto | unicode | ascii -``` +glyphs = "auto" # unicode | ascii | auto -`[skills] enabled` (and `[model.options]`) are config-file-only: they can only be changed by editing `config.toml`, never via an env var, `overrides.json`, or `/config set`. Egress and safety keys (`[tools] web`, `[model] base_url`, `[model] allow_cloud`, `[runtime] security_profile`) can be set in `config.toml` or via a confirm-gated `/config set` — never silently by an env var. `/config set` shows an amber warning and requires explicit confirmation before persisting any of these to `overrides.json`. User skills live in `/skills//SKILL.md`. ShellPilot also reads behavior instructions from `AGENTS.md`: the one in your config directory (global) is always loaded, while the workspace-root one (project) is gated behind a trust-on-first-use prompt shown the first time it is seen or whenever its contents change. It follows them and never writes those files. +[tools] +web = false # web_search / web_fetch stay unregistered until true -### Cloud models (opt-in) +[skills] +enabled = [] # e.g. ["debugging", "code-review"] +``` -Cloud models are **off by default**. To use an Ollama cloud model (any model whose name ends in `-cloud`, e.g. `nemotron-3-nano:30b-cloud`), add this to your `config.toml`: +Safety- and egress-relevant keys are deliberately hard to move: **none of them can be set by environment variable** (an ambient env var is not a deliberate act), `model.options` and `skills.enabled` are config-file-only outright, and the high-stakes keys (`tools.web`, `model.allow_cloud`, `model.base_url`, `runtime.security_profile`) change at runtime only through an explicitly confirmed, amber-warned `/config set` — with the per-session cloud-consent gate as the real egress boundary regardless. -```toml -[model] -allow_cloud = true -default = "nemotron-3-nano:30b-cloud" -``` +--- -`[model] allow_cloud` has **no environment-variable override** — an ambient env var must never enable cloud egress. It can be set in `config.toml` or via a confirm-gated `/config set` (which shows an amber warning and persists to `overrides.json`); enabling it must be a deliberate act either way. Whichever way it was set, the per-session consent prompt below is the real egress boundary. +## The security model -With `allow_cloud = true`, ShellPilot shows an honest disclosure prompt before any data leaves the device and asks for explicit **y/N** consent (defaulting to **N**). Declining — or running non-interactively — fails closed and the session does not start. Consent is per-session and never persisted; every launch re-asks. +ShellPilot's security stance is **deterministic policy + per-action approval + visible state**, hardened by a dedicated security-audit release (v0.10.0). No OS-level sandbox is used or claimed — instead, the model never gets an unreviewed side effect. The load-bearing pieces: -**What the disclosure covers:** when a cloud model is active, the entire prompt — file contents, command output, memory the model reads — is sent to the provider. Best-effort outbound redaction runs, but it is regex-based and not a confidentiality guarantee (novel secret formats and image data may egress unredacted). The provider's data retention, training, and jurisdiction are outside ShellPilot's control. ShellPilot records a `cloud_consent_granted` event and per-turn `model_request` audit events locally so you can audit what sessions egressed. +- **Deterministic command classification** (see [Safety and approvals](#safety-and-approvals)) — path-qualified executables, git-mutation detection, option-encoded paths, and reader-exec boundaries are all classified structurally; nothing risky rides on model judgment. +- **Terminal output sanitization at every sink.** Model-controlled text (responses, thinking, tool output, plan goals) is stripped of ANSI and control sequences before it reaches your terminal, in both UIs — a model cannot repaint your screen or spoof an approval prompt. +- **Secret redaction** at persistence and egress chokepoints: key-named values, JSON-shaped credentials, and prefixed secrets (`*_API_KEY`, `*_PASSWORD`, …) are scrubbed from transcripts, logs, and outbound requests. +- **Egress control.** HTTP clients run with `trust_env` disabled (no ambient proxies), web fetches re-validate every redirect hop against non-global addresses, and the only default endpoint is your local Ollama. +- **Fail-closed cloud gating** with an unspoofable active-cloud indicator, as described [above](#cloud-models-opt-in). +- **An audit trail.** Session-scoped JSONL audit events (`/logs`) record approvals, executions, consent grants, and model requests; session and audit files are written `0600`. +- **Hardened response parsing.** Malformed Ollama responses — including malformed *streaming chunks* — raise typed errors instead of propagating garbage, and upstream error bodies are never echoed into your terminal. -**Local-first (the default) remains the only full-privacy posture.** +Deliberately out of scope: a malicious local Ollama build, and anything you approve after reading it — the design goal is that you always *get* that reading, with resolved paths and full diffs, before consequences. -The same gate fires if you switch models mid-session with `/model use `: fresh consent is required before anything loads, and on decline the model does not switch. +--- -## Troubleshooting +## Architecture -- **`Ollama API: unreachable`** — Ollama is not serving. Start it (`ollama serve`, or launch the app) and re-run `shellpilot doctor`. -- **`Ollama binary: not on PATH`** — Ollama is not installed. Get it from [ollama.com](https://ollama.com). -- **`Models: none installed`** — pull the default model: `ollama pull gemma4:e4b`. -- **First turn is slow** — the model cold-starts on the first prompt. ShellPilot preloads the selected model at boot and `keep_alive` keeps it warm between turns; subsequent turns are much faster. -- **Web tools missing** — `web_search`/`web_fetch` only register when `[tools] web = true`. Set it in `config.toml`, or via a confirm-gated `/config set tools.web true` (it takes effect next session). There is no env-var toggle for it. -- **A command was rejected before any prompt** — a command that can't start (missing executable, packed shell line, stray shell operator) is rejected deterministically and never spends an approval; correct the arguments and retry. Look for a `did you mean` suggestion. +```mermaid +flowchart LR + subgraph UI["cli/ — two front ends"] + APP["Full-screen app
(app*.py, prompt_toolkit)"] + REPL["Legacy REPL
(terminal.py)"] + end + subgraph CORE["runtime/ — the conversation engine"] + CONV["ConversationRuntime
turn loop"] + EXEC["Executor
approval + dispatch"] + PLAN["Planner
propose_plan / update_plan"] + end + POL["policy/
deterministic risk classifier"] + TOOLS["tools/
read · write · patch · search
run_command · env · images"] + LLM["llm/
Ollama client: streaming,
typed errors, cancellation"] + SK["skills/
triggers + progressive disclosure"] + MEM["memory/
preferences · facts · AGENTS.md"] + PERS["persistence/
sessions · audit"] + WEB["web/
search + fetch, egress guards"] + CFG["config/
layered settings"] + + APP --> CONV + REPL --> CONV + CONV --> EXEC --> POL + EXEC --> TOOLS + CONV --> PLAN + CONV --> LLM + CONV --> SK + CONV --> MEM + CONV --> PERS + TOOLS --> WEB + CFG --> CONV +``` -## Design principles +**A turn, end to end:** your message enters the conversation runtime (in the app, on a worker thread, with every UI update marshaled back to the render loop). The model streams thinking and content; tool calls are validated, classified by the policy layer, gated through approval when required, and executed with bounded output. Results feed the next model step until the turn ends — every step redacted, audited, budgeted against the context window, and mirrored to the session transcript so `--resume` reconstructs exactly what happened. -- **Determinism where it earns its place.** Safety, correctness, and control flow are deterministic — risk classification, the approval gate, anchored read-before-write, pre-flight command checks, plan completion. The model's general capability is not babysat; the harness scaffolds the structure of the interaction, not the model's every output. -- **Fix problems in the harness, not the prompt.** Bad behavior is corrected by deterministic mechanism, never by telling the model "don't do that." -- **Local-first by default, no account.** Local Ollama is the default and recommended backend. State stays on disk; the only optional egress in a local session is per-request-approved web grounding with no keys. Cloud models are opt-in, off by default, and require explicit per-session consent. -- **Small-model focus.** Built and validated on `gemma4:e4b` on an 8 GB machine. Prompts, tool schemas, retries, and grounding guidance are sized for that baseline, with room to dial scaffolding down as models improve. -- **Docs as spec.** [`docs/DESIGN.md`](docs/DESIGN.md) is the spec of record and ships in the same commit as the behavior it describes. +| Package | Responsibility | +|---|---| +| `shellpilot/cli` | both UIs, rendering, theme, slash routing, completions, status bar, doctor | +| `shellpilot/runtime` | conversation loop, executor, planner, events | +| `shellpilot/policy` | command classification and deterministic risk explanations | +| `shellpilot/tools` | tool specs and implementations | +| `shellpilot/llm` | Ollama client: streaming, reasoning, cancellation, typed errors | +| `shellpilot/skills` | discovery, validation, triggers, prompt injection | +| `shellpilot/memory` | preference/fact stores, AGENTS.md trust, redaction helpers | +| `shellpilot/persistence` | session transcripts and the audit log | +| `shellpilot/web` | search provider and hardened fetching | +| `shellpilot/config` | dataclass settings, layered loading, overrides | +| `shellpilot/prompts` | system prompt assembly | + +Design decisions, rationale, and per-release engineering notes live in [`docs/DESIGN.md`](docs/DESIGN.md) — the specification the code is held to. + +--- ## Development ```bash +pip install -e ".[dev]" ruff check . && ruff format --check . && mypy shellpilot --strict && pytest ``` -The same four checks run in CI on Python 3.11 and 3.14, against the fake model only — no GPU or Ollama required. To re-measure a local model's capabilities (tool-call reliability, exact-span reproduction, chaining, stopping): +That four-part **phase gate** (lint, formatting, strict typing across the whole package, 1,600+ tests) is the bar for every commit, and CI runs it on Python 3.11 and 3.14. Tests use a fake model — CI never needs Ollama — and the suite is hermetic against the ambient terminal and color environment. Development is test-first, and behavior changes land in the same commit as their `docs/DESIGN.md` update. -```bash -python scripts/benchmark_model.py --model gemma4:e4b --trials 10 -``` +`scripts/benchmark_model.py` measures what actually matters for a harness model — tool-call format discipline, exact-span reproduction, multi-step chaining, knowing when to stop — and `docs/benchmarks/` holds the runs. Public leaderboard rank has not predicted in-harness behavior, so candidate models are gated on these numbers instead. + +--- + +## Platform support + +| Platform | Status | +|---|---| +| macOS | Primary development platform, tested continuously | +| Linux | CI-validated (full suite on Python 3.11 and 3.14) | +| Windows | Not supported — process-group control (`killpg`) and the POSIX-shell-centric safety policy are real porting work, deferred until they can be done properly | -## Status and roadmap +--- -Current release: **v0.10.1** — post-v0.10.0 hardening and cleanup. Recent milestones: +## Roadmap -- **v0.7.x** — Skills v2 with trigger-driven built-in guidance and read-only resources; instant high-risk approvals generated deterministically from classifier reasons. -- **v0.8.x** — web-grounding quality and hardening for small local models: fetch-before-answer, discover-first query shaping, fetch-recovery, current-generation checks; planner hardening for a single end-of-plan summary and idempotent re-proposals. -- **v0.9.0** — progressive disclosure: a `skill_read` tool and a readable-docs menu let skills carry depth without inflating every prompt. -- **v0.10.0** — security hardening pass (policy tightening, terminal-output sanitization, DNS rebinding guard, audit/session file modes, config-key hardening, AGENTS.md TOFU); opt-in cloud models behind `[model] allow_cloud` with a per-session consent gate, fail-closed non-TTY path, best-effort outbound redaction, `cloud_consent_granted` and `model_request` audit events, honest system-prompt when egressing, and an unmistakable active-cloud indicator (☁ status bar + amber model name + `/status` locality); a persistent status bar (directory · model · profile · locality + context %); four opt-in workflow skills (debugging, verification, code-review, git-workflow); boot banner with cheat-sheet and amber/green model-locality color; streamlined one-key boot picker; reject-and-steer `[e]dit` tool/command approvals; a `!` one-shot manual-shell escape; resolved-path display in approval panels; full-width diff bars; `/config` command consolidation; approval diff scrolling reveal for long diffs. +v0.11.0 is the full-screen UI release. Next, roughly in order of intent: -- **v0.10.1** — post-v0.10.0 hardening and cleanup: a wave of external-review logic and security fixes (classifier escalation of git mutating verbs and option-encoded paths, hard-bounded command output, `trust_env=False` on every HTTP client, broader secret redaction including prefixed and JSON-shaped keys, planner stuck-state and artifact-path fixes, Ollama stream done-sentinel and typed errors on malformed responses, session-id traversal guard, exact-byte web-fetch boundary) plus behavior-preserving internal cleanups. No model-facing behavior change — a default session is unchanged. +- **Skill script execution** under its own safety design (runner, approvals, resource caps) — scripts are currently discovered and validated but never executed. +- **Per-model execution profiles** — dial scaffolding per model instead of hard-coding for the weakest. +- A **`trusted-local` profile** and **`/undo`**. +- **1.0.0**, once the full-screen app has earned it. -Next up is a full-screen TUI input dock (framed input box, queueable input, completion-menu integration), deferred from v0.10.0. Later candidates include controlled skill-script execution under its own safety design, a `trusted-local` profile, and `/undo`. +--- ## License -[MIT](LICENSE) +[MIT](LICENSE) — © Lavindeep Dhillon. diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 8cc4af5..9b09980 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -800,6 +800,11 @@ corrective failure telling the model to apply the change successfully or record with `update_plan(blocker="")`. Pure-analysis steps and failure-then-successful-alternative paths complete freely. +If the user declines an approval-gated action mid-plan with a plain `[n]o`, the turn ends +and the plan is left paused on the active step (`Action declined; plan paused on step N.`); +the user's next message resumes or redirects it. A steered `[e]dit` decline instead +continues the turn so the model can re-propose (section 14.6). + 10. Runtime updates the plan file after each meaningful step. 11. Runtime writes a final outcome summary into the plan file when the task finishes. @@ -1209,6 +1214,7 @@ Policy should inspect: - Workspace boundary. - Known destructive flags. - Network activity. +- Application/URL launchers (`open`, `xdg-open`). - Package manager operations. - Git operations. - Secret-like paths. @@ -1224,6 +1230,7 @@ Examples: | `git commit` | Medium | | `git push` | Medium or high depending on config | | `rm file.txt` | Medium | +| `open file`, `open https://…` | Medium (launches an app/URL) | | `rm -rf path` | High | | `sudo ...` | High | | `curl ... | sh` | High | @@ -1261,6 +1268,8 @@ The boundary must be clear in the UI. Relative paths are resolved against the wo **Display-integrity invariant (v0.10.0).** Every user-facing path display — the tool-call line, the approval-panel head, and the write/patch diff-panel title — is derived from the *same* `resolve_in_workspace` result the tool acts on, never from the raw model argument. A spoofing path (`..` segments, `./x/../y`, symlink, trailing junk) is shown as its resolved, workspace-relative target, so the file the user approves is always the file actually touched; a path that escapes the boundary renders an honest `` marker rather than a fabricated-looking in-workspace path. The single helper `workspace_display(workspace, raw_path)` (`tools/base.py`) is the one display formatter, layered over the canonical resolver — there is no second, divergent resolution. This is a display-only change: the boundary checks and the resolution the action uses are unchanged. +**Live workspace (v0.10.1).** `AppUI` and `TerminalUI` resolve the tool-call path against the *live* workspace via a `workspace_fn: Callable[[], Path]` injected at construction (`lambda: runtime.status().workspace`), so a mid-session `/cwd set` is immediately honoured in the tool-call summary line. The approval-panel path display (`executor._display_value`) was already live; this closes the one remaining surface that used a stale build-time capture. + ### 14.6 Approval Outcomes And Reject-And-Steer An approval prompt has three outcomes, not two — `[y]es / [e]dit / [n]o` (`ApprovalReply`, `policy/approvals.py`): @@ -1271,6 +1280,8 @@ An approval prompt has three outcomes, not two — `[y]es / [e]dit / [n]o` (`App This is *reject-and-steer*, not inline-edit-and-run: the user never edits a command that then executes under the badge it was approved under. **Safety is automatic** because the un-approved action is dropped and the correction is a fresh tool call that is independently classified and re-gated — steering can never smuggle a higher-risk action past the prompt. The outcome is uniform across every approval-gated tool (commands and file writes alike); no per-tool editing logic exists because nothing is edited in place. +**A plain decline ends the turn; a steer continues it.** The declined outcome carries `stop_turn = not bool(steer)` (`runtime/executor.py`). On a plain `[n]o` the tool loop truncates any remaining tool calls in that reply, records the declined result, and returns without re-invoking the model — so a declined action cannot be silently retried later in the same turn. When a plan is active, the decline surfaces a status line `Action declined; plan paused on step N.` (the active step index, via `_active_plan_step`) and leaves the plan paused for the user's next instruction. A steered `[e]dit` decline does **not** stop the turn: `stop_turn` is `False` whenever steer text is present, so the correction is fed back and the loop continues so the model can re-propose. + **The HIGH-risk typed-`run` confirm is unchanged.** A HIGH-risk command still requires typing the literal `run` to execute; `[e]` is an additional option at that same prompt that steers without running, and Enter still cancels. Empty guidance after `[e]` is treated as a plain decline (nothing runs). A steered approval is audited as `decision="steered"` on the `approval` event (alongside `approved`/`rejected`). ## 15. Privacy @@ -2038,6 +2049,8 @@ Commands: | `/attach` | List currently staged images, or report "No attachments staged." *(v0.5.0)* | | `/skills` | List all discovered skills with root, trigger declarations, enabled/builtin/disabled/invalid status, decision-derived active state, resource/script summaries, skip reasons, and advisory warnings. *(v0.7.0)* | +In the full-screen input dock, `/model use ` has local model-name completion analogous to the filesystem path completion offered for `/cwd set `, `/attach `, and `/export ` (`cli/path_completion.py`). The menu is backed by a thread-safe in-memory cache seeded from the startup Ollama `/api/tags` result and refreshed once in a daemon background thread; typing never calls Ollama or blocks the prompt loop. The dispatcher still re-validates the selected model against Ollama when `/model use` runs, so stale completions cannot switch to a missing local model, and cloud/remote consent behavior remains in the command handler. Enter accepts and submits a highlighted model completion, while path completions remain Tab-only so Enter on `/cwd set ` submits the typed command into the normal confirmation flow. In app mode, `/model use` routing depends on the target: a switch to a **local** model runs on the worker thread with its success/error output captured straight into the app pane, while a switch to an **egressing** (cloud) model takes the real-terminal handoff — so the cloud-consent prompt and slow preload can run safely — with its output copied back into the pane afterward (`model_use_needs_terminal` in `cli/terminal.py`, `app_slash._model_use_can_run_in_pane`/`_model_use_can_copy_terminal_output`). + All commands scheduled for v0.3.0 (memory, prefs, compact auto, export) shipped. `/prefs` was retired in v0.10.0 — it had converged with `/memory show`, which now lists preferences with their (scope, source) and prints the store paths. `/context show` (a redacted dump of the assembled prompt) is deliberately deferred. The `/context` status table does not fully solve prompt inspection — it shows per-block sizes, not the prompt's actual text — but a verbatim dump must not leak secrets. When `show` lands it must reuse the v0.5.2 redaction helpers before printing any block content. @@ -2405,7 +2418,7 @@ Most rows here concern the v2 memory system. The prompt-injection and secret row | Model emits malformed tool call | Reject tool call, show compact error, and allow one retry. | | Model loops tool calls | Enforce turn/tool budgets and replan or stop. | | Reasoning mode unavailable | Per-model fallback: add the model to `_no_think`; retry once without `think`; other models keep sending `think`. `_reasoning` (the config-level flag) is never mutated. | -| Reasoning-only turn (think text, empty content) | The streamed `thinking` field is now accumulated and captured on the reply message (it is never echoed back to the API and never rendered), so a turn that reasons and then emits nothing is observable in the audit log rather than silently empty; the runtime nudges such a reply once the model has already run a tool (section 10.4). | +| Reasoning-only turn (think text, empty content) | The streamed `thinking` field is accumulated and captured on the reply message. It is never echoed back to the API; it may be surfaced live to the UI when a consumer is wired via `on_thinking` (`conversation` passes `self._ui.stream_thinking`). The default full-screen `AppUI.stream_thinking` is the real consumer — it drives the live reasoning readout and inline thinking trail (section 31.19); the legacy `TerminalUI` (opt-out via `--legacy-ui`) keeps it a no-op so a legacy session stays byte-identical. A turn that reasons then emits nothing is observable in the audit log; the runtime nudges such a reply once the model has already run a tool (section 10.4). | | Stream ends without `done` sentinel | Rejected as incomplete: `_stream_chat` raises `OllamaResponseError` after the `with` block closes without seeing a chunk whose `done` field is truthy. Legitimate early stops (`done_reason = "length"` or `"load"`) still carry the sentinel and are accepted normally. | ### 24.7 Privacy And Log Edge Cases @@ -2443,7 +2456,7 @@ The rebuild should stay light. The goal is a reliable local harness, not a frame | Memory system | Behavior/project memory, proposals, and optimization move to v2. V1 only reads `AGENTS.md`. Scheduled for v0.3.0 (settled 2026-06-11). | | Token-budget compaction | V1 uses oldest-first truncation; selective compaction is v2. Scheduled for v0.3.0 (settled 2026-06-11). | | `trusted-local` profile | Deferred from v1, and deferred again at the 2026-06-11 v2 scoping. Revisit for v3. | -| Session resume | Shipped in v0.3.0 (settled 2026-06-11): append-only JSONL transcripts at `.shellpilot/sessions/.jsonl`, written incrementally with secrets redacted; compaction trims memory, never the transcript. `shellpilot --resume [id]` restores the latest (or named) session's history; snapshots are never restored, so read-before-write forces fresh reads. `/export` renders the transcript to markdown. Tool-call arguments are redacted recursively (matching the audit log's `_redact_value` logic, now unified in `redact_structure` in `shellpilot/memory/redaction.py`) before they reach the JSONL transcript; `/export` inherits redaction by re-reading the transcript from disk. Fixed in v0.5.2. `session_markdown` re-applies redaction at export time so transcripts written before v0.5.2 (which may contain raw secrets on disk) cannot leak through `/export`; on-disk history is deliberately left untouched. Fixed in v0.5.2 review wave. Plan state now also restores on `--resume` (v0.6.0): an `active_plan` pointer in the transcript is read at boot; if the referenced plan sidecar is live (`proposed`/`active`/`blocked`), `PlanManager.restore` reinstates it (section 11.3). **Read-side traversal guard (v0.10.1):** `SessionStore.find` now rejects any session id whose resolved parent differs from the sessions directory, closing the `--resume ../../../../etc/x` path-traversal vector; the write path was already safe via `path.stem`. | +| Session resume | Shipped in v0.3.0 (settled 2026-06-11): append-only JSONL transcripts at `.shellpilot/sessions/.jsonl`, written incrementally with secrets redacted; compaction trims memory, never the transcript. `shellpilot --resume [id]` restores the latest (or named) session's history; snapshots are never restored, so read-before-write forces fresh reads. `/export` renders the transcript to markdown. Tool-call arguments are redacted recursively (matching the audit log's `_redact_value` logic, now unified in `redact_structure` in `shellpilot/memory/redaction.py`) before they reach the JSONL transcript; `/export` inherits redaction by re-reading the transcript from disk. Fixed in v0.5.2. `session_markdown` re-applies redaction at export time so transcripts written before v0.5.2 (which may contain raw secrets on disk) cannot leak through `/export`; on-disk history is deliberately left untouched. Fixed in v0.5.2 review wave. Plan state now also restores on `--resume` (v0.6.0): an `active_plan` pointer in the transcript is read at boot; if the referenced plan sidecar is live (`proposed`/`active`/`blocked`), `PlanManager.restore` reinstates it (section 11.3). **Read-side traversal guard (v0.10.1):** `SessionStore.find` now rejects any session id whose resolved parent differs from the sessions directory, closing the `--resume ../../../../etc/x` path-traversal vector; the write path was already safe via `path.stem`. **Reconciliation records:** the transcript stays append-only, so mid-turn corrections are records rather than rewrites — on load, `replace_last_message` replaces the last *assistant* record (a mid-batch decline truncates the reply's remaining tool calls, section 14.6) and `truncate_last_turn` deletes from the last assistant record to the end (a mid-tool cancel, section 31.15); a record of either kind with no assistant message present, or an unknown record kind, is ignored. | | Agent raw shell | Do not expose `raw_shell` as an agent tool in v1. Keep Manual Shell for direct user-controlled `shell=True`. | | Capability packs (Skills v2) | v0.6.0 shipped instruction-only SKILL.md discovery; v0.7.0 extends it with deterministic trigger selection, four markdown-only builtins, read-only references/templates, script manifest discovery without execution, and enriched `/skills` + `/context` visibility (section 23). | | Capability packs (heavier: tools/handlers/permissions) | Design later after core tools are stable. v3 candidate (2026-06-11). | @@ -2717,13 +2730,16 @@ Monochrome hierarchy on the user's terminal background — the app never sets it | Style | Value | Used for | |---|---|---| -| Emphasis | bold, bright white | Banner title, tool names, current plan step | +| Emphasis | bold, bright white | Banner title, tool names, current plan step; the shell command under approval (`sp.cmd`, `run_command`) | | Body | terminal default foreground | Conversation text, approval questions | -| Dim | `#6b6b6b` | Machinery: tool args, results, context line, reasons | +| Value | `#cfcfcf` | Approval stat-block values (`sp.value`): the why / effect / cwd — readable, never buried in gray | +| Dim | `#6b6b6b` | Machinery: tool args, results, context line, stat-block labels (`sp.label`) | | Faint | `#444444` | Panel borders, turn stats, ellipsis markers | -| Accent green | `#98c379` | Prompt chevron, success ✓, plan checks, diff additions | -| Red | `#e06c75` | High risk, diff removals, errors ✗ | -| Amber | `#e5c07b` | BLOCKED badge, context-usage warning | +| Accent green | `#98c379` | Prompt chevron, success ✓, plan checks, diff additions; approval `[y]es` (`sp.choice.yes`) | +| Red | `#e06c75` | High risk, diff removals, errors ✗; approval `[n]o` (`sp.choice.no`) and the typed-`run` confirm | +| Amber | `#e5c07b` | BLOCKED badge, context-usage warning; approval `[e]dit` (`sp.choice.edit`) | + +The v2 style pass (`feat/ui-style-pass`) keeps this palette — no new hues — and spends contrast and weight on the *actionable* surfaces (command under approval, the y/e/n choices, the stat-block values), while secondary machinery (reasoning trails, metadata, labels) stays dim. Color remains a meaning system: green = yes/success/local, amber = edit/warning, red = no/high-risk. Colors are truecolor values; rich downgrades automatically on 256/16-color terminals. All named styles live in one `rich.theme.Theme` in `cli/theme.py` — no inline hex anywhere else. @@ -2740,23 +2756,31 @@ Input is provided by `prompt_toolkit`: persistent up-arrow history (state dir `h ### 31.3 Activity lines -Tool calls render as `⏺` + bold tool name + dim `(args) · summary`. Results and continuations indent under a dim `⎿`, with a green `✓` or red `✗`. Command output streams dim-indented under `⎿`; when display truncation applies, a faint `… +N lines` marker is shown (capture and audit limits are unchanged from section 13). +A tool call renders by its **primary subject**, not a `name(args)` repr, so the thing that actually happens is the thing you read (`tool_call_block`, cli/render.py). Three shapes: + +- **Framed** — the consequential, approval-gated actions: `run_command` (the joined `argv` command) and `web_fetch` (the URL that egresses). The line is `⏺` + bold action label (`run command`, `web_fetch`) followed by a rounded `Panel` holding the actual subject in bright `sp.cmd` — impossible to miss. A low-risk command that auto-runs is framed the same way (the line never knows in advance whether it will be gated), which is acceptable: `run_command` is the one tool whose exact command always deserves prominence. +- **Inline** — tools with one clean subject: `⏺` + bold name + two spaces + the readable (`sp.value`) subject. `read_file`/`list_dir`/`view_image`/`write_file`/`patch_file` show the `path`, `web_search` the `query`, `search_text` the `pattern`, `skill_read` the `resource`. +- **Generic** — everything else (`env_info`, the memory tools): the legacy `⏺` + bold name + dim `(key=value, …)` summary, capped so a chatty argument set can't run off the side. + +A `path` subject is the **resolved, workspace-relative target** the handler acts on — never the raw, possibly spoofing argument (display-integrity, section 14.5) — via the injected `path_display`; secrets are redacted (`redact_structure`) and control chars sanitized before any of the three shapes render. Results and continuations indent under a dim `⎿`, with a green `✓` or red `✗`. Command output streams dim-indented under `⎿`; when display truncation applies, a faint `… +N lines` marker is shown (capture and audit limits are unchanged from section 13). ### 31.4 Diffs -Diffs render in a rich `Panel` titled with the filename: line-number gutter, full-line subtle red/green backgrounds for removals/additions, and brighter word-level highlight spans on the changed words. A changed line is always laid out the conventional unified-diff way — the old text on its own red removal row, then the new text on the next green addition row (removals before additions within a hunk), each carrying its own line-number gutter; the two are never paired onto one visual line. Every removal/addition fills its colored background to a uniform width (the widest changed-line content in the diff), so each renders as a distinct full-width bar rather than a short colored fragment. Word-level highlighting applies only when a removed/added line pair is similar (`difflib.SequenceMatcher` ratio >= 0.5); pure additions/removals get the full-line background only. Long lines wrap with the background following the text and a blank gutter on continuation rows. Tabs are expanded and control characters sanitized before rendering. +Diffs render in a rich `Panel` titled with the filename: line-number gutter, full-line subtle red/green backgrounds for removals/additions, and brighter word-level highlight spans on the changed words. A changed line is always laid out the conventional unified-diff way — the old text on its own red removal row, then the new text on the next green addition row (removals before additions within a hunk), each carrying its own line-number gutter; the two are never paired onto one visual line. Every removal/addition fills its colored background to a uniform width so each renders as a distinct full-width bar rather than a short colored fragment: `render_diff(..., width=W)` standardizes the panel to a fixed width `W` and fills bars to its inner width (the app passes the pane width so every diff window is the same size); without a width (the legacy UI and `/diff`) bars fill to the widest changed-line content. Word-level highlighting applies only when a removed/added line pair is similar (`difflib.SequenceMatcher` ratio >= 0.5); pure additions/removals get the full-line background only. A line wider than the panel folds onto continuation rows (the colored background follows the text, blank gutter on the continuation) instead of running off the side. Tabs are expanded and control characters sanitized before rendering. -When a file write/edit is proposed, the approval diff appears with a brief scrolling reveal before the prompt: `DiffReveal` (cli/streaming.py) opens a transient `Live` (`vertical_overflow="crop"`) on the main thread and progressively shows the diff rows, scaling the chunk size so the whole reveal finishes within a fixed `TOTAL_DURATION` (~0.6 s) regardless of length — a huge diff adds no real latency. It clears the region before stopping (same clear-before-stop as `ResponseStream.finish`) so a tall reveal never leaks into scrollback. The reveal rides the same motion toggle as the spinner (`[ui] spinner`): it is a no-op when motion is off, on a non-terminal, or for a short diff (at or below `ANIMATE_THRESHOLD = 20` rendered rows), and it never starts until the spinner's own `Live` has fully stopped — two concurrent `Live` on one Console would corrupt the display. After the reveal, the settled panel prints. A long diff settles into a bounded window of `WINDOW_ROWS = 24` rows with a single `… (+N more)` footer (`render_diff(..., max_rows=...)`, `sp.faint`, mirroring `output_truncation`); a short diff prints in full unchanged. The approval semantics are unchanged: the diff is shown, then the y/n (or typed-`run`) prompt, then the write applies only on approval. The source diff is already capped upstream at `MAX_PREVIEW_LINES = 60` (tools/patch.py), so the window/footer is a second display layer on an already-bounded preview. +When a file write/edit is proposed, the approval diff appears with a brief scrolling reveal before the prompt: `DiffReveal` (cli/streaming.py) opens a transient `Live` (`vertical_overflow="crop"`) on the main thread and progressively shows the diff rows, scaling the chunk size so the whole reveal finishes within a fixed `TOTAL_DURATION` (~0.6 s) regardless of length — a huge diff adds no real latency. It clears the region before stopping (same clear-before-stop as `ResponseStream.finish`) so a tall reveal never leaks into scrollback. The reveal rides the same motion toggle as the spinner (`[ui] spinner`): it is a no-op when motion is off, on a non-terminal, or for a short diff (at or below `ANIMATE_THRESHOLD = 20` rendered rows), and it never starts until the spinner's own `Live` has fully stopped — two concurrent `Live` on one Console would corrupt the display. After the reveal, the settled panel prints. A long diff settles into a bounded window of `WINDOW_ROWS = 24` rows with a single `… (+N more)` footer (`render_diff(..., max_rows=...)`, `sp.faint`, mirroring `output_truncation`); a short diff prints in full unchanged. The approval semantics are unchanged: the diff is shown, then the y/n (or typed-`run`) prompt, then the write applies only on approval. The approval **preview** diff (`spec.preview` → `unified_diff(..., max_lines=None)`, tools/patch.py) is rendered in full — every changed hunk, uncapped — so the reviewer can see the whole change (and expand it in the app, §31.16); only the diff returned to the MODEL in a tool result keeps the `MAX_PREVIEW_LINES = 60` cap, since that one rides the context budget. The display window/footer (legacy `WINDOW_ROWS`; app collapse) is therefore the only bound on how much of a large preview is shown at once. ### 31.5 Approvals -Badge blocks: an inverse chip anchors the request, followed by the dim reason or deterministic purpose, then the question. +Approvals are the focal control (v2 style pass). The action itself renders framed and bright in the tool-call line directly above (§31.3) — the command for `run_command`, the URL for `web_fetch` — so the approval block does not repeat it; an inverse risk-badge chip anchors the block, followed by a **stat block** — muted fixed-width labels (`WHY` the action is gated, its `EFFECT` from the deterministic purpose, the `CWD`) with readable values (`sp.value`) so the reason and consequence can't be buried in gray — then the colored choice line. The labels are deliberately limited to what the harness knows truthfully: there is no `WRITES`/read-only row because the classifier's coarse side-effect signal cannot honestly assert "read-only" for an arbitrary side-effecting command, and a fabricated safety claim is worse than none. The stat block replaces the old single dim `kind · reasons · purpose` line; the shared builders `approval_info` / `approval_cwd` produce it for both the app pane and the legacy REPL. + +In the **app pane** the block arrives as one **approval card** (`approval_card`, cli/render.py): a rounded panel composing the same shared builders (badge + WHY/EFFECT, then CWD) whose border carries the decision color — amber for any gated action, red when the risk is HIGH — so the badge and stat rows read as a single decision zone instead of loose flush-left lines. The choice line stays outside the card (the gate re-prompts it on its own on an unparseable answer). The legacy REPL keeps the flat stat block unchanged. - ` MEDIUM ` — white on gray `#3a3a3a`. - ` HIGH ` — white on red `#c14949`; keeps the typed-`run` flow, with `"run"` in red. - ` BLOCKED ` — black on amber; used by the roadblock protocol (section 11.6). -The approval question is `Approve? [y]es / [e]dit / [n]o` — uniform lowercase. It accepts `y`/`yes`/`n`/`no` case-insensitively and **Enter means no**; default-deny semantics are unchanged. `[e]dit` rejects-and-steers: it does not run the action but prompts `Tell the model what to do instead:` for one line of guidance fed back to the model (section 14.6); empty guidance is a plain decline. A HIGH-risk command keeps its typed-`run` gate, with `[e]dit` offered alongside (`Type "run" to execute, [e]dit to steer, or press Enter to cancel`). When color is unavailable, chips degrade to plain `[MEDIUM]` text. +The approval question is `Approve? [y]es / [e]dit / [n]o`, with the choices carried as a meaning system — `[y]es` green, `[e]dit` amber, `[n]o` red — so the actionable answer is unmistakable. One shared builder (`approval_choices` for commands/tools, `plan_choices` for plans) feeds both the app pane and the legacy input prompt, so the colors can't drift between UIs; the literal tokens are unchanged, so the deterministic input parsing is untouched. It accepts `y`/`yes`/`n`/`no` case-insensitively and **Enter means no**; default-deny semantics are unchanged. `[e]dit` rejects-and-steers: it does not run the action but prompts `Tell the model what to do instead:` for one line of guidance fed back to the model (section 14.6); empty guidance is a plain decline. A HIGH-risk **command** keeps its typed-`run` gate, with `"run"` in red and `[e]dit` in amber (`Type "run" to execute, [e]dit to steer, or press Enter to cancel`); a HIGH-risk **tool** (a sensitive read) keeps the standard y/e/n prompt. When color is unavailable, chips degrade to plain `[MEDIUM]` text and the choice tokens render uncolored. ### 31.6 Plans @@ -2766,6 +2790,8 @@ The proposal gate is a `Panel` titled `Plan · ` containing the goal and ` Model responses render as rich Markdown, updated live while tokens stream (`rich.live.Live` with overflow cropping). On completion the live region is replaced by exactly one final clean render, so scrollback always holds one perfect copy — including when the response is taller than the terminal and live repaints span multiple screens; the implementation ensures no repaint can leak lines into scrollback above the final print. Non-TTY output falls back to plain text streaming. +Response markdown follows the §31.1 palette rather than Rich's defaults, which paint inline code `bold cyan on black` and headings magenta — the black chip breaks the never-set-a-background rule on any terminal that isn't pure black. The theme re-pins the `markdown.*` styles (inline code bold accent-green; headings bold bright-white; rules/borders faint; links accent with dim URLs; block quotes dim), and every response sink builds its renderable through the one shared `response_markdown(text)` (cli/render.py) — sanitized, with fenced code rendered via the `ansi_dark` syntax theme (ANSI colors on the terminal's own background, never monokai's painted fill). The app pane's committed and in-progress responses and the legacy REPL's live stream and final render all route through it, so code rendering cannot drift between UIs. + ### 31.8 Status and stats - While the model works: an accent-colored aviation spinner — a compact-glide plane (`✈···` / `·✈··` / `··✈·` / `···✈` / `····`) gliding across a 4-cell track in `sp.accent` green, followed by a dim flight-phase phrase and elapsed seconds. It erases itself before the first token prints and is Ctrl-C safe (never leaves a stray line). Disable with `ui.spinner = false`; auto-disabled when not a TTY. @@ -2819,7 +2845,121 @@ A one-line status bar is pinned at the input as `prompt_toolkit`'s `bottom_toolb - **Right:** `% ctx` — context utilization, color-coded **green** (`< 50`) → **amber** (`50–79`) → **red** (`>= 80`). - **Cloud emphasis:** when egressing, the bar's separators carry a faint amber tint (the prototype's amber "wash" adapted to the terminal) so the whole line reads as cloud at a glance, distinct from a local (faint-grey) bar — alongside the amber model name, the `☁ CLOUD` locality, and the amber chevron. -`status_bar(...)` (`cli/status_bar.py`) is a **pure** builder mirroring `cli/banner.py`: it takes `workspace`/`model`/`profile`/`is_cloud`/`ctx_pct` and returns a `prompt_toolkit` `FormattedText`, with no I/O. Every field is sourced from runtime/session state (`runtime.status()`), never from model output. The `is_cloud` field is the caller's real `is_egressing(runtime.model, base_url)` value (recomputed each REPL iteration on the live model), keeping the active-cloud indicator unspoofable and harness-rendered (§15.2). `ctx_pct` reuses the runtime's own utilization formula via `ctx_percent(used, total)`. The workspace path is user-controlled, so it is run through the shared `_sanitize_line` control/ANSI strip before render (§36.2) — a crafted directory name cannot repaint the bar. The bar consolidates two readouts that previously printed separately each loop: the cloud indicator (was a bold-amber header line) and context utilization (was a per-turn stats line, §31.8). +`status_bar(...)` (`cli/status_bar.py`) is a **pure** builder mirroring `cli/banner.py`: it takes `workspace`/`model`/`profile`/`is_cloud`/`ctx_pct` and returns a `prompt_toolkit` `FormattedText`, with no I/O. Every field is sourced from runtime/session state (`runtime.status()`), never from model output. The `is_cloud` field is the caller's real `is_egressing(runtime.model, base_url)` value (recomputed each REPL iteration on the live model), keeping the active-cloud indicator unspoofable and harness-rendered (§15.2). `ctx_pct` reuses the runtime's own utilization formula via `ctx_percent(used, total)`. The workspace path is user-controlled, so it is run through the shared `_sanitize_line` control/ANSI strip before render (§36.2) — a crafted directory name cannot repaint the bar. The bar consolidates two readouts that previously printed separately each loop: the cloud indicator (was a bold-amber header line) and context utilization (was a per-turn stats line, §31.8). An optional `branch` segment (`⎇ `, ASCII `git: `) inserts after `profile` when the host supplies one; it is sanitized like the path, and omitting it leaves the toolbar caller's output byte-identical. + +### 31.12 Full-screen app shell (v2, in progress) + +The interactive REPL is migrating from a per-turn `prompt_toolkit` `prompt` to a persistent full-screen `Application` on the alternate screen (`cli/app.py`, `build_app(...)`), so the status bar and input dock stay pinned and never scroll away mid-turn. The layout is a scrolling chat pane on top, a custom rounded multi-line **input dock**, and the persistent status bar (§31.11) pinned at the bottom. The dock border is **hand-drawn** (`horizontal_border(width, box, top=...)` — rounded `╭─╮`/`╰─╯`, ASCII `+`/`-` fallback) rather than `prompt_toolkit`'s `Frame`, which only draws square corners and reserves completion-menu height inside the box (filling the terminal height instead of hugging the input); this is the one deliberate exception to the §31.9 "borders from rich primitives" rule, since the dock lives in a `prompt_toolkit` layout. The dock border width, the pane wrap width, and the terminal width are one shared value read at render time, so a resize re-derives the border with nothing cached. Enter submits (a literal newline is Alt+Enter); the dock grows to a 10-row cap then scrolls. The pane **follows the bottom** — auto-scrolling as a response streams in — by exposing its last line as the `FormattedTextControl` cursor (`get_cursor_position`), which prompt_toolkit scrolls to keep visible every render *regardless of focus*; a bare cursorless control would otherwise default to `(0,0)` and snap the pane to the top. PageUp pins an earlier line (so a reader is not yanked down when new output appends), and PageDown moves back toward the bottom — reaching it resumes following. (Mouse-wheel scroll-back is deferred to branch 9: the wheel nudges the window's own `vertical_scroll`, which the cursor-follow re-derives on the next render.) The pane now renders Rich content via `AppUI` (`cli/app_ui.py`), which implements the `RuntimeUI` protocol. `AppUI` holds a `list[RenderableType]` as the transcript source of truth, with an optional in-progress open-response accumulator (accumulated token text rendered as `rich.Markdown`). `AppUI._render_ansi()` renders the renderables to a single ANSI string at the shared terminal width via a `Console(force_terminal=True, width=W)`; the result is cached by width so a resize re-derives it automatically (cache miss) when the pane's `FormattedTextControl(lambda: ANSI(ui._render_ansi()))` renders. `wrap_lines=False` on the pane window lets Rich own all wrapping. Secret redaction (`redact_structure`) and display-integrity path resolution (`workspace_display`) are applied in `show_tool_call` before the summary reaches the pane. Approval methods raise `NotImplementedError` until branch 7 wires the focus-swap; no silent default is acceptable there. `AppUI` never calls `get_app()` or `invalidate()` — those are branch-4 concerns — so it is fully testable without a running app. The non-TTY `PlainInput` path (§31.2) is unchanged. + +The user echo (`show_user_message`) carries the brightness hierarchy rather than a flat tint: the chevron renders `sp.chevron` (bold accent) and the message text `sp.emph` (bold bright-white), so the user's own words read as content, not green harness machinery. A blank spacer line is appended above the echo whenever the transcript already has content, giving each turn visible breathing room; the first message into an empty pane gets no leading blank. + +### 31.13 Worker-thread turn + loop-thread marshaling (v2, in progress) + +`ConversationRuntime.run_turn` drives the UI synchronously on the calling thread, calling `self._ui.` for every event. In the full-screen app that thread is the prompt_toolkit event loop, so a long model turn would freeze the whole UI. Branch 4 (`cli/app_turn.py`) runs the one synchronous turn on a **worker thread** and marshals every UI callback back onto the loop thread, so `AppUI` state and the pane repaint are only ever touched on the loop thread. + +`ThreadedUI` is a `RuntimeUI` wrapping the real `AppUI` (`inner`) and an injected `schedule: Callable[[Callable[[], None]], None]`. Every fire-and-forget content method (`stream_token`, `stream_thinking`, `begin_response`/`end_response`, `turn_finished`, `show_status`, `show_error`, `show_tool_call`/`show_tool_result`, `show_command_output`, `show_plan_progress`) enqueues the inner call via `schedule(functools.partial(inner.method, *args))` — `functools.partial`, never a closure over a loop variable, so args bind at call time and cannot be rebound before the queued call runs. The blocking approval methods return a value, so they cannot be fire-and-forget: `ask_approval`/`ask_plan_approval` delegate straight to the inner UI and run on the worker thread; the inner `AppUI` still raises `NotImplementedError` (the focus-swap `Future` handshake is branch 7), and the worker's `try/except` surfaces that as a pane error — no approval ever silently defaults. + +`TurnRunner` owns the single worker thread for one turn and a `busy` flag. The core safety property: `busy` is SET in `start` (the dock-submit handler, which runs on the loop thread) and CLEARED in `_mark_done` (scheduled back onto the loop thread from the worker's `finally`), so it is only ever read or written on the loop thread — no lock needed. `start` ignores a submit while `busy` (single model, single conversation, single worker; the one-message queue is branch 9). The worker body `_run` touches nothing on the loop thread directly — it only calls `conversation.run_turn` (whose UI is the marshaling `ThreadedUI`, so every UI call is already marshaled) and routes its own error/completion through `schedule`; a `run_turn` exception is rendered as a pane error instead of silently killing the daemon thread, and the `finally` clears `busy` so a crashed turn never wedges the app (Ctrl-C turn cancellation is branch 6). `schedule` is injected so CI drives it synchronously; the real app uses `TurnRunner.schedule`, which reads `app.loop` lazily and schedules (via `loop.call_soon_threadsafe`) a callback that runs `fn()` then `app.invalidate()` (fails closed when the app is not running). prompt_toolkit coalesces the per-call `invalidate()` into one render tick, so per-token marshal+invalidate is correct without throttling. + +The runnable entry is `run_app` (`cli/app_main.py`). **The full-screen app is the default for an interactive TTY**; the legacy line-based REPL is opt-out via `--legacy-ui`, the `SHELLPILOT_UI=legacy` env var, or any non-TTY (piped/redirected) session, which the app cannot drive (`app_mode = tty and not (legacy_ui or SHELLPILOT_UI=="legacy")`). The legacy REPL shares the app's renderers when selected — tool-call blocks, approval choices, and the `response_markdown` sink are the same builders — so both UIs present one visual language; it remains the supported line-based path for non-TTY and SSH use. The construction cycle — `schedule` needs `app`, `app` needs `runner.start`, `runner` needs the conversation, the conversation needs the `ThreadedUI`/`schedule` — is broken by build order and deferred attribute assignment: `AppUI` is built first (its `width_fn` reads `get_app()` lazily), then `TurnRunner` (its `schedule` reads `runner.app` lazily), then `ThreadedUI` over them; the conversation is constructed **with that `ThreadedUI` as its UI** (so its plan tools capture the marshaling bound methods — not repointed after the fact); then `build_app(on_submit=runner.start, ui=app_ui, …)`; then `runner.app`/`runner.conversation` are set after the app exists and before `app.run()`, by which time they are only read once a turn runs. The UI choice is made in `run_interactive` where the conversation is built (a `ThreadedUI` for app mode, the same `TerminalUI` as before otherwise). The boot banner is built once and used by both UIs: app mode seeds it as `AppUI`'s first transcript renderable (`intro=`) so it shows inside the alt-screen pane (a `console.print` would be lost behind the full-screen app); the legacy REPL `console.print`s the same `Panel`. `session_end` is audited identically in both modes — `run_app` stays UI-only and `run_interactive` writes the event right after the app loop exits, mirroring the legacy loop. One known minor gap remains: a turn still in flight when `/exit` clears `app.loop` drops its final `busy`-clear in the already-dead app (harmless while the process is exiting). + +### 31.14 Turn-scoped thinking indicator (v2) + +The full-screen pane shows a **live frontier line** while a turn runs, so a long (especially cloud-reasoning) turn is visibly alive instead of a blind spinner. The indicator is **turn-scoped**: it spans the whole tool loop, from the turn's first model call to the end of the turn, and never resets on a per-tool-call boundary. + +**Placement (it moves down).** The active line always renders LAST: `AppUI._render_ansi` builds the render list as `[*renderables, , ]`. Completed content (responses, tool calls) is appended to `_renderables`, so it lands ABOVE the live line, which therefore "moves down" the transcript as the turn progresses and ends at the bottom. Because the pane auto-follows the bottom (§31.12), the frontier line stays visible the whole turn, then freezes in place. + +**State machine.** `AppUI` holds a `_TurnIndicator | None` (`start`, `reasoning_chars`) and an injected clock `time_fn` (defaults to `time.monotonic`; tests pass a fake). `begin_response` starts the indicator **only if one is not already active** (records `start = time_fn()` once); a later `begin_response` in the same turn — the next tool-loop iteration — is a no-op for the indicator, so the timer, phrase, and reasoning count carry across tool calls (this is the deliberate departure from the v0.10.1 AviationSpinner, which `start`/`stop`ed per model call and reset every tool call). `end_response` does NOT touch the indicator — it only closes the open response, so the indicator keeps running across the tool call into the next model call. `stream_thinking(text)` adds `len(text)` to `reasoning_chars` while active (so the count climbs only while the model thinks, freezing when it stops and resuming if it thinks again). ONLY `turn_finished` freezes the indicator: it appends a permanent `✓ done` line and clears `_indicator`. + +**Lines.** Live: `{plane} {phrase}… {N}s · {fmt(reasoning)} reasoning`, where the plane is `Glyphs.spinner_frames` advanced one cell every `FRAME_SECONDS` (≈0.15s) of elapsed time, the phrase is a DETERMINISTIC pick from the current flight phase's pool (reusing `cli/streaming.py` `phase_for_elapsed`, indexed by 10-second buckets — not random, so it is testable), `N = int(elapsed)` with `elapsed = time_fn() - start`, and `reasoning` is the chars→tokens estimate `ceil(reasoning_chars / CHARS_PER_TOKEN)` (consistent with `budget.estimate_tokens` — an estimate, not exact). Done: `✓ done · {N}s · {fmt(reasoning)} reasoning · {fmt(total)} total`, where `N = int(stats.elapsed_s)` (the runtime's authoritative elapsed, not the UI clock) and `total = stats.output_tokens` (the exact summed `eval_count`). `fmt` is a k/m abbreviation (`999`→`"999"`, `1800`→`"1.8k"`, `2_400_000`→`"2.4m"`). + +**Reasoning gate.** The reasoning/total readout is gated on `settings.ui.show_reasoning_summary` (default true; previously a dead flag — this is its consumer), threaded into `AppUI(show_reasoning=…)` at both construction sites (`build_app`, and `terminal.py` app mode passing the setting). When false, the live line is `{plane} {phrase}… {N}s` and the done line is `✓ done · {N}s` (plane/phrase/timer only). This is display-only; audit capture of thinking is unaffected. + +**Animation, repaint, and caching (perf).** There is **no** `refresh_interval`: prompt_toolkit's built-in one starts a single background task that invalidates on a fixed timer *forever*, redrawing the static transcript even when idle — CPU the old REPL never spent. Instead `run_app` runs a **gated** background coroutine that invalidates ONLY while `AppUI.is_animating` (a turn is in flight), so the plane glides and the elapsed timer ticks between thinking chunks during a turn, while an idle app schedules no timer redraws at all. Streaming and scrolling are unaffected by dropping the timer because they already repaint explicitly: `TurnRunner.schedule` calls `app.invalidate()` after every marshaled UI callback, and input (keystroke / PageUp / wheel) repaints through prompt_toolkit. While an indicator is active, `_render_ansi` bypasses the width cache (elapsed changes every render). The pane additionally caches the **parsed** transcript: `build_app`'s `_pane_view` holds the `(string, ANSI, line-count)` for the last `_render_ansi` result and reuses it while that string is unchanged (an identity compare — `_render_ansi` returns the same cached string object when content+width are unchanged). This matters because constructing `ANSI(...)` re-parses the whole transcript (it parses in `__init__`) and `FormattedTextControl`'s fragment cache is keyed by the per-render counter, so without `_pane_view` every redraw — including every scroll tick — re-parsed the entire transcript, which was the scroll lag. `AppUI` stays pure state+render — the clock is injected and the repaint is the app's job, so it never calls `get_app()`/`invalidate()`. A turn that errors before `turn_finished` (or a Ctrl-C cancel, §31.15) leaves the indicator active; as a minimal cross-turn guard, `show_user_message` discards any such dangling indicator at the start of the next turn, so an errored turn never poisons the following turn's timer (without it, the next `begin_response` would be a no-op against the stale indicator and count from the old start). + +### 31.15 Model-stream cancellation (v2) + +Ctrl-C aborts a model turn mid-stream — the motivating case is a long invisible thinking phase (§31.14) the user wants to stop. The cancellation contract is deterministic and history-safe. + +**Ctrl-C is a key press, not a SIGINT.** The full-screen app runs in raw mode (ISIG disabled), so `c-c` is an ordinary key binding. `build_app` takes `on_interrupt: Callable[[], bool] | None`; the `c-c` handler is `if on_interrupt is not None and on_interrupt(): return` (a turn was cancelled) `else: show_idle_hint(_IDLE_HINT)` (idle). The idle hint is **deduped** (`AppUI.show_idle_hint` no-ops while `_idle_hint_shown`, re-armed at the next turn via `show_user_message` or a `/clear`), so repeated Ctrl-C while idle shows the hint once rather than stacking it. `run_app` wires `on_interrupt=runner.request_cancel`. + +**Cancel event, set on the loop thread, observed on the worker.** `TurnRunner.start` creates a fresh `threading.Event` per turn and passes it into `run_turn(text, cancel=…)`. `request_cancel()` (loop thread) returns True and `Event.set()`s it when `_busy` and a cancel exists, else False — the only cross-thread signal (the app only ever *sets* the event; thread-safe). `ConversationRuntime.run_turn` reassigns `self._cancel = cancel` at the top (so a stale event can never leak across turns) and the tool loop passes it to every `self._llm.chat(...)`. + +**Reader thread + worker drain — responsive even on a hung read.** A naive "check the event at the top of `for line in response.iter_lines()`" only observes cancel *between* chunks: a hung cloud read (no chunk arriving) blocks the loop until the connection read timeout, so Ctrl-C did nothing during a stall (observed live: a ~115 s hung cloud call that ignored Ctrl-C until it 502'd). So `OllamaClient._stream_chat` splits the work: a daemon **reader thread** owns the entire `with self._client.stream(...)` block — it opens, `iter_lines()`, and closes the response **all in-thread** (the response is therefore NEVER closed cross-thread; `response.close()` cross-thread is still avoided), pushing each line onto a `queue.Queue`. This thread — the worker — runs `_drain_stream`, which `queue.get(timeout=_STREAM_POLL_SECONDS)` (0.1 s) and on each poll checks `cancel`: a Ctrl-C aborts **instantly** even when the read is hung, because the worker is waiting on the queue, not the socket. The orphaned reader (a daemon) finishes on its own at the connection read timeout. The reader ALSO breaks its `iter_lines` loop when `cancel` is set, so a still-flowing stream stops being pulled early. `GenerationCancelled` (defined in `llm/client.py`, deliberately NOT an `OllamaError` subclass) propagates cleanly out through `chat()` (and through the think-retry, which threads `cancel` into the retried call) to the runtime. A reader **terminal error** (status/transport/timeout) is captured into a one-slot box and surfaced by the worker AFTER the drain loop, *before* the cancel check — so a think-unsupported 400 still reaches the retry rather than being masked by a set cancel. + +**Inactivity cap + leak-free errors.** The drain also enforces a worker-side inactivity cap: if no line arrives within `generate_timeout_seconds` (default 300 s, = the connection read timeout, so no regression; the clock is injectable for tests), it raises a typed `OllamaTimeoutError` rather than hanging — and a real `httpx.TimeoutException` in the reader maps to the same type. Error *messages* are sanitized at the source: a `>= 400` response keeps only `Ollama API error ` in the exception (with `status_code` set), stashing the raw body in a non-displayed `detail` field for the internal think-unsupported check — a cloud gateway body carries internal IPs / infra JSON that must never reach the pane. The pure `describe_turn_error(exc)` maps the typed errors to a friendly one-liner (timeout → "stopped responding"; unreachable → "is it running? `ollama serve`"; gateway 502/503/504 → "unavailable or timed out (HTTP …)"; unknown → the class name only, never `str(exc)`). + +**History integrity (load-bearing).** `run_turn` records the user message *before* the tool loop, then lets `GenerationCancelled` PROPAGATE — it never catches it, records a partial reply, or calls `turn_finished`. The model call raises at the `reply = self._llm.chat(...)` site, so the subsequent `self._record(reply)` is skipped: a cancelled turn leaves the user message in history but NO partial assistant reply. The partial is discarded, not truncated-and-stored. + +**Worker: clean abort vs failure.** `TurnRunner._run` catches `GenerationCancelled` SEPARATELY from `Exception`: cancel → `schedule(inner_ui.abort_turn)` (a clean abort, not a "Turn failed" error); any other exception → `schedule(inner_ui.fail_turn, describe_turn_error(exc))`. `fail_turn` (app-side) tears down the dangling live indicator (`self._indicator = None`) THEN surfaces the friendly, leak-free error — without it, a network failure left the thinking indicator running on screen (the cancel path cleared it, the error path did not). The `finally` schedules `_mark_done` on EVERY path, so a failed/cancelled turn clears `_busy` and the next submit works — the app never wedges. + +**`AppUI.abort_turn` (app-side, not a RuntimeUI method).** Clears the dangling live indicator (`self._indicator = None`) then appends a `⏹ aborted` marker (ASCII fallback `Glyphs.cross`, style `sp.warn`). `_add_renderable` closes the open response first, so the partial streamed text stays visible — finalized — with the marker below it. This is display-only; the history discard is the runtime's. + +**Subprocess cancellation (branch 6b — killpg).** A cancel requested while a tool *command* is running now kills that child immediately, instead of waiting out its timeout. The per-turn cancel event threads through `ToolContext` (`base.py`) to `run_command_process`, whose wait loop polls it on a `_POLL_SECONDS` (0.1 s) interval and `killpg`s the child's process group the instant the event is set — the worker owns its own `Popen`, so it kills its own child (no cross-thread registry; the `Event` stays the only cross-thread signal). The tool loop then raises `GenerationCancelled` keyed off the cancel `Event` (the general check fires for any tool, immediately after `executor.execute`, before the plan-guard bookkeeping), routing through the same `abort_turn` (`⏹ aborted`) path as the model-stream cancel. Before raising, it rolls this model step out of history (`del self._history[history_before_reply:]`) so no orphaned tool_call — an assistant `tool_call` with no matching result — is re-sent on the next turn; this matches the model-stream cancel, which never records its partial reply. The persisted transcript is reconciled the same way: a `truncate_last_turn` record is appended before the cancel propagates (the reply was already written to disk when the tool loop began), so a later `--resume` drops the same step instead of re-sending the orphan. Prior completed steps stay, and partial command output already streamed to the pane stays visible. The `cancel=None` legacy path (the default REPL executor) is behaviorally equivalent to before — same `exit_code`/`output`/`timed_out`/`truncated`; the poll loop merely chunks the wait (a runaway child is still killed at the timeout deadline, within one poll interval). + +### 31.16 Approval focus-swap (v2) + +Approvals are the exception to the fire-and-forget marshaling of §31.13: `ask_approval` / `ask_plan_approval` RETURN a value, so they cannot be enqueued and forgotten. The full-screen app resolves them with a **focus-swap handshake** (`cli/app_approval.py`, `ApprovalGate`): the worker thread blocks on a `concurrent.futures.Future` while the loop thread renders the prompt into the pane and the dock becomes the approval input. + +**Worker blocks, loop thread answers.** `ThreadedUI.ask_approval` delegates to `gate.ask_command(request)` (and `ask_plan_approval` → `gate.ask_plan`). The gate schedules a loop-thread setup thunk (via the same `TurnRunner.schedule` marshaler) and then calls `future.result()`, which blocks the worker. The thunk renders the prompt block — `AppUI.show_approval` (the collapsible diff window, the risk/reason info line, the CWD line) or `AppUI.show_plan_approval` (the plan panel + the sanitized artifact path) — and arms `gate._pending`. Both `Future` operations are thread-safe stdlib primitives, so no lock is needed; `_pending` is touched only on the loop thread (the setup thunk arms it; the dock keybindings read+clear it). + +**The diff is a collapsible window.** `show_approval` appends a stateful `_Diff` (not a one-shot panel) to the transcript, so it re-renders at the live pane width on every frame (`_render_diff`): standardized width, long lines folded (§31.4). Collapsed it caps at `DiffReveal.WINDOW_ROWS` (24) rows with a `… (+N more)` footer plus a `click or ctrl-o to expand` hint; expanded shows every row. A diff that already fits the cap gets no hint (nothing to toggle). Toggling is by **click** on the diff (the shared line→element index, §31.19) — which reaches any diff in scrollback — or, as a keyboard fallback for terminals without mouse reporting, **`Ctrl-O`** (with *no active approval*) flips the **latest** diff's `expanded`. `total_rows` is captured once when the diff is appended, so the hint decision never re-parses per frame. + +**Three-way outcome, unchanged contract (§14.6).** The dock's submit keybinding routes the typed line to `gate.submit(line)` when `gate.active`, BEFORE the `/exit` check (a mid-approval `/exit` is an approval answer, not a quit). The pure helpers `parse_command_choice` / `parse_plan_choice` carry the contract: a HIGH-risk **command** executes only on the literal `run` (the typed-`run` gate); every other request takes y/e/n; an unrecognized plan token re-prompts. `[e]dit` enters a steer phase — the next line becomes `ApprovalReply.steer_text` (empty = plain decline), which re-enters the deterministic classifier as a re-proposal exactly as in the REPL; the un-approved action never runs. The accepted line is echoed into the pane (sanitized, via `show_status` — never `show_user_message`, which would reset the live turn indicator). + +**The dock is modal, and it says so.** While a prompt is active the dock itself changes state instead of relying on the pane scroll: the border and side bars take the decision color (red when `dock_risk` is HIGH, amber for any other prompt — the same hue as the §31.5 approval card), the chevron follows, and the top border embeds a short state label (`horizontal_border(..., label=...)` → `╭─ approve? ───╮`; dropped whole when the width cannot fit it, never on the bottom border). The gate exposes the state as two loop-thread-only properties read per render: `dock_risk` (the request's risk; None for a plan prompt) and `dock_hint` — `approve?` for a y/e/n command/tool prompt, `type "run" to execute` for a HIGH command, `approve plan?` for a plan, and the steer/revision phases update it to `tell the model what to do instead` / `describe the changes you want`. Both are set by the `_enter` thunks, updated on phase transitions inside `feed()`, and cleared on every resolution path (submit-done, cancel, setup/feed exceptions), so the dock can never keep a stale approval color after the prompt resolves. The chevron is deliberately NOT keyed on `is_busy` — a render must never consume a busy read that the submit keybinding's stage-or-dispatch decision depends on. + +**Ctrl-C / EOF decline THIS action.** During an approval the worker is blocked on the Future, not in a model-stream read, so the §31.15 model-cancel path would not fire. The `c-c` and `c-d` handlers therefore check `gate.active` FIRST and call `gate.cancel()` (resolve as decline — `DECLINE` for a command, `("n", "")` for a plan), mirroring `TerminalUI.ask_approval`'s `except KeyboardInterrupt: return DECLINE`. The turn continues; turn-level cancel is available again once the prompt returns. + +**Shutdown ceiling.** A pending approval at app exit leaves the daemon worker blocked on `future.result()`; harmless (process exit reaps the daemon worker). A cleaner shutdown would resolve pending approvals as DECLINE on exit — a future refinement, not a correctness issue since the process is already terminating. + +### 31.17 Slash-command routing (v2) + +A typed `/...` or `!...` line is a harness control, not a model turn. The dock's submit keybinding (after the §31.16 approval-gate check and the `/exit` quit) routes any such line to the `SlashRouter` (`cli/app_slash.py`) via the `on_slash` callback, instead of sending it to the model. `/exit` stays a direct quit; a normal line still starts a turn. The router splits the work by what would **hang** the loop. + +**The invariant is "never hang", not "never run synchronous code".** The event loop must not block on anything that can stall for a *user-perceptible* or *unbounded* time — a model call or a network call (a hung Ollama would otherwise freeze the TUI for the client timeout, with Ctrl-C dead because it is a keypress the blocked loop never reads). It runs fast, bounded, local work inline like any keybinding: rendering, and the small-file reads behind display commands (`/logs`, `/memory show`, `/config reload`, `/export`) — the same synchronous handling the default REPL ships, completing in sub-millisecond-to-low-millisecond time with no hang. So the dividing line for off-loop execution is *network/model*, deliberately not *all I/O*. + +**Fast display commands run on the loop thread.** A non-interactive slash command that does only in-memory work or a small local-file read (`/help`, `/status`, `/context`, `/logs`, `/memory show`, `/config reload`, …) runs against a fresh pane-capturing `rich.Console` (force-terminal, truecolor, at the current pane width); the captured ANSI is parsed back with `Text.from_ansi` and appended to the pane by `AppUI.show_slash_output`. No suspension, no terminal hand-off. + +**Interactive / slow / own-stdout commands run via `run_in_terminal`.** A command that calls `confirm()`, prompts for cloud consent, prints to its own `Console()`, or does slow preload work (`/model use`) must reach the real terminal. `slash.needs_terminal(line)` classifies these deterministically (`/shell`, `/clear`, `/plan cancel`, `/cwd set`, `/config set|reset`, `/memory add|forget|compact`, `/model use`); the router schedules them under `run_in_terminal`, which suspends the full-screen app, restores the real terminal, runs the handler synchronously, then redraws. `needs_terminal` enumerates every confirm/consent/own-stdout/preload form — a new one must be added there too. + +**Blocking commands run on the worker thread.** Two kinds of slash command block and so cannot run on the loop thread. (1) `/plan revise ` calls `runtime.run_turn` — a model turn; running it on the loop thread would freeze the UI and the §31.16 approval-gate `Future` could only be resolved by the now-blocked loop → deadlock, and `run_in_terminal` would suspend the app while the turn marshals UI to it. `slash.needs_worker(line)` selects it (only `/plan revise` with non-empty text; a bare `/plan revise` just prints usage on the loop path); the router echoes the line (it is a turn) and runs it via `TurnRunner.start_action`, so its output reaches the pane through the marshaling UI and its approvals use the focus-swap gate. (2) `/doctor` (Ollama health + model list + writable-path probes), `/model list` (`GET /api/tags`), and `/attach ` (`POST /api/show`) make blocking network or filesystem I/O calls but need no real terminal; `slash.needs_background(line)` selects them. Both kinds run on the same worker (`start_action`); the background commands are NOT echoed and their captured output is marshaled into the pane (`schedule` → `show_slash_output`). A failed `/doctor` also appends a dim `exit code 1` note in the pane, mirroring one-shot `!` failures. The criterion is "blocks the loop", not just confirm/consent — a hung Ollama must never freeze the TUI for the client timeout. A new command that drives `run_turn` goes in `needs_worker`; one that makes a blocking network or filesystem I/O call goes in `needs_background`. + +**One dispatcher, two consoles, a confirm safety net.** Both the loop and terminal paths drive the app's single `SlashDispatcher`. The injected `dispatch(line, console)` closure swaps the dispatcher's console per call: the loop path passes the capturing console; the terminal path passes the real console. It also swaps the confirm — the loop path uses `_decline` (always declines, never `input()`), so a slash form misclassified as fast can never block the event loop; the terminal path uses the real blocking `_default_confirm`. Both are restored in `finally`. + +**`!` / `/shell` and the busy guard.** A one-shot `!` runs **captured on the worker** (`start_action` → `run_manual_command_captured`) and its combined output (stderr interleaved into stdout) is rendered into the pane line-by-line through the sanitizing `show_command_output` sink, with a dim `exit code N` note on failure — no app suspend, no terminal flash. Bare `!` and `/shell` are the **interactive** manual-shell loop, which reads live stdin, so they still run on the real terminal (`run_in_terminal` → `manual_shell_loop`). Both use the live workspace so a prior `/cwd set` is honoured; the `manual_shell_command` audit event is unchanged (one shared `_audit_manual_command`). A slash submitted from the dock while a turn is in flight is now QUEUED (the §31.18 one-message queue), not rejected: it is staged and fired through the router at turn end. The `SlashRouter`'s own `is_busy` guard (`route()` shows a "Busy" status when a turn is in flight) is therefore defense-in-depth — the dock stages before `route()` is reached, so a queued slash always runs `route()` from idle — but the guard stays as a fail-safe for any non-dock caller. The default (non-app) REPL is untouched: it routes slash/`!`/manual-shell exactly as before. + +### 31.18 Input-dock polish (v2) + +The dock gains four refinements; each is additive and a default (non-app) session is byte-identical, since the new `build_app` parameters all default to absent. + +**One-message queue.** A normal-line submit while a turn is in flight is staged, not dropped — `build_app` holds a single loop-thread slot (`pending`). The submit keybinding keeps its order: approval-gate check, then `/exit` quit, then the empty check, then — when `is_busy()` is true — it stages the line (a second submit replaces the first; one slot) and returns; otherwise it dispatches immediately. A faint `ConditionalContainer` **chip** (`⏳ queued: …` in unicode; the `queued:` label alone, no glyph, in ASCII) renders just above the dock border while a line is staged; the preview is `_sanitize_line`-stripped and single-lined (user-controlled text). The slot **fires at turn end**: `TurnRunner.on_idle` (set after construction, like `.app`/`.conversation`) is invoked inside `_mark_done` after `_busy` clears, and `build_app` registers `_fire_pending` through `register_idle`; `_fire_pending` clears the slot *before* dispatching, so the fired turn's own end finds an empty slot — no loop. Firing reuses the same dispatch routing as a live submit (slash/`!` → `on_slash`, normal → echo + `on_submit`). A Ctrl-C that **cancels** the turn also DRAINS the slot (the `c-c` handler clears `pending` on the loop thread before the worker's `_mark_done` schedules `on_idle`), so "stop everything" never leaves a staged follow-up to fire after the abort. + +**Up-arrow recall.** Pressing Up in an *empty* dock with a line staged pulls it back into the box (cursor at end) and clears the slot, so the user can edit, clear, or re-send it; the chip disappears. The keybinding is filtered on `empty dock AND something staged`, so a non-empty box or an empty slot leaves the default Up (cursor/history) untouched. + +**Mouse-wheel scroll.** The chat pane is cursorless-with-an-exposed-cursor-line (§31, `pane_scroll["line"]`, `None` = follow the bottom); the Window's own `vertical_scroll` is re-derived from that cursor each render, so a wheel nudge to `vertical_scroll` is lost. A `FormattedTextControl` subclass intercepts `SCROLL_UP`/`SCROLL_DOWN` at the control and folds them into the **same** cursor-line model as PageUp/PageDown (three lines per notch via `_scroll_up`/`_scroll_down`), so wheel and keyboard share one scroll/auto-follow behaviour; any other mouse event delegates to the base handler. + +**Live status values.** The status bar reads its dir/model/profile/locality/ctx through an optional `status_fn` callable evaluated per render, so `/model use` (the cloud indicator!), `/profile use`, `/cwd set`, and context growth reflect immediately. The cloud bit stays the real `is_egressing` signal — now live, never model output, still unspoofable (§15.2). The git branch segment follows the live workspace too: `_branch_resolver` re-reads `.git/HEAD` **only when the workspace changes** (a `/cwd`), not per render, so a mid-session `/cwd set` into a repo with a different branch — or out of any repo (the segment disappears) — refreshes the branch without an `.git` read on every repaint. With `status_fn` absent the bar uses the build-time params, unchanged. + +### 31.19 Thinking trail (inline, collapsible) + +The live indicator (§31.14) only ever surfaced a *count* of reasoning; the model's actual thinking was captured (`reply.thinking`) but never shown. The trail surfaces it inline, in place, as a faint collapsible block — display-only, additive, and a default session with no streamed thinking is byte-identical. + +**One trail per model call.** `stream_thinking` retains the thinking *text* on the current `_Trail` instead of only bumping the reasoning-char count. The trail is an entry in the `AppUI._renderables` transcript (not a separate channel), so it lands at its own position in scroll order. A reasoning phase opens a fresh trail at the current transcript position; subsequent thinking — including a trailing thought a model emits *after* its answer has already started streaming — joins that **same** trail. An answer token does **not** finalize the trail, and a resumed thought does **not** close the open response: within one model call the thinking is one trail and the answer is one response, regardless of how the stream interleaves them. (Tying the trail to the answer-token boundary instead would fragment an interleaving model into two trails and a split answer — the §31.19 regression fixed by moving finalization to the call boundary.) The active trail is finalized (`_finalize_active_trail` clears the single `_active_trail` pointer) at the **model-call boundary** — `end_response`, which `conversation.py` wraps around each `chat()` call — and by any non-thinking transcript content routed through `_add_renderable`/`show_plan_progress` (a tool call, status line, done line, aborted marker, user echo). So the *next* model call's reasoning — a genuine second phase, e.g. after a tool result — opens a new block, while interleaving *within* a call does not. Finalizing is the **only** cleanup: a finished trail stays visible and never has its state reset. + +**Collapsed by default, expandable.** The collapsed view shows the first `TRAIL_COLLAPSED_LINES` (6) non-blank lines under a `thinking · N reasoning` header (blank lines are dropped so the cap counts real reasoning lines); when more are hidden a footer reads `… +N hidden lines · click to expand`. Expanded shows every line and a `click to collapse` footer. A caret in the header (`▸`/`▾`, ASCII `>`/`v`) shows the state. Thinking text is model-controlled, so **every displayed line is `_sanitize_line`-stripped** at the sink. + +**Click toggles any trail.** A pane click toggles whichever trail (or diff, §31.16) it lands on, via the shared line→element index: `_render_ansi` records each trail's/diff's transcript line range `[start, end)` in the same single render pass (a `_LineCountingWriter` tallies newlines as Rich writes, so one Console covers the whole transcript even on a per-frame refresh), and `_PaneControl.mouse_handler` maps a `MOUSE_UP` position — prompt_toolkit hands it in content coordinates, so scroll is already accounted for — through `AppUI.toggle_at(y)` to the containing element. Because the click reaches the element directly, **any** trail is toggleable regardless of age — the win over a single "latest only" keybinding. A click outside every element returns `NotImplemented`/`False`, so default mouse handling (e.g. scroll) still runs. There is no keyboard toggle for trails (the earlier bare-`t` and `Ctrl-T` bindings are gone — bare `t` swallowed the first character of any message starting with `t`; the diff keeps a `Ctrl-O` keyboard fallback, §31.16). `show_reasoning=False` builds no trail at all (the readout is hidden too), so there is nothing to toggle. + +**Display-only.** The trail text never changes model history and is never fed back to the model (the same discard as the §31.15 aborted partial). `AppUI` has no history — it is a pure state+render object — so the trail lives only in the pane. + +### 31.20 In-app slash menu + +The dock replaces prompt_toolkit's default `CompletionsMenu` float (a low-contrast popup) with a custom menu docked **directly above the input frame** — so the dock `Buffer` carries **no `completer`** and Tab is free for the menu. The menu is a `ConditionalContainer` over a `FormattedTextControl`, visible only while `slash_menu_open(text)` holds: the text begins with `/` and has **no whitespace yet** (the first space ends the command token — the user has filled a command or moved into its args — so the menu closes), there is no active approval (the dock is the approval input then), and at least one command matches. + +**Rows come from `HELP_ROWS`.** `slash_menu_items()` builds one `SlashMenuItem` per help row: `fill` is the command with its `` placeholders dropped (what Tab/Enter inserts), `label` keeps the placeholders (what the user sees), and `takes_args` is set when the entry has a `<…>`. `slash_menu_matches(text, items)` keeps the items whose `fill` starts with the typed text (case-insensitive; a bare `/` matches all). The menu shows a `MENU_VISIBLE_ROWS` (3) window over the matches — `slash_menu_window(index, total)` slides it to keep the selection on screen — with the selected row in accent and the rest dim. The selection `index` resets to the top on every dock-text change (`on_text_changed`), so each keystroke re-filters from the first match. + +**Keys (all gated on the menu being open, registered after `_submit`/`_recall` so they win the shared keys when open).** `↑`/`↓` move the selection (the `_recall` `↑` filter is false here — its dock is empty, the menu's starts with `/`). **Tab** always *fills*: it sets the dock to `fill + " "`, whose trailing space closes the menu so the user types args or runs. **Enter is smart** — an **argless** command (`takes_args` false) runs immediately (the handler sets the dock text to `fill` and calls the shared `_submit_current`, the same effect as the Enter binding); an **arg** command *fills* like Tab and waits. A line with no leading `/` never engages the menu and submits normally. There is no keyboard menu toggle beyond these; the menu is purely a typing aid and the actual routing stays the §31.17 `SlashRouter`. ## 32. Model Selection And Preload diff --git a/pyproject.toml b/pyproject.toml index 577c524..24d995c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "shellpilot" -version = "0.10.0" +version = "0.11.0" description = "A local-first AI shell harness for your terminal, powered by Ollama." readme = "README.md" requires-python = ">=3.11" diff --git a/shellpilot/__init__.py b/shellpilot/__init__.py index c512b3b..d517410 100644 --- a/shellpilot/__init__.py +++ b/shellpilot/__init__.py @@ -1,3 +1,3 @@ """ShellPilot: a local-first AI shell harness.""" -__version__ = "0.10.1" +__version__ = "0.11.0" diff --git a/shellpilot/cli/app.py b/shellpilot/cli/app.py new file mode 100644 index 0000000..64606aa --- /dev/null +++ b/shellpilot/cli/app.py @@ -0,0 +1,886 @@ +"""Full-screen TUI app shell (UI v2 — design section 31). + +The interactive REPL is moving from a per-turn ``prompt`` (which scrolls the +status bar and input away mid-turn) to a persistent full-screen +``prompt_toolkit`` ``Application`` on the alternate screen, so the status bar +and the input dock never vanish. This module is the **inert app shell**: it +builds the layout — a scrolling chat pane, a custom rounded multi-line input +dock, and the persistent status bar — with no runtime, model, or AI turn wired +in. Submitting echoes the dock text into the pane and clears the dock; ``/exit`` +quits. Later branches wire the conversation, render Rich content in the pane, +add the thinking indicator, and handle Ctrl-C turn cancellation. + +The dock border is drawn by hand (rounded corners, ASCII fallback) rather than +with ``prompt_toolkit``'s ``Frame``: ``Frame`` only draws square corners and +reserves completion-menu height inside the box, so it fills the terminal height +instead of hugging the input — this section is the one deliberate exception to +the §31.9 "borders come from rich primitives" contract, because the dock lives +in a ``prompt_toolkit`` layout, not a Rich render. +""" + +from __future__ import annotations + +from collections.abc import Callable, Sequence +from dataclasses import dataclass +from pathlib import Path +from typing import TYPE_CHECKING, TypedDict + +from prompt_toolkit.application import Application, get_app +from prompt_toolkit.buffer import Buffer +from prompt_toolkit.data_structures import Point +from prompt_toolkit.filters import Condition, has_focus +from prompt_toolkit.formatted_text import ANSI, StyleAndTextTuples +from prompt_toolkit.input import Input +from prompt_toolkit.key_binding import KeyBindings, KeyPressEvent +from prompt_toolkit.layout.containers import ( + ConditionalContainer, + HSplit, + VSplit, + Window, +) +from prompt_toolkit.layout.controls import BufferControl, FormattedTextControl +from prompt_toolkit.layout.dimension import Dimension +from prompt_toolkit.layout.layout import Layout +from prompt_toolkit.mouse_events import MouseEvent, MouseEventType +from prompt_toolkit.output import Output + +from shellpilot.cli.app_ui import AppUI +from shellpilot.cli.model_completion import ModelCompletionMatch, model_completion_matches +from shellpilot.cli.path_completion import PathCompletionMatch, path_completion_matches +from shellpilot.cli.render import _sanitize_line +from shellpilot.cli.slash import ( + SlashMenuItem, + slash_menu_items, + slash_menu_matches, + slash_menu_open, + slash_menu_window, +) +from shellpilot.cli.status_bar import status_bar +from shellpilot.cli.theme import ( + COLOR_ACCENT, + COLOR_ERROR, + COLOR_FAINT, + COLOR_WARN, + UNICODE_GLYPHS, + Glyphs, +) +from shellpilot.policy.risk import RiskLevel + +if TYPE_CHECKING: + from shellpilot.cli.app_approval import ApprovalGate + from shellpilot.llm.ollama import LocalModel + +# The dock grows to fit multi-line input up to this many rows, then scrolls +# internally. NOTE: a fixed cap — a per-terminal-height fraction would be nicer +# on a very short terminal, but a flat cap is enough until the live wiring +# (branch 4) shows whether it bites. +DOCK_MAX_ROWS = 10 + +# The slash menu (§31.20) shows this many command rows at once; ↑/↓ scroll the +# window through a longer filtered list. +MENU_VISIBLE_ROWS = 3 + +# Idle Ctrl-C hint, shown only when no turn is in flight. Branch 6 (§31.15) wired +# real turn cancellation through on_interrupt; this hint is the idle fallback. +# NOTE: subprocess-kill on cancel is branch 6b. +_IDLE_HINT = "(idle — type /exit to quit)" + + +class _SlashMenuState(TypedDict): + index: int + query: str + preview: bool + suppress_change: bool + + +class _PathMenuState(TypedDict): + index: int + + +@dataclass(frozen=True) +class BoxChars: + """Border glyphs for the rounded input dock; the ASCII set degrades them.""" + + top_left: str + top_right: str + bottom_left: str + bottom_right: str + horizontal: str + vertical: str + + +UNICODE_BOX = BoxChars("╭", "╮", "╰", "╯", "─", "│") +ASCII_BOX = BoxChars("+", "+", "+", "+", "-", "|") + + +def horizontal_border(width: int, box: BoxChars, *, top: bool, label: str | None = None) -> str: + """A horizontal dock border line of exactly ``width`` cells. + + Pure function of ``width`` (and the glyph set): ``╭───╮`` for the top row, + ``╰───╯`` for the bottom. The border, the pane wrap width, and the terminal + width are one shared value (see :func:`build_app`), so this is rebuilt per + render from the live terminal width and nothing caches a stale size. + + ``label`` (modal dock, §31.16) embeds a short state hint into the TOP + border — ``╭─ approve? ───╮`` — so the dock says what it is asking while an + approval owns the input. Dropped whole when it cannot fit the width; the + bottom border never carries it. + """ + if width < 2: + return box.horizontal * max(0, width) + left = box.top_left if top else box.bottom_left + right = box.top_right if top else box.bottom_right + if top and label: + decorated = f"{box.horizontal} {label} " + if len(decorated) + 2 <= width: + fill = box.horizontal * (width - 2 - len(decorated)) + return left + decorated + fill + right + return left + box.horizontal * (width - 2) + right + + +def _read_git_branch(workspace: Path) -> str | None: + """Current git branch from ``/.git/HEAD``, or None. + + Pure (one ``read_text``, no other I/O). Fails closed to None on every + non-branch case: not a repo (no ``.git``), a worktree (``.git`` is a plain + file → descending into it raises ``NotADirectoryError`` ⊂ ``OSError``), + permission errors, and a detached HEAD (a bare SHA with no ``ref:`` prefix). + """ + try: + head = (workspace / ".git" / "HEAD").read_text(encoding="utf-8", errors="replace") + except OSError: + return None + # First line only + length cap: a real HEAD is one short ref line, but a + # crafted clone's HEAD must not inject extra lines or unbounded text into the + # status-bar dock that hosts the (unspoofable) cloud indicator. + content = head.splitlines()[0].strip() if head else "" + prefix = "ref: refs/heads/" + if content.startswith(prefix): + return content[len(prefix) :][:128] or None + return None + + +def _branch_resolver( + initial_workspace: Path, initial_branch: str | None +) -> Callable[[Path], str | None]: + """Map the live workspace to its git branch, re-reading ``.git/HEAD`` ONLY when + the workspace changes — a ``/cwd`` — never per render (§31.18). + + Seeded with the build-time ``(workspace, branch)`` so the first frames cost no + I/O. When the status bar's live workspace differs from the last one resolved, + the branch is re-read (``None`` when the new directory is not a repo) and + cached, so the status-bar segment follows cwd into and out of a repo without an + ``.git`` read on every repaint. + """ + last_workspace = initial_workspace + branch = initial_branch + + def resolve(workspace: Path) -> str | None: + nonlocal last_workspace, branch + if workspace != last_workspace: + last_workspace = workspace + branch = _read_git_branch(workspace) + return branch + + return resolve + + +def _scroll_up(scroll: int | None, last_line: int, page: int) -> int: + """PageUp: move the pane's pinned cursor line up ``page`` lines. + + ``scroll`` is the currently pinned line, or None when following the bottom + (then the cursor sits at ``last_line``). Returns the new pinned line — always + an int, so PageUp leaves follow mode and the reader stays put as new output + appends below. + """ + current = last_line if scroll is None else scroll + return max(0, current - page) + + +def _scroll_down(scroll: int | None, last_line: int, page: int) -> int | None: + """PageDown: move the pane's pinned cursor line down ``page`` lines. + + Returns None once the cursor reaches the last line — i.e. scrolling back to + the bottom resumes following (auto-scroll as the response streams in). + """ + current = last_line if scroll is None else scroll + new = min(last_line, current + page) + return None if new >= last_line else new + + +@dataclass(frozen=True) +class StatusValues: + """Live status-bar inputs, read per render so a mid-session ``/model use``, + ``/profile use``, ``/cwd set``, or context growth reflects immediately + (§31.18). ``branch`` is deliberately NOT a field — it is derived from + ``workspace`` by ``_branch_resolver``, which re-reads ``.git/HEAD`` only when + the workspace changes, so the segment follows ``/cwd`` without an ``.git`` read + on every repaint.""" + + workspace: Path + model: str + profile: str + is_cloud: bool + ctx_pct: int + + +def build_app( + *, + workspace: Path, + model: str, + profile: str, + glyphs: Glyphs, + commands: Sequence[str], + is_cloud: bool = False, + ctx_pct: int = 0, + show_reasoning: bool = True, + input: Input | None = None, + output: Output | None = None, + ui: AppUI | None = None, + on_submit: Callable[[str], None] | None = None, + on_interrupt: Callable[[], bool] | None = None, + on_slash: Callable[[str], None] | None = None, + approval_gate: ApprovalGate | None = None, + is_busy: Callable[[], bool] | None = None, + register_idle: Callable[[Callable[[], None]], None] | None = None, + status_fn: Callable[[], StatusValues] | None = None, + model_completion_models: Callable[[], list[LocalModel]] | None = None, +) -> Application[None]: + """Build the full-screen app shell. + + ``input``/``output`` default to the real terminal; tests inject a pipe input + and ``DummyOutput`` to drive the shell headlessly. ``on_submit`` receives the + dock text on submit (branch 4 passes ``TurnRunner.start``); when it is None + the shell falls back to the inert branch-3 echo so the standalone shell and + its headless tests stay working. The non-TTY ``PlainInput`` path is untouched. + """ + # One unicode/ascii decision drives the border, the branch glyph, and the + # bar — recovered from the resolved glyph set the caller already probed + # (``resolve_glyphs``), so we don't re-probe the console encoding here. + unicode_mode = glyphs == UNICODE_GLYPHS + box = UNICODE_BOX if unicode_mode else ASCII_BOX + branch_glyph = "⎇" if unicode_mode else "git:" + # The chip text already says "queued:", so the marker is a unicode-only + # accent — no ASCII glyph (avoids a redundant "[queued] queued:"). + queued_marker = "⏳ " if unicode_mode else "" + # The branch segment follows the live workspace (a /cwd) but re-reads .git/HEAD + # only when the workspace changes, not per render (§31.18). + resolve_branch = _branch_resolver(workspace, _read_git_branch(workspace)) + + # Chat pane: a FormattedTextControl rendering the AppUI transcript as Rich→ANSI. + # Rich handles all wrapping at width W (wrap_lines=False on the Window), so the + # ANSI string already contains the correct line breaks for the current width. + # The AppUI re-renders when the width changes (cache miss in _render_ansi). + if ui is None: + ui = AppUI( + glyphs=glyphs, + workspace=workspace, + width_fn=lambda: get_app().output.get_size().columns, + show_reasoning=show_reasoning, + ) + # Explicit AppUI annotation (not AppUI | None) so the closures below capture a + # non-optional type; ui is already narrowed to AppUI by the guard above. + _ui: AppUI = ui + + # Pane scroll state: "line" is the transcript line the pane keeps in view, or + # None to follow the bottom (auto-scroll as a response streams in). A bare + # FormattedTextControl has no cursor, so prompt_toolkit defaults it to (0,0) + # and snaps the pane to the TOP every render — overriding any manual + # vertical_scroll. Exposing this line as the UIContent cursor instead makes + # pt scroll to keep it visible (independent of focus): following → bottom, and + # a reader who paged up (a pinned line) is not yanked down when output appends. + pane_scroll: dict[str, int | None] = {"line": None} + + # One-message queue (§31.18): a single staged line, set when a submit lands + # while a turn is in flight, fired at turn end by _fire_pending. None = empty. + # Loop-thread-only state, like pane_scroll. + pending: dict[str, str | None] = {"text": None} + + # Pane render cache (perf): _render_ansi already returns the SAME string object + # while the transcript content and width are unchanged (its width cache), so we + # parse the ANSI and count its lines ONLY when that string actually changes — an + # identity compare, O(1) on a scroll or idle-refresh tick. Constructing ANSI(...) + # re-parses the whole transcript (ANSI parses in __init__) and prompt_toolkit's + # FormattedTextControl cache is keyed by the per-render counter, so without this + # every redraw re-parsed the entire transcript — the source of the scroll lag. + pane_render: dict[str, tuple[str, ANSI, int]] = {} + + def _pane_view() -> tuple[ANSI, int]: + text = _ui._render_ansi() + cached = pane_render.get("v") + if cached is None or cached[0] is not text: + n = text.count("\n") + # Rich ends each renderable with a newline, so the last real line is n-1. + last = max(0, n - 1) if text.endswith("\n") else n + cached = (text, ANSI(text), last) + pane_render["v"] = cached + return cached[1], cached[2] + + def _pane_last_line() -> int: + return _pane_view()[1] + + def _pane_cursor() -> Point: + last = _pane_last_line() + line = pane_scroll["line"] + return Point(x=0, y=last if line is None else max(0, min(line, last))) + + class _PaneControl(FormattedTextControl): + # Mouse-wheel scroll through the SAME cursor-line model as PageUp/PageDown + # (§31.18): the Window's own vertical_scroll is overridden by the cursor + # each render, so the wheel must be intercepted at the control and folded + # into pane_scroll. Three lines per notch; returns None when handled. + def mouse_handler(self, mouse_event: MouseEvent) -> object: + if mouse_event.event_type == MouseEventType.SCROLL_UP: + pane_scroll["line"] = _scroll_up(pane_scroll["line"], _pane_last_line(), 3) + return None + if mouse_event.event_type == MouseEventType.SCROLL_DOWN: + pane_scroll["line"] = _scroll_down(pane_scroll["line"], _pane_last_line(), 3) + return None + if mouse_event.event_type == MouseEventType.MOUSE_UP: + # Click on a diff/trail toggles its collapse state (§31.16/§31.19). + # prompt_toolkit hands us the position in CONTENT coordinates (the + # document row, scroll already accounted for), which is exactly the + # transcript line toggle_at maps. Returning None when handled lets the + # standard post-event redraw repaint (toggle_at cleared the cache). + if _ui.toggle_at(mouse_event.position.y): + return None + return super().mouse_handler(mouse_event) + + pane_window = Window( + _PaneControl( + lambda: _pane_view()[0], + focusable=False, + show_cursor=False, + get_cursor_position=_pane_cursor, + ), + wrap_lines=False, + ) + + def _pane_page() -> int: + info = pane_window.render_info + return max(1, (info.window_height - 1) if info is not None else 10) + + # Input dock: a focused, multi-line buffer. Slash completion is the custom + # in-app menu below (§31.20), NOT prompt_toolkit's completer/CompletionsMenu — + # so there is no `completer` here and Tab is free for the menu's fill binding. + dock_buffer = Buffer(name="dock", multiline=True) + dock_focused = has_focus(dock_buffer) + + # The slash menu (§31.20): rows derived once from HELP_ROWS; `index` is the + # current selection. `query` is the user-typed slash token used for filtering; + # arrow-key previews can write multi-word commands into the dock while this + # query keeps the menu open over the original sibling set. + menu_items = slash_menu_items() + menu_label_width = max((len(it.label) for it in menu_items), default=0) + slash_menu: _SlashMenuState = { + "index": 0, + "query": "", + "preview": False, + "suppress_change": False, + } + path_menu: _PathMenuState = {"index": 0} + + def _clear_menu_state() -> None: + slash_menu["index"] = 0 + slash_menu["query"] = "" + slash_menu["preview"] = False + slash_menu["suppress_change"] = False + path_menu["index"] = 0 + + def _reset_menu_index(_buffer: Buffer) -> None: + if slash_menu["suppress_change"]: + return + slash_menu["index"] = 0 + slash_menu["query"] = dock_buffer.text + slash_menu["preview"] = False + path_menu["index"] = 0 + + dock_buffer.on_text_changed += _reset_menu_index + + def _current_workspace() -> Path: + values = status_fn() if status_fn is not None else None + return values.workspace if values is not None else workspace + + def _menu_query() -> str: + if slash_menu["preview"]: + return str(slash_menu["query"]) + return dock_buffer.text + + def _menu_matches() -> list[SlashMenuItem]: + return slash_menu_matches(_menu_query(), menu_items) + + def _menu_open() -> bool: + # Closed during an approval (the dock is the approval input then) and when + # nothing matches; otherwise open while the command token is being typed. + if approval_gate is not None and approval_gate.active: + return False + return slash_menu_open(_menu_query()) and bool(_menu_matches()) + + def _menu_index() -> int: + matches = _menu_matches() + if not matches: + return 0 + return max(0, min(slash_menu["index"], len(matches) - 1)) + + menu_open = Condition(_menu_open) + + def _path_matches() -> list[PathCompletionMatch | ModelCompletionMatch]: + path_matches: list[PathCompletionMatch | ModelCompletionMatch] = list( + path_completion_matches( + dock_buffer.document.text_before_cursor, + _current_workspace(), + ) + ) + if path_matches: + return path_matches + if model_completion_models is None: + return [] + return list( + model_completion_matches( + dock_buffer.document.text_before_cursor, + model_completion_models(), + ) + ) + + def _path_menu_open() -> bool: + if approval_gate is not None and approval_gate.active: + return False + return not _menu_open() and bool(_path_matches()) + + def _path_menu_index() -> int: + matches = _path_matches() + if not matches: + return 0 + return max(0, min(path_menu["index"], len(matches) - 1)) + + path_menu_open = Condition(_path_menu_open) + + def _dock_color() -> str: + # The modal dock (§31.16): during an approval the border carries the + # decision color — red for a HIGH-risk prompt, amber for any other — + # matching the approval card in the pane; idle it stays faint. Read + # per render, so the swap tracks the gate exactly. + if approval_gate is not None and approval_gate.active: + return COLOR_ERROR if approval_gate.dock_risk is RiskLevel.HIGH else COLOR_WARN + return COLOR_FAINT + + def _dock_label() -> str | None: + if approval_gate is not None and approval_gate.active: + return approval_gate.dock_hint + return None + + def _dock_prefix(line_number: int, wrap_count: int) -> StyleAndTextTuples: + if line_number == 0 and wrap_count == 0: + # The chevron follows the mode: the approval color while a prompt + # owns the dock, accent green otherwise. Deliberately NOT keyed on + # is_busy — a render must never consume a busy read the submit + # keybinding's stage-or-dispatch decision depends on. + if approval_gate is not None and approval_gate.active: + color = _dock_color() + else: + color = COLOR_ACCENT + return [(f"fg:{color} bold", f"{glyphs.chevron} ")] + return [("", " ")] + + dock_window = Window( + BufferControl(buffer=dock_buffer), + height=Dimension(min=1, max=DOCK_MAX_ROWS), + dont_extend_height=True, + wrap_lines=True, + get_line_prefix=_dock_prefix, + ) + + def _border(*, top: bool) -> Callable[[], StyleAndTextTuples]: + # One shared width: the live terminal columns, read at render time so a + # resize re-derives the line and nothing is pinned to a stale size. The + # color and the top-border label follow the approval state per render + # (modal dock, §31.16). + def _render() -> StyleAndTextTuples: + width = get_app().output.get_size().columns + label = _dock_label() if top else None + return [(f"fg:{_dock_color()}", horizontal_border(width, box, top=top, label=label))] + + return _render + + def _status() -> StyleAndTextTuples: + # Live values (§31.18) when status_fn is wired — workspace/model/profile/ + # cloud/ctx re-read per render so /model use (the cloud indicator!), + # /profile use, /cwd set, and context growth reflect immediately. The + # branch segment follows the live workspace via resolve_branch, which + # re-reads .git/HEAD only when the workspace changes (a /cwd) — not per + # render. The cloud bit still comes from the real is_egressing signal + # (unspoofable). Falls back to the static params (standalone shell + + # existing tests) when status_fn is None. + v = status_fn() if status_fn is not None else None + ws = v.workspace if v is not None else workspace + return list( + status_bar( + workspace=ws, + model=v.model if v is not None else model, + profile=v.profile if v is not None else profile, + is_cloud=v.is_cloud if v is not None else is_cloud, + ctx_pct=v.ctx_pct if v is not None else ctx_pct, + branch=resolve_branch(ws), + branch_glyph=branch_glyph, + ) + ) + + def _chip() -> StyleAndTextTuples: + # A faint one-line "queued" chip above the dock border while a message is + # staged (§31.18). The preview is user-controlled, so it is sanitized; + # newlines collapse to spaces to keep the chip a single line, and it is + # capped at ~60 cells. The glyph degrades to "[queued]" in ASCII mode. + preview = _sanitize_line(pending["text"] or "").replace("\n", " ") + if len(preview) > 60: + preview = preview[:60] + glyphs.ellipsis + return [(f"fg:{COLOR_FAINT}", f"{queued_marker}queued: {preview}")] + + chip_window = ConditionalContainer( + content=Window(FormattedTextControl(_chip), height=1), + filter=Condition(lambda: pending["text"] is not None), + ) + + def _menu_content() -> StyleAndTextTuples: + # The slash menu rows (§31.20): a MENU_VISIBLE_ROWS window over the filtered + # matches, the selected row carried in accent, others dim — higher contrast + # than the default completion popup. Command labels (with their + # placeholders) are padded to one column so descriptions align. + matches = _menu_matches() + if not matches: + return [] + index = _menu_index() + start = slash_menu_window(index, len(matches), MENU_VISIBLE_ROWS) + caret_on = "▸ " if glyphs is UNICODE_GLYPHS else "> " + frags: StyleAndTextTuples = [] + for offset, item in enumerate(matches[start : start + MENU_VISIBLE_ROWS]): + selected = (start + offset) == index + caret = caret_on if selected else " " + label_style = f"fg:{COLOR_ACCENT} bold" if selected else "" + desc_style = f"fg:{COLOR_ACCENT}" if selected else f"fg:{COLOR_FAINT}" + frags.append((label_style, f" {caret}{item.label}".ljust(menu_label_width + 4))) + frags.append((desc_style, f" {item.description}")) + frags.append(("", "\n")) + frags.pop() # drop the trailing newline so the window height is exact + return frags + + menu_window = ConditionalContainer( + content=Window( + FormattedTextControl(_menu_content), + height=Dimension(max=MENU_VISIBLE_ROWS), + dont_extend_height=True, + ), + filter=menu_open, + ) + + def _path_menu_content() -> StyleAndTextTuples: + matches = _path_matches() + if not matches: + return [] + index = _path_menu_index() + start = slash_menu_window(index, len(matches), MENU_VISIBLE_ROWS) + caret_on = "▸ " if glyphs is UNICODE_GLYPHS else "> " + frags: StyleAndTextTuples = [] + for offset, item in enumerate(matches[start : start + MENU_VISIBLE_ROWS]): + selected = (start + offset) == index + caret = caret_on if selected else " " + label_style = f"fg:{COLOR_ACCENT} bold" if selected else "" + hint_style = f"fg:{COLOR_ACCENT}" if selected else f"fg:{COLOR_FAINT}" + kind = ( + item.hint + if isinstance(item, ModelCompletionMatch) + else ("dir" if item.label.endswith("/") else "file") + ) + frags.append((label_style, f" {caret}{item.label}")) + frags.append((hint_style, f" {kind}")) + frags.append(("", "\n")) + frags.pop() + return frags + + path_menu_window = ConditionalContainer( + content=Window( + FormattedTextControl(_path_menu_content), + height=Dimension(max=MENU_VISIBLE_ROWS), + dont_extend_height=True, + ), + filter=path_menu_open, + ) + + def _bar() -> Window: + # The side bars share the border's approval-state color (callable style, + # re-evaluated per render). + return Window(width=1, char=box.vertical, style=lambda: f"fg:{_dock_color()}") + + dock_row = VSplit( + [ + _bar(), + Window(width=1, char=" "), + dock_window, + Window(width=1, char=" "), + _bar(), + ] + ) + + root = HSplit( + [ + pane_window, + chip_window, + menu_window, + path_menu_window, + Window(FormattedTextControl(_border(top=True)), height=1), + dock_row, + Window(FormattedTextControl(_border(top=False)), height=1), + Window(FormattedTextControl(_status), height=1), + ] + ) + + def _dispatch_line(text: str) -> None: + # The routing a non-staged submit takes (§31.18): slash/`!` → on_slash, + # a normal line → echo + on_submit (or the inert show_status fallback). + # A new turn jumps the pane back to the bottom to watch it stream. + pane_scroll["line"] = None + stripped = text.strip() + if on_slash is not None and stripped and stripped[0] in "/!": + # A typed slash or `!` line is a harness control, not a model turn: + # route it to the SlashRouter (capture / run_in_terminal / manual + # shell / exit), §31.17. + on_slash(text) + return + if on_submit is not None: + # Echo the typed line into the pane on the loop thread, then run the + # turn. The live indicator renders just below this echo (§31.14). + _ui.show_user_message(text) + on_submit(text) + else: + _ui.show_status(text) + + def _fire_pending() -> None: + # Loop-thread idle callback (TurnRunner.on_idle): fire the staged line, if + # any, as a fresh turn. pending is cleared FIRST, so the new turn's own end + # fires _fire_pending again against an empty slot — no loop (§31.18). + if pending["text"] is None: + return + text = pending["text"] + pending["text"] = None + _dispatch_line(text) + + if register_idle is not None: + register_idle(_fire_pending) + + kb = KeyBindings() + + # Enter submits. NOTE: a pipe sends LF (``\n`` → ``c-j``) and a real terminal + # sends CR (``\r`` → ``enter``); both submit. A literal newline for multi-line + # input is Alt+Enter (``escape, enter``), the prompt_toolkit convention. + def _submit_current() -> None: + # The submit effect, callable from the Enter binding AND the slash menu's + # smart-Enter on an argless command (which sets the dock text first). + # During an approval the dock IS the approval input: route the line to the + # gate (which resolves the worker's Future) BEFORE the /exit check, so a + # mid-approval "/exit" is an approval answer, not a quit (§31.16). + if approval_gate is not None and approval_gate.active: + line = dock_buffer.text + dock_buffer.reset() + _clear_menu_state() + approval_gate.submit(line) + return + text = dock_buffer.text + dock_buffer.reset() + _clear_menu_state() + if text.strip() == "/exit": + get_app().exit() + return + if not text.strip(): + return + if is_busy is not None and is_busy(): + # A submit while a turn is in flight is STAGED, not dropped (§31.18): + # one slot, so a second submit replaces the first. It fires at turn + # end via _fire_pending (TurnRunner.on_idle). + pending["text"] = text + return + _dispatch_line(text) + + @kb.add("enter", filter=dock_focused) + @kb.add("c-j", filter=dock_focused) + def _submit(event: KeyPressEvent) -> None: + if _path_menu_open(): + matches = _path_matches() + if matches and isinstance(matches[_path_menu_index()], ModelCompletionMatch): + _accept_path_menu_completion(submit_model=True) + return + _submit_current() + + @kb.add( + "up", + filter=dock_focused + & Condition( + lambda: ( + not dock_buffer.text + and pending["text"] is not None + and (approval_gate is None or not approval_gate.active) + ) + ), + ) + def _recall(event: KeyPressEvent) -> None: + # Up in an EMPTY dock with a staged message pulls it back into the box to + # edit/clear/re-send; the chip disappears (pending cleared), §31.18. When + # the box is non-empty or nothing is staged the filter is false and the + # default Up (cursor up / history) applies — this binding is not reached. + # During an active approval the dock IS the approval input (§31.16), so + # recall is suppressed there too — Enter would otherwise feed the staged + # text back to the approval gate as its answer. + dock_buffer.text = pending["text"] or "" + dock_buffer.cursor_position = len(dock_buffer.text) + pending["text"] = None + + # Slash-menu navigation (§31.20). Registered AFTER _submit/_recall so that when + # the menu is open these win the shared keys (last matching binding wins): ↑/↓ + # move the selection (the _recall ↑ filter is false here — its dock is empty, + # this one's starts with '/'), Enter is smart, Tab fills. All gated on menu_open + # so a normal message types literally. + def _menu_fill(item: SlashMenuItem) -> None: + # Put the command (no placeholders) in the box + a trailing space; the + # space ends the token so the menu closes and the user types args (or runs). + dock_buffer.text = item.fill + " " + dock_buffer.cursor_position = len(dock_buffer.text) + + def _menu_preview(item: SlashMenuItem) -> None: + # Arrow selection previews the highlighted command without accepting it. + # The original typed query remains active so multi-word previews such as + # "/model list" do not close the menu before the user can keep arrowing. + slash_menu["suppress_change"] = True + try: + dock_buffer.text = item.fill + dock_buffer.cursor_position = len(dock_buffer.text) + finally: + slash_menu["suppress_change"] = False + slash_menu["preview"] = True + + @kb.add("up", filter=dock_focused & menu_open) + def _menu_up(event: KeyPressEvent) -> None: + matches = _menu_matches() + slash_menu["index"] = max(0, _menu_index() - 1) + _menu_preview(matches[_menu_index()]) + + @kb.add("down", filter=dock_focused & menu_open) + def _menu_down(event: KeyPressEvent) -> None: + matches = _menu_matches() + slash_menu["index"] = min(len(matches) - 1, _menu_index() + 1) + _menu_preview(matches[_menu_index()]) + + @kb.add("enter", filter=dock_focused & menu_open) + def _menu_enter(event: KeyPressEvent) -> None: + # Smart Enter: an argless command runs now; an arg command fills and waits. + item = _menu_matches()[_menu_index()] + if item.takes_args: + _menu_fill(item) + else: + dock_buffer.text = item.fill + _submit_current() + + @kb.add("tab", filter=dock_focused & menu_open) + def _menu_tab(event: KeyPressEvent) -> None: + _menu_fill(_menu_matches()[_menu_index()]) + + @kb.add("up", filter=dock_focused & path_menu_open) + def _path_menu_up(event: KeyPressEvent) -> None: + matches = _path_matches() + path_menu["index"] = max(0, _path_menu_index() - 1) + if path_menu["index"] >= len(matches): + path_menu["index"] = max(0, len(matches) - 1) + + @kb.add("down", filter=dock_focused & path_menu_open) + def _path_menu_down(event: KeyPressEvent) -> None: + matches = _path_matches() + path_menu["index"] = min(len(matches) - 1, _path_menu_index() + 1) + + @kb.add("tab", filter=dock_focused & path_menu_open) + def _path_menu_tab(event: KeyPressEvent) -> None: + _accept_path_menu_completion(submit_model=False) + + def _accept_path_menu_completion(*, submit_model: bool) -> None: + matches = _path_matches() + if not matches: + return + item = matches[_path_menu_index()] + fill = item.fill + suffix = dock_buffer.document.text_after_cursor + dock_buffer.text = fill + suffix + dock_buffer.cursor_position = len(fill) + if submit_model and isinstance(item, ModelCompletionMatch): + _submit_current() + + @kb.add( + "c-o", + filter=dock_focused & Condition(lambda: approval_gate is None or not approval_gate.active), + ) + def _toggle_diff(event: KeyPressEvent) -> None: + # Ctrl-O toggles the LATEST diff's collapse state (§31.16) — the keyboard + # fallback for terminals without mouse reporting, where the per-element + # click toggle is unavailable. A modifier key (not a bare letter) so it + # never collides with typing a message; overrides prompt_toolkit's default + # c-o the same way the app's c-c/c-d bindings override theirs. No-op when + # there is no diff yet, so a stray press is harmless. Diffs and trails are + # both reachable by CLICK; this fallback intentionally covers diffs only. + _ui.toggle_latest_diff() + + @kb.add("escape", "enter", filter=dock_focused) + def _newline(event: KeyPressEvent) -> None: + dock_buffer.insert_text("\n") + + @kb.add("pageup") + def _page_up(event: KeyPressEvent) -> None: + pane_scroll["line"] = _scroll_up(pane_scroll["line"], _pane_last_line(), _pane_page()) + + @kb.add("pagedown") + def _page_down(event: KeyPressEvent) -> None: + pane_scroll["line"] = _scroll_down(pane_scroll["line"], _pane_last_line(), _pane_page()) + + @kb.add("c-c") + def _interrupt(event: KeyPressEvent) -> None: + # During an approval the worker is blocked on the gate's Future, not in a + # model-stream read, so on_interrupt (model cancel) would not fire. Ctrl-C + # must resolve the approval as a decline of THIS action; the turn continues + # and turn-level cancel is available again after the prompt returns (mirrors + # TerminalUI.ask_approval's KeyboardInterrupt → DECLINE) (§31.16). + if approval_gate is not None and approval_gate.active: + approval_gate.cancel() + return + # Raw mode disables ISIG, so Ctrl-C is a normal key press, not a SIGINT + # (branch 6, §31.15). When a turn is in flight, on_interrupt cancels it and + # returns True (the worker aborts the stream and renders the marker), so we + # show nothing. Otherwise it is idle — fall back to the idle hint. + if on_interrupt is not None and on_interrupt(): + # "Stop everything": a Ctrl-C that aborts the turn also DRAINS any + # staged message, so a queued follow-up does not fire after the abort + # (§31.18). Cleared here on the loop thread BEFORE the worker's + # _mark_done schedules on_idle, so _fire_pending sees an empty slot. + pending["text"] = None + return + # Deduped so repeated Ctrl-C while idle shows the hint once, not a stack. + _ui.show_idle_hint(_IDLE_HINT) + + @kb.add("c-d", filter=dock_focused) + def _eof(event: KeyPressEvent) -> None: + # EOF during an approval declines THIS action (same as Ctrl-C). Otherwise a + # no-op: the dock owns c-d so a stray press never tears down the app. + if approval_gate is not None and approval_gate.active: + approval_gate.cancel() + + # NOTE: keyboard PageUp/PageDown and mouse-wheel scroll both drive the pane + # via the cursor-line model — PageUp/PageDown through the keybindings above, + # the wheel through _PaneControl.mouse_handler (§31.18), since the Window's + # own vertical_scroll is re-derived from the cursor each render. + return Application[None]( + layout=Layout(root, focused_element=dock_window), + key_bindings=kb, + full_screen=True, + mouse_support=True, + # No built-in refresh_interval (perf, §31.14): it starts ONE background task + # that invalidates on a fixed timer forever, redrawing the static transcript + # even when idle — wasted CPU the old REPL never spent. The live thinking + # indicator is instead animated by run_app's gated refresh loop, which + # invalidates ONLY while a turn is in flight (AppUI.is_animating); idle stays + # purely event-driven (a redraw happens on a keystroke or scroll, not a timer). + input=input, + output=output, + ) diff --git a/shellpilot/cli/app_approval.py b/shellpilot/cli/app_approval.py new file mode 100644 index 0000000..b5d57b9 --- /dev/null +++ b/shellpilot/cli/app_approval.py @@ -0,0 +1,274 @@ +"""Approval focus-swap rendezvous for the full-screen app (design section 31.16). + +The full-screen app runs one conversation turn on a worker thread while the +prompt_toolkit event loop owns the main thread (see ``app_turn.py``). Every +fire-and-forget UI call is marshaled loop-ward, but ``ask_approval`` / +``ask_plan_approval`` RETURN a value, so they cannot be fire-and-forget. This is +the focus-swap handshake: the worker blocks on a ``concurrent.futures.Future`` +while the loop thread renders the prompt into the pane, reads the user's +keystrokes in the dock, and resolves the future. + +The pure parse helpers (:func:`parse_command_choice` / :func:`parse_plan_choice`) +carry the three-way y/e/n + HIGH typed-"run" contract and are unit-tested +directly, with no threading. +""" + +from __future__ import annotations + +import functools +from collections.abc import Callable +from concurrent.futures import Future +from dataclasses import dataclass +from typing import TYPE_CHECKING + +from shellpilot.cli.render import _sanitize_line, approval_choices, plan_choices +from shellpilot.cli.theme import UNICODE_GLYPHS, Glyphs +from shellpilot.policy.approvals import APPROVE, DECLINE, ApprovalReply +from shellpilot.policy.risk import RiskLevel + +if TYPE_CHECKING: + from shellpilot.cli.app_turn import Schedule + from shellpilot.cli.app_ui import AppUI + from shellpilot.policy.approvals import ApprovalRequest + from shellpilot.runtime.planner import TaskPlan + + +class _NeedSteer: + """Sentinel: the user chose [e]dit at the choice phase; read steering next.""" + + +NEED_STEER = _NeedSteer() + + +def parse_command_choice(request: ApprovalRequest, answer: str) -> ApprovalReply | _NeedSteer: + """Map a command/tool approval keystroke to its three-way outcome. + + A HIGH-risk COMMAND requires the literal "run" to execute (the typed-"run" + gate, design section 14.6); [e]dit steers without running and anything else + declines. Every other request takes the y/e/n path. + """ + low = answer.strip().lower() + if request.risk is RiskLevel.HIGH and request.kind == "command": + if low == "run": + return APPROVE + if low in ("e", "edit"): + return NEED_STEER + return DECLINE + if low in ("y", "yes"): + return APPROVE + if low in ("e", "edit"): + return NEED_STEER + return DECLINE + + +def parse_plan_choice(answer: str) -> str | None: + """Map a plan-approval keystroke to 'y' / 'e' / 'n', or None to re-prompt. + + Mirrors :meth:`TerminalUI.ask_plan_approval`: y/e/n plus empty→n; an + unrecognized non-empty token re-prompts (its loop), modeled here as None. + """ + low = answer.strip().lower() + if low in ("y", "yes"): + return "y" + if low in ("e", "edit"): + return "e" + if low in ("n", "no", ""): + return "n" + return None + + +@dataclass +class _Pending: + """The in-flight prompt. Loop-thread-only (see :class:`ApprovalGate`).""" + + future: Future[object] + # feed(line) -> True when the future was resolved (prompt done), False when + # another input line is wanted (steer/revision phase, or a re-prompt). + feed: Callable[[str], bool] + # Resolve as decline (Ctrl-C/EOF during approval cancels THIS action only). + on_cancel: Callable[[], None] + + +class ApprovalGate: + """Thread-safe approval rendezvous (design section 31.16). + + ``_pending`` is touched ONLY on the loop thread: :meth:`_enter_command` / + :meth:`_enter_plan` set it (scheduled loop-ward by the worker), and + :meth:`submit` / :meth:`cancel` read+clear it (called from keybindings, which + run on the loop thread). The worker thread only ever touches the ``Future`` + (``result()`` blocks it; the loop thread's ``set_result`` unblocks it) — both + are thread-safe stdlib primitives, so no lock is needed. + + NOTE: a pending approval at app exit leaves the daemon worker blocked on + result(); harmless (process exit reaps it). On promotion to the shipping + default, resolve pending approvals as DECLINE on exit. + """ + + def __init__(self, *, ui: AppUI, schedule: Schedule, glyphs: Glyphs = UNICODE_GLYPHS) -> None: + self._ui = ui + self._schedule = schedule + self._glyphs = glyphs + self._pending: _Pending | None = None + # Modal-dock state (§31.16): what the dock border shows while a prompt + # is active. Loop-thread-only, exactly like _pending — set by the + # _enter thunks, updated on phase transitions inside feed(), cleared on + # every resolution path (submit-done, cancel, setup/feed exceptions). + self._dock_hint: str | None = None + self._dock_risk: RiskLevel | None = None + + # ------------------------------------------------------------------ + # Worker-thread entry points — block on the future. + # ------------------------------------------------------------------ + + def ask_command(self, request: ApprovalRequest) -> ApprovalReply: + future: Future[object] = Future() + self._schedule(functools.partial(self._enter_command, request, future)) + result = future.result() # BLOCKS the worker thread + assert isinstance(result, ApprovalReply) + return result + + def ask_plan(self, plan: TaskPlan, path: str) -> tuple[str, str]: + future: Future[object] = Future() + self._schedule(functools.partial(self._enter_plan, plan, path, future)) + result = future.result() # BLOCKS the worker thread + assert isinstance(result, tuple) + return result + + # ------------------------------------------------------------------ + # Loop-thread prompt setup (scheduled by the worker entry points). + # ------------------------------------------------------------------ + + def _echo(self, line: str) -> None: + # Keep the accepted input visible after the dock clears. show_status + # re-sanitizes, so user-controlled text never reaches the pane raw; it + # also (unlike show_user_message) does NOT reset the live turn indicator. + self._ui.show_status(f" {self._glyphs.chevron} {_sanitize_line(line)}") + + def _enter_command(self, request: ApprovalRequest, future: Future[object]) -> None: + # Fail closed: a render sink raising on the loop thread must resolve the + # worker's Future (via set_exception) instead of leaving it blocked on + # result() forever — the turn then ends through TurnRunner._run's except + # ("Turn failed") rather than wedging the app. Never approves by accident. + try: + self._ui.show_approval(request) + self._ui.show_choices(approval_choices(request)) + except Exception as exc: # noqa: BLE001 - never leave the worker hung + self._clear_dock() + self._pending = None + if not future.done(): + future.set_exception(exc) + return + phase = {"steer": False} + + def feed(line: str) -> bool: + if phase["steer"]: + # Empty steer = plain decline (matches TerminalUI._read_steer). + self._echo(line) + future.set_result(ApprovalReply(approved=False, steer_text=line.strip() or None)) + return True + decision = parse_command_choice(request, line) + if isinstance(decision, _NeedSteer): + phase["steer"] = True + self._dock_hint = "tell the model what to do instead" + self._ui.show_status(" Tell the model what to do instead:") + return False + self._echo(line) + future.set_result(decision) + return True + + if request.risk is RiskLevel.HIGH and request.kind == "command": + self._dock_hint = 'type "run" to execute' + else: + self._dock_hint = "approve?" + self._dock_risk = request.risk + self._pending = _Pending(future, feed, lambda: future.set_result(DECLINE)) + + def _enter_plan(self, plan: TaskPlan, path: str, future: Future[object]) -> None: + # Fail closed (see _enter_command): a render sink raising here resolves + # the worker's Future rather than wedging it on result(). + try: + self._ui.show_plan_approval(plan, path) + self._ui.show_choices(plan_choices()) + except Exception as exc: # noqa: BLE001 - never leave the worker hung + self._clear_dock() + self._pending = None + if not future.done(): + future.set_exception(exc) + return + phase = {"revision": False} + + def feed(line: str) -> bool: + if phase["revision"]: + self._echo(line) + future.set_result(("e", line.strip())) + return True + choice = parse_plan_choice(line) + if choice is None: + # Re-prompt on an unparseable answer (matches the TerminalUI loop). + self._ui.show_choices(plan_choices()) + return False + if choice == "e": + phase["revision"] = True + self._dock_hint = "describe the changes you want" + self._ui.show_status(" Describe the changes you want:") + return False + self._echo(line) + future.set_result((choice, "")) + return True + + # A plan carries no command risk — the dock shows the amber default. + self._dock_hint = "approve plan?" + self._dock_risk = None + self._pending = _Pending(future, feed, lambda: future.set_result(("n", ""))) + + # ------------------------------------------------------------------ + # Loop-thread public surface — driven by the dock keybindings. + # ------------------------------------------------------------------ + + @property + def active(self) -> bool: + return self._pending is not None + + @property + def dock_hint(self) -> str | None: + """Short label the dock's top border carries while a prompt is active + (§31.16) — None when idle. Loop-thread-only, read per render.""" + return self._dock_hint + + @property + def dock_risk(self) -> RiskLevel | None: + """Risk coloring the dock border during an approval: HIGH → red, any + other active prompt → amber. None for a plan prompt (amber default) + and while idle. Loop-thread-only, read per render.""" + return self._dock_risk + + def _clear_dock(self) -> None: + self._dock_hint = None + self._dock_risk = None + + def submit(self, line: str) -> None: + pending = self._pending + if pending is None: + return + # Fail closed: feed() echoes via show_status BEFORE resolving the Future, + # so a sink raising there must still resolve it — never leave the worker + # blocked on result(). + try: + done = pending.feed(line) + except Exception as exc: # noqa: BLE001 - never leave the worker hung + self._clear_dock() + self._pending = None + if not pending.future.done(): + pending.future.set_exception(exc) + return + if done: + self._clear_dock() + self._pending = None + + def cancel(self) -> None: + pending = self._pending + if pending is None: + return + self._clear_dock() + self._pending = None + pending.on_cancel() diff --git a/shellpilot/cli/app_main.py b/shellpilot/cli/app_main.py new file mode 100644 index 0000000..5a17f2b --- /dev/null +++ b/shellpilot/cli/app_main.py @@ -0,0 +1,114 @@ +"""Runnable entry for the full-screen app (design section 31.13). + +This is the LIVE glue that constructs the full-screen ``Application``, drives a +:class:`~shellpilot.runtime.conversation.ConversationRuntime` through a +worker-thread :class:`~shellpilot.cli.app_turn.TurnRunner`, and ``app.run()``s +it. It is the DEFAULT path for an interactive TTY (``run_interactive`` selects it +unless ``--legacy-ui`` / ``SHELLPILOT_UI=legacy`` is set or the session is +non-TTY, in which case the legacy line-based REPL runs instead). + +The construction cycle is broken by deferred attribute assignment — see the +ordering note in :func:`run_app`. +""" + +from __future__ import annotations + +import asyncio +from collections.abc import Callable, Sequence +from pathlib import Path +from typing import TYPE_CHECKING + +from shellpilot.cli.app import StatusValues, build_app +from shellpilot.cli.app_turn import TurnRunner + +# Active-turn indicator animation cadence (§31.14). Only fires while a turn is in +# flight, so the cost is bounded to when the app is genuinely working; idle ticks +# do no work (the loop just sleeps). ~0.15s matches the plane-glide FRAME_SECONDS. +_REFRESH_SECONDS = 0.15 + +if TYPE_CHECKING: + from shellpilot.cli.app_approval import ApprovalGate + from shellpilot.cli.app_ui import AppUI + from shellpilot.cli.theme import Glyphs + from shellpilot.llm.ollama import LocalModel + from shellpilot.runtime.conversation import ConversationRuntime + + +def run_app( + runtime: ConversationRuntime, + runner: TurnRunner, + app_ui: AppUI, + *, + workspace: Path, + model: str, + profile: str, + glyphs: Glyphs, + commands: Sequence[str], + is_cloud: bool = False, + ctx_pct: int = 0, + approval_gate: ApprovalGate | None = None, + on_slash: Callable[[str], None] | None = None, + is_busy: Callable[[], bool] | None = None, + register_idle: Callable[[Callable[[], None]], None] | None = None, + status_fn: Callable[[], StatusValues] | None = None, + model_completion_models: Callable[[], list[LocalModel]] | None = None, +) -> int: + """Build the full-screen app around an already-wired conversation and run it. + + The caller has already chosen the conversation's UI as the marshaling + :class:`ThreadedUI` whose inner is ``app_ui`` and whose schedule is + ``runner.schedule`` (so the runtime's plan tools captured the marshaling + bound methods at construction). This function only completes the wiring: + + 1. ``app_ui`` was built first (its ``width_fn`` reads ``get_app()`` lazily, + so it needs no running app yet); ``runner`` was built next (its + ``schedule`` reads ``runner.app`` lazily); the conversation was built + with the ``ThreadedUI`` over ``app_ui``. + 2. Build the ``Application`` from ``runner.start`` (the dock-submit handler) + and ``app_ui`` (the pane source of truth). + 3. Set ``runner.app`` and ``runner.conversation`` AFTER the app exists and + BEFORE ``app.run()`` — both are read only once a turn runs, by which time + ``app.run()`` has set ``app.loop``. + + NOTE: ``run_app`` stays UI-only and does not itself write the ``session_end`` + audit event — the caller (``run_interactive``) writes it right after this + returns, mirroring the legacy REPL so both UIs audit the session identically. + Returns the app's exit code. + """ + app = build_app( + workspace=workspace, + model=model, + profile=profile, + glyphs=glyphs, + commands=commands, + is_cloud=is_cloud, + ctx_pct=ctx_pct, + ui=app_ui, + on_submit=runner.start, + on_interrupt=runner.request_cancel, + on_slash=on_slash, + approval_gate=approval_gate, + is_busy=is_busy, + register_idle=register_idle, + status_fn=status_fn, + model_completion_models=model_completion_models, + ) + runner.app = app + runner.conversation = runtime + + async def _refresh_loop() -> None: + # Animate the live thinking indicator while a turn runs; when idle, do + # nothing (redraws stay event-driven) so the app spends no CPU sitting at + # the prompt — replacing prompt_toolkit's unconditional refresh_interval. + while True: + await asyncio.sleep(_REFRESH_SECONDS) + if app_ui.is_animating: + app.invalidate() + + def _start_refresh() -> None: + # pre_run fires once the event loop is up, so create_background_task is safe; + # prompt_toolkit cancels the task when the app exits. + app.create_background_task(_refresh_loop()) + + app.run(pre_run=_start_refresh) + return 0 diff --git a/shellpilot/cli/app_slash.py b/shellpilot/cli/app_slash.py new file mode 100644 index 0000000..6eaf968 --- /dev/null +++ b/shellpilot/cli/app_slash.py @@ -0,0 +1,284 @@ +"""Slash / manual-shell routing for the full-screen app (design section 31.17). + +The dock-submit keybinding sends every ``/`` and ``!`` line here on the LOOP +thread (the prompt_toolkit event loop). The event loop owns the terminal and +must never block, so this router splits the work two ways: + +* **Fast, display-only** slash commands run on the loop thread against a fresh + pane-capturing :class:`~rich.console.Console`; the captured ANSI is pushed + into the pane (``AppUI.show_slash_output``). +* **Interactive / slow / own-stdout / manual-shell** commands run via + ``run_in_terminal`` (the app suspends, the real terminal is restored, the + handler runs synchronously, then the app redraws) so ``confirm()`` / + cloud-consent ``input()`` and slow model preload never freeze the TUI. + :func:`~shellpilot.cli.slash.needs_terminal` + classifies which form needs the real terminal. + +The router is built to be testable by INJECTING its effects (``dispatch``, +``run_terminal``, ``manual_shell``, ``on_exit``, ``is_busy``) — no running +prompt_toolkit app is needed in CI. +""" + +from __future__ import annotations + +import io +import shlex +from collections.abc import Callable +from pathlib import Path +from typing import IO, cast + +from rich.console import Console + +from shellpilot.cli.app_ui import AppUI +from shellpilot.cli.slash import SlashAction, needs_background, needs_terminal, needs_worker +from shellpilot.cli.theme import SHELLPILOT_THEME, UNICODE_GLYPHS, Glyphs + + +class _TeeWriter: + """Write terminal command output to the real console and a capture buffer.""" + + def __init__(self, primary: IO[str], capture: io.StringIO) -> None: + self._primary = primary + self._capture = capture + self.encoding = str(getattr(primary, "encoding", "utf-8")) + + def write(self, text: str) -> int: + written = self._primary.write(text) + self._capture.write(text) + return written + + def flush(self) -> None: + self._primary.flush() + + def isatty(self) -> bool: + return self._primary.isatty() + + def writable(self) -> bool: + return True + + +def _split_model_use(line: str) -> list[str] | None: + try: + parts = shlex.split(line) + except ValueError: + return None + if len(parts) < 2 or parts[0].lower() != "/model" or parts[1].lower() != "use": + return None + return parts + + +class SlashRouter: + """Routes a dock-submitted ``/`` or ``!`` line to the right execution context. + + ``dispatch(line, console)`` runs the app's single ``SlashDispatcher`` against + the GIVEN console (and the right confirm — see the wiring's loop-path + ``_decline`` safety net): the loop path passes a capturing console, the + terminal path passes ``real_console``. ``run_terminal`` schedules a zero-arg + fn under ``run_in_terminal``; ``manual_shell`` runs a ``!``/``/shell`` line on + the real terminal; ``on_exit`` exits the app; ``is_busy`` reports whether a + turn is in flight (a slash is rejected while busy). + """ + + def __init__( + self, + *, + ui: AppUI, + dispatch: Callable[[str, Console], SlashAction], + real_console: Console, + width_fn: Callable[[], int], + run_terminal: Callable[[Callable[[], None]], None], + run_worker: Callable[[Callable[[], None]], bool], + schedule: Callable[[Callable[[], None]], None], + manual_shell: Callable[[str], None], + run_shell: Callable[[str], tuple[int, str]], + on_exit: Callable[[], None], + is_busy: Callable[[], bool], + workspace_fn: Callable[[], Path] | None = None, + model_use_needs_terminal: Callable[[str], bool] | None = None, + glyphs: Glyphs = UNICODE_GLYPHS, + ) -> None: + self._ui = ui + self._dispatch = dispatch + self._real_console = real_console + self._width_fn = width_fn + self._run_terminal = run_terminal + self._run_worker = run_worker + # Marshal a callback from the worker thread back onto the loop thread (for + # the worker path's captured output → pane); = TurnRunner.schedule. + self._schedule = schedule + # Bare `!` / `/shell` → the interactive manual-shell loop (real terminal). + self._manual_shell = manual_shell + # One-shot `!` → run captured, returns (exit_code, output) for the pane. + self._run_shell = run_shell + self._on_exit = on_exit + self._is_busy = is_busy + self._workspace_fn = workspace_fn or Path.cwd + self._model_use_needs_terminal = model_use_needs_terminal or (lambda _line: True) + self._glyphs = glyphs + + def route(self, line: str) -> None: + # Called on the LOOP thread from the dock submit keybinding. + stripped = line.strip() + # NOTE: defense-in-depth. Since §31.18 the dock QUEUES a slash submitted + # while busy and fires it (via _fire_pending) only after the turn ends, so + # route() is reached from the dock only when idle. This guard stays as a + # fail-safe for any non-dock caller. + if self._is_busy(): + self._ui.show_status("Busy — finish or cancel the current turn first.") + return + if stripped.startswith("!"): + command = stripped[1:].strip() + if command: + # One-shot `!` → run captured on the worker and render its + # output in the pane (no app suspend, no terminal flash, §31.17). + # Echo the command first like a turn submission so the transcript + # shows what ran. The loop thread must not block on the subprocess. + self._ui.show_user_message(stripped) + self._run_worker(lambda: self._dispatch_shell(command)) + else: + # Bare `!` → the interactive manual shell, which needs the real + # terminal (it reads live stdin), so it suspends the app. + self._run_terminal(lambda: self._manual_shell(stripped)) + return + if needs_worker(stripped) or needs_background(stripped): + # Off the loop thread, both via TurnRunner.start_action: + # * needs_worker (/plan revise) — a model turn: its output reaches the + # pane via the runtime UI and approvals use the focus-swap gate, so + # echo the command first (mirrors a turn submission). + # * needs_background (/doctor, /model list, /attach ) — a + # non-interactive blocking network or filesystem I/O call: NO echo; + # its captured output IS the result and is marshaled to the pane. + # The loop thread must never block on either. Read the width here (loop + # thread) and pass it in — get_app() is unavailable on the worker. + if needs_worker(stripped): + self._ui.show_user_message(stripped) + width = self._width_fn() + self._run_worker(lambda: self._dispatch_worker(stripped, width)) + return + if self._cwd_set_can_run_in_pane(stripped): + self._dispatch_loop(stripped) + return + if self._model_use_can_run_in_pane(stripped): + width = self._width_fn() + self._run_worker(lambda: self._dispatch_worker(stripped, width)) + return + if self._model_use_can_copy_terminal_output(stripped): + width = self._width_fn() + self._run_terminal( + lambda: self._dispatch_terminal(stripped, copy_output=True, width=width) + ) + return + if needs_terminal(stripped): + self._run_terminal(lambda: self._dispatch_terminal(stripped)) + return + self._dispatch_loop(stripped) # fast display command — capture into the pane + + def _cwd_set_can_run_in_pane(self, line: str) -> bool: + try: + parts = shlex.split(line) + except ValueError: + return True + if len(parts) < 2 or parts[0].lower() != "/cwd" or parts[1].lower() != "set": + return False + if len(parts) < 3: + return True + candidate = Path(parts[2]).expanduser() + if not candidate.is_absolute(): + candidate = (self._workspace_fn() / parts[2]).resolve() + return not candidate.is_dir() + + def _model_use_can_copy_terminal_output(self, line: str) -> bool: + parts = _split_model_use(line) + return parts is not None and self._model_use_needs_terminal(line) + + def _model_use_can_run_in_pane(self, line: str) -> bool: + parts = _split_model_use(line) + if parts is None: + return False + if len(parts) < 3: + return True + return not self._model_use_needs_terminal(line) + + def _capturing_console(self, width: int) -> tuple[Console, io.StringIO]: + buf = io.StringIO() + console = Console( + file=buf, + force_terminal=True, + color_system="truecolor", + theme=SHELLPILOT_THEME, + width=width, + ) + return console, buf + + def _dispatch_loop(self, line: str) -> None: + # Fast, non-interactive: run on the loop thread with a fresh capturing + # console at the current pane width; push captured output to the pane. + console, buf = self._capturing_console(self._width_fn()) + action = self._dispatch(line, console) + self._ui.show_slash_output(buf.getvalue()) + self._after_action(action) + + def _dispatch_worker(self, line: str, width: int) -> None: + # Runs on the WORKER thread (a /plan revise turn or a slow /model list // + # /attach ). Capture the dispatcher's console and marshal any output + # to the pane on the loop thread. For /plan revise the captured output is a + # blank line (the turn's real output streams via the runtime's marshaling + # UI during the call); for the background commands it IS the result. + console, buf = self._capturing_console(width) + action = self._dispatch(line, console) + output = buf.getvalue() + self._schedule(lambda: self._deliver_worker(output, action)) + + def _deliver_worker(self, output: str, action: SlashAction) -> None: + # Loop thread (marshaled from _dispatch_worker): push the captured output + # (blank is ignored by show_slash_output), then handle the action. + self._ui.show_slash_output(output) + self._after_action(action) + + def _dispatch_shell(self, command: str) -> None: + # WORKER thread: run one `!` capturing its combined output, then + # marshal the result to the pane on the loop thread (§31.17). Blocking the + # worker (not the loop) keeps a slow command from freezing the UI. + exit_code, output = self._run_shell(command) + self._schedule(lambda: self._deliver_shell(exit_code, output)) + + def _deliver_shell(self, exit_code: int, output: str) -> None: + # Loop thread: render captured output through the sanitizing command-output + # sink (per line, control chars stripped), then a dim exit-code note when + # the command failed — mirroring the manual-shell loop's own feedback. + for line in output.splitlines(): + self._ui.show_command_output(line) + if exit_code != 0: + self._ui.show_status(f"exit code {exit_code}") + + def _dispatch_terminal( + self, line: str, *, copy_output: bool = False, width: int | None = None + ) -> None: + # Runs inside run_in_terminal (app suspended, real terminal available). + # The dispatch is given the REAL console, so confirm()/consent input() work. + if not copy_output: + action = self._dispatch(line, self._real_console) + self._after_action(action) + return + + buf = io.StringIO() + console = Console( + file=cast(IO[str], _TeeWriter(self._real_console.file, buf)), + force_terminal=True, + color_system="truecolor", + theme=SHELLPILOT_THEME, + width=width or self._width_fn(), + ) + action = self._dispatch(line, console) + self._ui.show_slash_output(buf.getvalue()) + self._after_action(action) + + def _after_action(self, action: SlashAction) -> None: + if action is SlashAction.EXIT: + self._on_exit() + elif action is SlashAction.CLEAR: + self._ui.clear_conversation("Conversation cleared.") + elif action is SlashAction.MANUAL_SHELL: + # /shell → drop into the manual shell. Already inside run_in_terminal + # (needs_terminal('/shell') is True), so call it directly. + self._manual_shell("/shell") diff --git a/shellpilot/cli/app_turn.py b/shellpilot/cli/app_turn.py new file mode 100644 index 0000000..e70d423 --- /dev/null +++ b/shellpilot/cli/app_turn.py @@ -0,0 +1,354 @@ +"""Worker-thread turn execution and loop-thread UI marshaling (design section 31.13). + +Branch 4 of the UI v2 rework. Today the runtime drives the UI synchronously on +the calling thread inside :meth:`ConversationRuntime.run_turn`; in the +full-screen app that thread is the prompt_toolkit event loop, so a long model +turn would freeze the whole UI (no repaint, no scroll, no Ctrl-C). This module +runs the one synchronous turn on a **worker thread** and marshals every UI +callback back onto the loop thread, so ``AppUI`` state and the pane repaint are +only ever touched on the loop thread. + +Two pieces, both built to be driven synchronously in CI (the ``schedule`` +callable is injected): + +* :class:`ThreadedUI` — a ``RuntimeUI`` that wraps the real ``AppUI`` and + enqueues every fire-and-forget content call onto the loop thread instead of + running it inline. The blocking approval methods cannot be fire-and-forget + (they return a value): with an :class:`ApprovalGate` wired they hand off to it + (the worker blocks on the gate's Future); with no gate they raise + ``NotImplementedError`` so an approval can never silently default. +* :class:`TurnRunner` — owns the single worker thread for one turn and a + ``busy`` flag that is only ever touched on the loop thread (no lock needed). +""" + +from __future__ import annotations + +import functools +import threading +from collections.abc import Callable +from typing import TYPE_CHECKING + +from shellpilot.llm.client import GenerationCancelled +from shellpilot.llm.ollama import describe_turn_error + +if TYPE_CHECKING: + from prompt_toolkit.application import Application + + from shellpilot.cli.app_approval import ApprovalGate + from shellpilot.cli.app_ui import AppUI + from shellpilot.policy.approvals import ApprovalReply, ApprovalRequest + from shellpilot.runtime.conversation import ConversationRuntime + from shellpilot.runtime.events import TurnStats + from shellpilot.runtime.planner import TaskPlan + +# A zero-arg callback scheduled to run on the loop thread. +Scheduled = Callable[[], None] +# Marshals a loop-thread callback from a worker thread. +Schedule = Callable[[Scheduled], None] + + +class ThreadedUI: + """A ``RuntimeUI`` that marshals every fire-and-forget call onto the loop thread. + + Wraps the real ``AppUI`` (``inner``) and a ``schedule`` callable that + enqueues a zero-arg callback to run on the prompt_toolkit event-loop thread. + The runtime calls these methods from the **worker** thread (see + :class:`TurnRunner`); this wrapper guarantees ``AppUI`` state and the pane + repaint are only ever touched on the loop thread. + + Args are captured with :func:`functools.partial` (never a closure over a + loop variable) so a later call cannot rebind them before the queued call + runs — ``stream_token("a")`` then ``stream_token("b")`` drain as ``"a"``, + ``"b"``, not ``"b"``, ``"b"``. + """ + + def __init__( + self, + *, + inner: AppUI, + schedule: Schedule, + approval_gate: ApprovalGate | None = None, + ) -> None: + self._inner = inner + self._schedule = schedule + # Branch 7 (§31.16): the focus-swap handshake for the two blocking + # approval methods. None falls back to the inner UI (back-compat — the + # inner raises NotImplementedError until a gate is wired). + self._approval_gate = approval_gate + + # ------------------------------------------------------------------ + # Fire-and-forget content methods — enqueued, never run inline. + # ------------------------------------------------------------------ + + def stream_token(self, token: str) -> None: + self._schedule(functools.partial(self._inner.stream_token, token)) + + # stream_thinking drives the live reasoning readout in AppUI (§31.14); it is + # marshaled onto the loop thread like every other content call. + def stream_thinking(self, text: str) -> None: + self._schedule(functools.partial(self._inner.stream_thinking, text)) + + # begin_response starts AppUI's turn-scoped thinking indicator (§31.14). No + # args → the bound method is already a zero-arg Scheduled, so no partial is + # needed. + def begin_response(self) -> None: + self._schedule(self._inner.begin_response) + + def end_response(self) -> None: + self._schedule(self._inner.end_response) + + def turn_finished(self, stats: TurnStats) -> None: + self._schedule(functools.partial(self._inner.turn_finished, stats)) + + def show_status(self, text: str) -> None: + self._schedule(functools.partial(self._inner.show_status, text)) + + def show_error(self, text: str) -> None: + self._schedule(functools.partial(self._inner.show_error, text)) + + def show_tool_call(self, name: str, arguments: dict[str, object]) -> None: + self._schedule(functools.partial(self._inner.show_tool_call, name, arguments)) + + def show_tool_result(self, name: str, success: bool, summary: str) -> None: + self._schedule(functools.partial(self._inner.show_tool_result, name, success, summary)) + + def show_command_output(self, line: str) -> None: + self._schedule(functools.partial(self._inner.show_command_output, line)) + + def show_plan_progress(self, plan: TaskPlan) -> None: + self._schedule(functools.partial(self._inner.show_plan_progress, plan)) + + # ------------------------------------------------------------------ + # Blocking methods — return a value, so they CANNOT be fire-and-forget. + # They delegate straight to the inner UI and run on the worker thread. + # ------------------------------------------------------------------ + + # Branch 7 (§31.16): with a gate wired, the worker blocks on the gate's + # Future while the loop thread renders the prompt and reads the dock. With no + # gate (CI back-compat only — the real app always wires one), they raise so an + # approval can never silently default; the worker's try/except surfaces it as + # a pane error (see TurnRunner._run). The inner AppUI is content-only and has + # no approval methods, hence the direct raise rather than an inner delegation. + def ask_approval(self, request: ApprovalRequest) -> ApprovalReply: + if self._approval_gate is not None: + return self._approval_gate.ask_command(request) + raise NotImplementedError("approval requires an ApprovalGate (no gate wired)") + + def ask_plan_approval(self, plan: TaskPlan, path: str) -> tuple[str, str]: + if self._approval_gate is not None: + return self._approval_gate.ask_plan(plan, path) + raise NotImplementedError("plan approval requires an ApprovalGate (no gate wired)") + + +class TurnRunner: + """Owns the single worker thread for one turn and the loop-thread busy flag. + + **Threading invariant (the core safety property of branch 4):** ``_busy`` is + SET in :meth:`start` (called on the loop thread — it is the dock-submit + handler) and CLEARED in :meth:`_mark_done` (scheduled back onto the loop + thread from the worker's ``finally`` block). It is therefore only ever read + or written on the loop thread, so no lock is needed. The worker thread + (:meth:`_run`) never touches ``_busy`` or the inner UI directly — it only + calls ``conversation.run_turn`` (whose UI is the marshaling + :class:`ThreadedUI`, so every UI call it makes is already marshaled) and + routes its own error/completion through ``schedule``. + + ``schedule`` is injected so CI can drive it synchronously; the real app uses + :meth:`schedule`, which reads :attr:`app` lazily (set after construction) so + the construction cycle — schedule needs app, app needs ``start``, the runner + needs the conversation — is broken by deferred attribute assignment. + """ + + def __init__(self, *, inner_ui: AppUI, schedule: Schedule | None = None) -> None: + # The raw content UI (NOT the marshaling ThreadedUI): the worker calls + # show_error / abort_turn on it via schedule. Typed AppUI because the + # cancel path needs abort_turn, which is an app-side AppUI method (like + # show_user_message) and deliberately not on the RuntimeUI protocol. + self._inner_ui = inner_ui + # busy: only ever touched on the loop thread (see class docstring). + self._busy = False + # The worker handle, kept so a test (and branch-6 cancellation) can join + # it. Spawned fresh per turn. + self._thread: threading.Thread | None = None + # Branch-6 per-turn cancel event (§31.15): created fresh in start(), + # signaled by request_cancel() on the loop thread, observed by the + # worker's model call. None when no turn is in flight. + self._cancel: threading.Event | None = None + # Set after construction, before app.run(), to break the build cycle; + # read only during/after run(). + self.app: Application[None] | None = None + self.conversation: ConversationRuntime | None = None + # Loop-thread callback fired once a turn ends (set after construction, + # like .app/.conversation). The input dock wires it to fire a staged + # message at turn end (§31.18); None in CI / the inert shell. + self.on_idle: Callable[[], None] | None = None + # Default to the loop-marshaling schedule (reads self.app lazily); CI + # injects a synchronous one. + self._schedule: Schedule = schedule if schedule is not None else self.schedule + + @property + def busy(self) -> bool: + """Whether a turn is in flight. Read on the loop thread (slash routing + rejects a slash while busy, §31.17), consistent with the _busy invariant.""" + return self._busy + + def schedule(self, fn: Scheduled) -> None: + """Marshal ``fn`` onto the loop thread, then request one repaint. + + Reads :attr:`app` lazily so the construction cycle is broken (``app`` is + set after the app is built). prompt_toolkit coalesces the per-call + ``invalidate()`` into one render tick, so per-token marshal+invalidate is + correct (no throttling needed — measured at the live test, branch 4). + Fails closed when the app is not running (``app``/``loop`` is None). + """ + app = self.app + if app is None: + return + loop = app.loop + if loop is None: + # NOTE: the app is shutting down — run_async clears app.loop on exit. A + # turn still in flight at /exit drops its final _mark_done here, leaving + # _busy True in an already-dead app (harmless for this opt-in dev entry). + # On promotion to the shipping default, drain/join the worker on exit. + return + + def _apply() -> None: + fn() + app.invalidate() + + loop.call_soon_threadsafe(_apply) + + def start(self, text: str) -> None: + """Spawn the worker for one turn. Runs on the loop thread (dock submit). + + Ignores the call when a turn is already in flight — single model, single + conversation, single worker; no parallel turns. A submit-while-busy is + NOT dropped: the dock stages it (one slot) and fires it here at turn end + via ``on_idle`` (§31.18). + + A fresh cancel ``Event`` is created here (before the worker spawns, so the + worker sees it) and passed into ``run_turn``; ``request_cancel`` signals + it from the loop thread to abort the turn (branch 6, §31.15). + """ + if self._busy: + return + self._busy = True + cancel = threading.Event() + self._cancel = cancel + # Pass the event as a thread ARG (a captured local), not via self._cancel, + # so the worker never reads the mutable instance attribute. `request_cancel` + # sets the SAME object from the loop thread, so the worker's local sees it — + # but the threading invariant ("the worker only touches its args + schedule; + # self._cancel/_busy are loop-thread-only") then holds literally, without + # relying on the _busy guard to keep self._cancel from being reassigned. + self._thread = threading.Thread(target=self._run, args=(text, cancel), daemon=True) + self._thread.start() + + def start_action(self, fn: Callable[[], None]) -> bool: + """Run a model-invoking slash command (e.g. ``/plan revise``) on the worker. + + Like :meth:`start` but runs an arbitrary ``fn`` — which itself drives a + model turn through the runtime's marshaling UI and the approval gate — + instead of ``run_turn`` directly. A ``/plan revise`` must NOT run on the + loop thread (it would freeze the UI and the approval-gate Future could + only be resolved by the now-blocked loop) nor under ``run_in_terminal`` + (which suspends the app while the turn marshals to it). Returns False when + a turn is already in flight so the caller can surface a hint. + + # NOTE: no cancel event — a /plan revise turn is not Ctrl-C-cancellable + # yet (it does not thread a cancel into run_turn). Fold it into the cancel + # spine if that gap bites. + """ + if self._busy: + return False + self._busy = True + self._cancel = None + self._thread = threading.Thread(target=self._run_action, args=(fn,), daemon=True) + self._thread.start() + return True + + def request_cancel(self) -> bool: + """Signal the in-flight turn to abort. Runs on the loop thread (Ctrl-C). + + Returns True when a turn was in flight (the cancel event is now set, and + the worker's next model-call read boundary will raise + ``GenerationCancelled``); False when idle (nothing to cancel — the caller + then shows the idle hint). ``_busy``/``_cancel`` are read on the loop + thread; ``Event.set()`` is the one thread-safe cross-thread signal to the + worker (raw mode disables ISIG, so Ctrl-C is a key press, not a SIGINT). + """ + if self._busy and self._cancel is not None: + self._cancel.set() + return True + return False + + def _run(self, text: str, cancel: threading.Event) -> None: + """Worker-thread body: run ONE synchronous turn off the loop thread. + + ``cancel`` is the per-turn event, received as an ARG (a captured local) so + the worker never reads ``self._cancel`` — keeping the threading invariant + literal. Touches nothing on the loop thread directly — only ``run_turn`` + (its UI is the :class:`ThreadedUI`, so every UI call is marshaled) and + ``schedule``. A ``GenerationCancelled`` (the user hit Ctrl-C mid-stream) + is a CLEAN abort, routed to ``abort_turn`` — never the error path. Any + OTHER ``run_turn`` exception (an approval-needing turn raises + ``NotImplementedError`` until branch 7, or an ``OllamaError`` surfaces) is + rendered as a pane error instead of silently killing the daemon thread. + The ``finally`` clears ``busy`` on the loop thread on EVERY path + (success/cancel/error) so a turn never wedges the app. + + NOTE (branch 6b — subprocess killpg): a cancel requested while a tool + COMMAND is running now kills that child immediately. The same cancel + event threads through ToolContext to run_command_process, which polls it + and killpg's the child's process group; the tool loop then raises + GenerationCancelled, reaching this same clean-abort path (§31.15). + """ + try: + conversation = self.conversation + if conversation is None: + # Build-order violation (conversation must be set before start()). + # Raise INSIDE the try so it surfaces as a pane error and the + # finally still clears busy — never a silent daemon death + wedge. + raise RuntimeError("TurnRunner.conversation not set before start()") + conversation.run_turn(text, cancel=cancel) + except GenerationCancelled: + # Clean Ctrl-C abort, NOT a failure: clear the dangling indicator and + # mark the partial aborted. The partial assistant reply was never + # recorded in history (run_turn let the exception propagate before the + # record site), so this is a display-only finalization. + self._schedule(self._inner_ui.abort_turn) + except Exception as exc: # noqa: BLE001 - surface ANY worker failure to the pane + # A turn failure (e.g. an Ollama 502 / read timeout) is NOT a clean + # abort: route through fail_turn so the dangling thinking indicator is + # torn down, and describe_turn_error maps the exception to a friendly, + # leak-free line — the raw upstream body (internal IPs / infra JSON) is + # never shown. + self._schedule(functools.partial(self._inner_ui.fail_turn, describe_turn_error(exc))) + finally: + self._schedule(self._mark_done) + + def _run_action(self, fn: Callable[[], None]) -> None: + """Worker body for :meth:`start_action`: run ``fn``, surface any failure to + the pane, and always clear busy on the loop thread (mirrors :meth:`_run`).""" + try: + fn() + except Exception as exc: # noqa: BLE001 - surface ANY worker failure to the pane + # Same rationale as _run: describe_turn_error maps the exception to a + # friendly, leak-free line — never the raw str(exc), which can carry an + # upstream stream body. + self._schedule( + functools.partial( + self._inner_ui.show_error, f"Command failed: {describe_turn_error(exc)}" + ) + ) + finally: + self._schedule(self._mark_done) + + def _mark_done(self) -> None: + """Clear the busy flag, then fire the idle callback. Scheduled onto the + loop thread, so the flag is only ever mutated there (paired with the set + in :meth:`start`). ``on_idle`` (when wired) fires a staged dock message at + turn end (§31.18); it runs AFTER busy clears so the fired turn sees idle.""" + self._busy = False + if self.on_idle is not None: + self.on_idle() diff --git a/shellpilot/cli/app_ui.py b/shellpilot/cli/app_ui.py new file mode 100644 index 0000000..0fc8c01 --- /dev/null +++ b/shellpilot/cli/app_ui.py @@ -0,0 +1,660 @@ +"""AppUI: RuntimeUI implementation that routes content into the full-screen app pane. + +This is the content seam for UI v2 (design section 31.12). ``AppUI`` holds a list +of Rich renderables as the transcript source of truth, renders them to a single +ANSI string at the shared terminal width, and caches the result until the content +or the width changes. The pane control in ``app.py`` is a ``FormattedTextControl`` +over ``ANSI(ui._render_ansi())``, so Rich handles all wrapping and theming at width +W while prompt_toolkit handles layout and scrolling. + +``AppUI`` is a pure state-and-render object: it never calls ``get_app()`` or +``invalidate()`` — periodic repaint is the app's job (``run_app``'s gated refresh +loop, which invalidates only while a turn is in flight) and the clock is injected +(``time_fn``) — so it is fully testable without a running app. +""" + +from __future__ import annotations + +import io +import math +import time +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path +from typing import IO, TYPE_CHECKING, cast + +from rich.console import Console, Group, RenderableType +from rich.padding import Padding +from rich.text import Text + +from shellpilot.cli.render import ( + _sanitize_line, + approval_card, + diff_row_count, + plan_panel, + plan_step_line, + render_diff, + response_markdown, + tool_call_block, +) +from shellpilot.cli.render import tool_result as render_tool_result +from shellpilot.cli.streaming import DiffReveal, phase_for_elapsed +from shellpilot.cli.theme import SHELLPILOT_THEME, UNICODE_GLYPHS, Glyphs +from shellpilot.memory.redaction import redact_structure +from shellpilot.runtime.budget import CHARS_PER_TOKEN +from shellpilot.tools.base import workspace_display + +if TYPE_CHECKING: + from shellpilot.policy.approvals import ApprovalRequest + from shellpilot.runtime.events import TurnStats + from shellpilot.runtime.planner import TaskPlan + +# The plane glyph advances one track cell every FRAME_SECONDS of elapsed time, so +# the glide is a pure function of the (injected) clock — no animation thread. +FRAME_SECONDS = 0.15 + +# A collapsed thinking trail shows this many non-blank reasoning lines; the rest +# fold behind a "+N hidden lines" footer until the trail is clicked (design §31.19). +TRAIL_COLLAPSED_LINES = 6 + + +def _fmt_count(n: int) -> str: + """k/m abbreviation for token counts (design section 31.14). + + ``999`` → ``"999"``; ``1800`` → ``"1.8k"``; ``2_400_000`` → ``"2.4m"``. The + 1m check precedes the 1k check so a seven-figure count never renders as ``k``. + """ + if n >= 1_000_000: + return f"{n / 1_000_000:.1f}m" + if n >= 1000: + return f"{n / 1000:.1f}k" + return str(n) + + +@dataclass +class _TurnIndicator: + """Turn-scoped live-frontier state (design section 31.14). + + ``start`` is the injected-clock timestamp of the turn's first model call; + ``reasoning_chars`` accumulates the length of every thinking chunk. The token + estimate is derived at render time (``ceil(chars / CHARS_PER_TOKEN)``, + consistent with ``budget.estimate_tokens``) — chars are stored, not tokens. + """ + + start: float + reasoning_chars: int = 0 + + +@dataclass +class _Trail: + """An inline, display-only thinking trail for one reasoning phase (§31.19). + + ``text`` accumulates raw model thinking (display only — never fed back to the + model or recorded in history). ``expanded`` is the per-trail collapse state + that a CLICK on the trail toggles (toggle_at, §31.16). The collapsed view shows + the first ``TRAIL_COLLAPSED_LINES`` non-blank lines; a footer reports the hidden + remainder. No ``finished`` flag is needed — a trail stops accumulating the + moment it is no longer ``AppUI._active_trail``. + """ + + text: str = "" + expanded: bool = False + + +@dataclass +class _Diff: + """A collapsible diff panel in the transcript (§31.16). + + ``diff_text`` is the raw unified diff; ``expanded`` is the per-diff collapse + state. Collapsed shows the first ``DiffReveal.WINDOW_ROWS`` rows; expanded + shows all. The toggle target is found by CLICK (the panel's transcript line + range) or the Ctrl-O keyboard fallback (the latest diff). ``total_rows`` is + the rendered row count, captured once at construction so the click/collapse + hint is suppressed for a diff that already fits — no re-parse per frame. + """ + + diff_text: str + total_rows: int + expanded: bool = False + + +class _LineCountingWriter: + """A text sink that tallies newlines as Rich writes through it. + + Lets ``_render_ansi`` record each element's transcript line range in the SAME + single render pass (one Console, not one per element) — the render runs every + refresh tick during an active turn, so the per-element-Console alternative + would multiply Console construction by the transcript length each frame. + """ + + def __init__(self) -> None: + self._buf = io.StringIO() + self.lines = 0 + + def write(self, text: str) -> int: + self.lines += text.count("\n") + return self._buf.write(text) + + def flush(self) -> None: + self._buf.flush() + + def getvalue(self) -> str: + return self._buf.getvalue() + + +class AppUI: + """RuntimeUI implementation that routes content into the full-screen app pane. + + All methods must be called from the UI thread. Cross-thread marshaling and + ``app.invalidate()`` are branch-4 concerns; this class stays pure state+render. + """ + + def __init__( + self, + *, + glyphs: Glyphs = UNICODE_GLYPHS, + workspace: Path | None = None, + workspace_fn: Callable[[], Path] | None = None, + width_fn: Callable[[], int], + show_reasoning: bool = True, + time_fn: Callable[[], float] = time.monotonic, + intro: RenderableType | None = None, + ) -> None: + self._glyphs = glyphs + # Workspace for display-integrity (design section 14.5): when set, a + # `path` argument in the tool-call line is shown as its resolved, + # workspace-relative target — the SAME resolution the tool acts on. + # workspace_fn (preferred in production) is called at render time so a + # mid-session /cwd change is immediately reflected; workspace is the + # static fallback for test doubles that construct without a live runtime. + self._workspace = workspace + self._workspace_fn = workspace_fn + self._width_fn = width_fn + # Gate for the reasoning-token readout (settings.ui.show_reasoning_summary, + # design section 31.14): when False, the live/done lines show plane+phrase+ + # timer only — no reasoning estimate, no total. Display-only; audit capture + # of thinking is unaffected. + self._show_reasoning = show_reasoning + # Injected clock so the indicator's elapsed/animation is testable with a + # fixed time_fn rather than the wall clock. + self._time_fn = time_fn + # Durable pane chrome. The boot banner is not conversation history, so it + # is restored after /clear even though transcript renderables are dropped. + self._intro = intro + # Source of truth: every committed renderable in the transcript. A _Trail + # entry is a live thinking block rendered via _render_trail (§31.19). The + # boot banner (when provided) is seeded as the first transcript entry so it + # renders inside the alt-screen pane — a console.print would be lost behind + # the full-screen app (§31.13). + self._renderables: list[RenderableType | _Trail | _Diff] = self._initial_renderables() + # Accumulated token text for the current open response. + # None means no response is open; an open response is the last renderable + # in spirit but lives here until end_response() finalizes it. + self._open_response: str | None = None + # Turn-scoped live indicator (design section 31.14): None when idle, a + # _TurnIndicator while a turn runs. begin_response starts it; turn_finished + # freezes it to a permanent done line and clears it. + self._indicator: _TurnIndicator | None = None + # Inline thinking trails (§31.19): _active_trail is the one currently + # accumulating stream_thinking text (None between reasoning phases). A + # trail is toggled by CLICKING it (toggle_at), so there is no latest-trail + # pointer — older trails stay individually reachable. _active_trail is part + # of _renderables; this is just a pointer into it. + self._active_trail: _Trail | None = None + # Latest diff in the transcript — the Ctrl-O keyboard fallback's target + # (§31.16). Clicking reaches any diff; Ctrl-O only the most recent one. + self._latest_diff: _Diff | None = None + # Transcript line ranges of the click-toggleable elements (trails + diffs), + # rebuilt whenever the ANSI is rebuilt (see _render_ansi). A pane click maps + # its row → the element whose [start, end) range contains it (toggle_at). + self._toggle_ranges: list[tuple[int, int, _Trail | _Diff]] = [] + # Width-keyed ANSI cache: (width, ansi_string), or None when stale. + self._cache: tuple[int, str] | None = None + # Whether the idle hint is already the last thing shown — so repeated + # Ctrl-C while idle doesn't stack it (§31.17). Re-armed at each new turn. + self._idle_hint_shown = False + + @property + def is_animating(self) -> bool: + """True while a turn's live indicator should keep animating (a turn is in + flight). ``run_app``'s gated refresh loop polls this to invalidate the app + ONLY during a turn, so an idle app schedules no timer redraws (§31.14).""" + return self._indicator is not None + + # ------------------------------------------------------------------ + # Internal helpers + # ------------------------------------------------------------------ + + def _initial_renderables(self) -> list[RenderableType | _Trail | _Diff]: + return [self._intro] if self._intro is not None else [] + + def _close_open_response(self) -> None: + """Finalize the open response as a Markdown renderable, if one is open.""" + if self._open_response is not None: + self._renderables.append(response_markdown(self._open_response)) + self._open_response = None + self._cache = None + + def _finalize_active_trail(self) -> None: + # Any non-thinking transcript content ends the active reasoning phase. The + # trail STAYS in _renderables (finished trails remain visible and stay + # click-toggleable individually, regardless of age); it just stops + # accumulating. This is the ONLY cleanup the spec's "clear active unfinished + # trail state, never reset prior finished trails" needs — we touch a single + # pointer, never another trail's expanded state. + self._active_trail = None + + def _add_renderable(self, renderable: RenderableType) -> None: + """Close any open response first, then append a renderable to the transcript. + + Preserves ordering: response → tool call → response produces three distinct + transcript entries (the open response closes around the tool call). + """ + self._finalize_active_trail() + self._close_open_response() + self._renderables.append(renderable) + self._cache = None + + def _render_ansi(self) -> str: + """Render the full transcript to an ANSI string at the current terminal width. + + The width is read from ``width_fn`` on every call and compared against the + cache key; a resize automatically re-derives the ANSI (Rich re-wraps at the + new width) and updates the cache. ``AppUI`` never calls ``get_app()``; the + caller supplies the width source via ``width_fn`` at construction time. + """ + width = self._width_fn() + active = self._indicator is not None + # An active indicator's elapsed/animation changes every render, so it + # bypasses the width cache entirely (never read, never write). Idle + # renders stay cached by width — a refresh tick with no active turn is a + # cheap cache hit. NOTE: gating refresh on an active turn is the upgrade + # path (branch 5) if profiling ever shows the idle tick costs anything. + if not active and self._cache is not None and self._cache[0] == width: + return self._cache[1] + writer = _LineCountingWriter() + console = Console( + # Rich only needs `.write`/`.flush`; the writer tallies lines as it goes. + file=cast("IO[str]", writer), + force_terminal=True, + color_system="truecolor", + theme=SHELLPILOT_THEME, + width=width, + ) + renderables: list[RenderableType | _Trail | _Diff] = list(self._renderables) + if self._open_response is not None: + # Include in-progress response without committing it yet. Sanitize the + # model text at the sink (mirrors ResponseStream, streaming.py:171) — + # _sanitize_line keeps LF, so markdown structure survives. + renderables.append(response_markdown(self._open_response)) + if active: + # The live frontier line always renders LAST, below all completed + # content, so it "moves down" as tool calls/responses append above it. + renderables.append(self._indicator_line()) + # Rebuild the click-toggle line index alongside the ANSI: a trail/diff + # occupies document lines [start, end) and toggle_at(y) maps a pane click + # back to it. Kept in lockstep with the ANSI we are about to return. + ranges: list[tuple[int, int, _Trail | _Diff]] = [] + for renderable in renderables: + start = writer.lines + if isinstance(renderable, _Trail): + console.print(self._render_trail(renderable)) + ranges.append((start, writer.lines, renderable)) + elif isinstance(renderable, _Diff): + console.print(self._render_diff(renderable)) + ranges.append((start, writer.lines, renderable)) + else: + console.print(renderable) + self._toggle_ranges = ranges + ansi = writer.getvalue() + if not active: + self._cache = (width, ansi) + return ansi + + def _indicator_line(self) -> Text: + """The live frontier line for the active turn (design section 31.14). + + ``{plane} {phrase}… {N}s · {fmt(reasoning)} reasoning`` — the plane glides + a track cell every ``FRAME_SECONDS`` and the phrase is a DETERMINISTIC pick + from the current flight phase's pool (10-second buckets, not random), so + the whole line is reproducible under a fixed ``time_fn``. + """ + indicator = self._indicator + assert indicator is not None # only called from _render_ansi when active + elapsed = self._time_fn() - indicator.start + frames = self._glyphs.spinner_frames + plane = frames[int(elapsed / FRAME_SECONDS) % len(frames)] + pool = phase_for_elapsed(elapsed).pool + phrase = pool[int(elapsed / 10) % len(pool)] + line = Text() + line.append(plane, style="sp.accent") + line.append(f" {phrase}{self._glyphs.ellipsis} {int(elapsed)}s", style="sp.dim") + if self._show_reasoning: + reasoning = math.ceil(indicator.reasoning_chars / CHARS_PER_TOKEN) + line.append(f" · {_fmt_count(reasoning)} reasoning", style="sp.dim") + return line + + def _render_trail(self, trail: _Trail) -> Text: + # Display-only thinking (§31.19), faint + indented, header carries the + # reasoning-token estimate. Model-controlled text → EVERY line sanitized. + # Blank lines are dropped so TRAIL_COLLAPSED_LINES counts real reasoning lines. + lines = [ln for ln in trail.text.splitlines() if ln.strip()] + reasoning = math.ceil(len(trail.text) / CHARS_PER_TOKEN) + caret = ( + ("▾" if trail.expanded else "▸") + if self._glyphs == UNICODE_GLYPHS + else ("v" if trail.expanded else ">") + ) + parts: list[Text] = [ + Text(f"{caret} thinking · {_fmt_count(reasoning)} reasoning", style="sp.dim") + ] + shown = lines if trail.expanded else lines[:TRAIL_COLLAPSED_LINES] + parts.extend(Text(" " + _sanitize_line(ln), style="sp.faint") for ln in shown) + if trail.expanded: + parts.append(Text(" click to collapse", style="sp.faint")) + else: + hidden = len(lines) - len(shown) + if hidden > 0: + parts.append( + Text( + f" {self._glyphs.ellipsis} +{hidden} hidden lines · click to expand", + style="sp.faint", + ) + ) + return Text("\n").join(parts) + + def _render_diff(self, d: _Diff) -> RenderableType: + # Collapsible diff panel (§31.16). Standardized to the pane width (minus the + # 2-col indent) with long lines folded; diff text is sanitized inside + # render_diff/_diff_rows, so this is a safe sink. Collapsed caps at + # DiffReveal.WINDOW_ROWS; expanded shows all rows. The click/Ctrl-O hint is + # added only when the diff overflows the cap (nothing to toggle otherwise). + panel = render_diff( + d.diff_text, + self._glyphs, + width=max(1, self._width_fn() - 2), + max_rows=None if d.expanded else DiffReveal.WINDOW_ROWS, + ) + body: RenderableType = panel + if d.total_rows > DiffReveal.WINDOW_ROWS: + hint = "click or ctrl-o to collapse" if d.expanded else "click or ctrl-o to expand" + body = Group(panel, Text(hint, style="sp.faint")) + return Padding(body, (0, 0, 0, 2)) + + # ------------------------------------------------------------------ + # RuntimeUI content methods — mirroring TerminalUI exactly + # ------------------------------------------------------------------ + + def stream_token(self, token: str) -> None: + """Accumulate a streaming token into the open response.""" + # A response token does NOT end the reasoning phase (§31.19). A model may + # emit a trailing thought AFTER the answer starts; that fragment must join + # the SAME trail, not cut the answer into two blocks. The trail is finalized + # at the model-call boundary (end_response) instead of on the first token. + if self._open_response is None: + self._open_response = token + else: + self._open_response += token + self._cache = None + + def end_response(self) -> None: + """Close the open response and finalize the call's reasoning trail (§31.19). + + end_response bounds one model call (conversation.py wraps each chat() in + begin_response/end_response). Finalizing the active trail HERE — not on the + first response token — is what keeps interleaved thinking/answer within a + call as one trail + one response, while a genuine second reasoning phase in + the NEXT call still opens its own fresh trail. + """ + self._close_open_response() + self._finalize_active_trail() + + def begin_response(self) -> None: + # Turn-scoped: the FIRST model call of a turn starts the live indicator; + # later calls within the same tool loop do NOT restart it (the elapsed + # timer and reasoning count span the whole turn, not each model call). + if self._indicator is None: + self._indicator = _TurnIndicator(start=self._time_fn()) + self._cache = None + + def turn_finished(self, stats: TurnStats) -> None: + # Freeze the live frontier into a permanent done line and clear the active + # indicator. Elapsed comes from the runtime's authoritative stats.elapsed_s + # (not the UI clock); reasoning is frozen from the accumulated chars; total + # is the exact summed output-token count. + # NOTE: only the success path freezes — a turn that errors before + # turn_finished (or a Ctrl-C cancel) leaves the indicator active; that + # turn-failure/cancel robustness is branch 6's. + reasoning_chars = self._indicator.reasoning_chars if self._indicator is not None else 0 + self._indicator = None + line = Text() + line.append(self._glyphs.check, style="sp.accent") + line.append(" done", style="sp.dim") + line.append(f" · {int(stats.elapsed_s)}s", style="sp.dim") + if self._show_reasoning: + reasoning = math.ceil(reasoning_chars / CHARS_PER_TOKEN) + line.append(f" · {_fmt_count(reasoning)} reasoning", style="sp.dim") + line.append(f" · {_fmt_count(stats.output_tokens)} total", style="sp.dim") + self._add_renderable(line) + + def abort_turn(self) -> None: + # Branch 6 (§31.15): a turn was cancelled mid-stream (Ctrl-C). App-side + # (NOT a RuntimeUI protocol method) — TurnRunner calls it on the raw AppUI. + # Clear the dangling live indicator (stop the timer/plane) FIRST, then + # append the aborted marker. _add_renderable closes the open response + # first, so any partial streamed text stays visible — finalized — with + # the marker line below it. The partial assistant reply is never recorded + # in history; that discard is the runtime's (run_turn lets + # GenerationCancelled propagate before the record site). This is purely + # display: mark what the user already saw as stopped. + self._indicator = None + # `⏹` is not in the Glyphs set; gate it on the same unicode/ascii signal + # build_app uses, falling back to the ASCII cross glyph. + marker = "⏹" if self._glyphs == UNICODE_GLYPHS else self._glyphs.cross + self._add_renderable(Text(f"{marker} aborted", style="sp.warn")) + + def fail_turn(self, message: str) -> None: + # A turn RAISED (e.g. a network/API error), NOT a clean user Ctrl-C: tear + # down the dangling live indicator (stop the timer/plane) and surface the + # error. No "aborted" marker — that's reserved for a user cancel + # (abort_turn); this is a failure. show_error → _add_renderable closes the + # open response first, so any partial streamed text stays visible above + # the error line. Without this the indicator dangled after a turn error. + self._indicator = None + self.show_error(message) + + def toggle_at(self, line: int) -> bool: + # Map a pane click (its document row) to the trail/diff whose transcript + # line range contains it and flip that element's collapse state (§31.16/ + # §31.19). Clicking reaches ANY element — including older ones scrolled + # back — which a single "latest" keybinding cannot. Returns False when the + # click landed outside every toggleable element, so the caller lets the + # default mouse handling (e.g. scroll) run. + for start, end, element in self._toggle_ranges: + if start <= line < end: + element.expanded = not element.expanded + self._cache = None + return True + return False + + def toggle_latest_diff(self) -> bool: + # Flip the most-recent diff's collapse state — the Ctrl-O keyboard fallback + # for terminals without mouse reporting (§31.16). Returns False (a harmless + # no-op) when no diff exists yet. + if self._latest_diff is None: + return False + self._latest_diff.expanded = not self._latest_diff.expanded + self._cache = None + return True + + def stream_thinking(self, text: str) -> None: + # The reasoning count climbs ONLY while the model is thinking, so it freezes + # naturally when thinking stops and resumes if it thinks again. A no-op when + # no turn is active (defensive; begin_response runs first in the tool loop). + if self._indicator is None: + return # no active turn — nothing to attribute the thinking to + self._indicator.reasoning_chars += len(text) + self._cache = None + # Inline trail (§31.19): retain the thinking TEXT for display only (never fed + # back to the model). Gated on show_reasoning — when off, no trail is built + # (the readout is hidden too). A new reasoning phase (no active trail) opens a + # fresh trail block at the current transcript position; close any open + # response first so the block lands after streamed answer text. + if not self._show_reasoning: + return + if self._active_trail is None: + # First reasoning fragment of this model call → open a fresh trail at the + # current transcript position. Do NOT close the open response: a trailing + # thought after the answer started joins the call's reasoning, it does not + # cut the answer (§31.19). The trail resets at end_response. + trail = _Trail() + self._renderables.append(trail) + self._active_trail = trail + self._active_trail.text += text + self._cache = None + + def show_user_message(self, text: str) -> None: + # Echo the submitted user message into the transcript. App-side (NOT a + # RuntimeUI protocol method) — the full-screen analogue of the REPL + # leaving the typed line in scrollback. The user-controlled text is + # control-char sanitized (display-integrity) before it reaches the pane. + # + # A new turn begins here, so discard any indicator left dangling by a + # PRIOR turn that errored before turn_finished (the error was already + # surfaced via show_error). Without this, the next begin_response would be + # a no-op against the stale indicator and the new turn's timer would count + # from the old turn's start. Full turn-failure/cancel UX is branch 6's; this + # is the minimal guard so an error never poisons the following turn. + self._indicator = None + self._idle_hint_shown = False # a new turn re-arms the idle hint + # Breathing room between turns (§31.12): a blank spacer above the echo + # keeps consecutive turns from blurring together. The first message + # into an empty pane (or over only an open response) skips it. + if self._renderables: + self._add_renderable(Text("")) + # Brightness hierarchy: the chevron carries the accent, the user's own + # words render bright — not tinted green like harness machinery. + echo = Text.assemble( + (f"{self._glyphs.chevron} ", "sp.chevron"), + (_sanitize_line(text), "sp.emph"), + ) + self._add_renderable(echo) + + def show_idle_hint(self, text: str) -> None: + # An idle Ctrl-C hint (§31.17). Deduped: repeated Ctrl-C while idle shows + # it once, not a growing stack — the flag re-arms only at the next turn + # (show_user_message) or a /clear. The text is sanitized like any status. + if self._idle_hint_shown: + return + self._idle_hint_shown = True + self.show_status(text) + + def show_slash_output(self, text: str) -> None: + # Slash output rendered by the dispatcher's capturing console (ANSI) + # becomes a pane renderable (§31.17). Text.from_ansi parses the ANSI + # styling back into a Rich Text so the pane re-emits it. _add_renderable + # closes any open response first, so the slash block lands after it. + # NOTE: captured at the call-time width; a later resize will not re-wrap + # this block (acceptable for slash output). + stripped = text.rstrip("\n") + if stripped: + self._add_renderable(Text.from_ansi(stripped)) + + def clear_conversation(self, message: str | None = None) -> None: + """Reset the visible pane after the runtime history has been cleared.""" + self._renderables = self._initial_renderables() + self._open_response = None + self._indicator = None + self._active_trail = None + self._latest_diff = None + self._toggle_ranges = [] + self._cache = None + self._idle_hint_shown = False # a cleared pane re-arms the idle hint + if message: + self.show_status(message) + + def show_status(self, text: str) -> None: + self._add_renderable(Text(_sanitize_line(text), style="sp.dim")) + + def show_choices(self, choices: Text) -> None: + # The styled approval/plan choice line (colored y/e/n) — already built by + # render.approval_choices / plan_choices from hardcoded tokens (no user or + # model content), so it carries its own styling and needs no sanitization. + self._add_renderable(choices) + + def show_error(self, text: str) -> None: + self._add_renderable(Text(_sanitize_line(text), style="sp.error")) + + def show_tool_call(self, name: str, arguments: dict[str, object]) -> None: + # Redact secrets so an auto-approved tool call never exposes credentials + # in the visible pane channel. tool_call_block frames the actual + # command/url (run_command, web_fetch) or shows a clean inline subject + # (paths/queries) instead of the name(args) repr (§31.3); a `path` subject + # is the resolved, workspace-relative target so it can't be spoofed and + # matches the file actually touched (§14.5). A framed tool call appends + # two renderables (header + box); _add_renderable closes the open response + # on the first, keeping the response→tool-call ordering intact. + redacted = redact_structure(arguments) + assert isinstance(redacted, dict) + for renderable in tool_call_block( + name, redacted, self._glyphs, path_display=self._path_display + ): + self._add_renderable(renderable) + + def _path_display(self, path: str) -> str: + # Resolve a `path` argument to its workspace-relative target (§14.5). + # Prefer the live workspace (workspace_fn, set in production) so a + # mid-session /cwd is honoured; fall back to the build-time workspace, + # then verbatim (a test-double with neither set — production always wires + # workspace_fn, so the path display never drifts from the action). + workspace = self._workspace_fn() if self._workspace_fn is not None else self._workspace + return workspace_display(workspace, path) if workspace is not None else path + + def show_tool_result(self, name: str, success: bool, summary: str) -> None: + self._add_renderable(render_tool_result(success, summary, self._glyphs)) + + def show_command_output(self, line: str) -> None: + # " " prefix + control-char sanitization + dim, no markup, no highlight — + # mirroring TerminalUI (markup=False/highlight=False are implied by Text). + self._add_renderable(Text(" " + _sanitize_line(line), style="sp.dim")) + + def show_plan_progress(self, plan: TaskPlan) -> None: + # Uses plan_step_line (not plan_panel) with 2-cell left indent, then a blank + # line — mirroring TerminalUI.show_plan_progress exactly. This is the one + # content-appender that bypasses _add_renderable, so it finalizes the active + # trail itself — keeping the §31.19 invariant (any non-thinking content ends + # the reasoning phase) uniformly true rather than relying on the caller. + self._finalize_active_trail() + self._close_open_response() + for index, step in enumerate(plan.steps, 1): + self._renderables.append( + Padding(plan_step_line(index, step, self._glyphs), (0, 0, 0, 2)) + ) + self._renderables.append(Text("")) # blank line separator + self._cache = None + + # ------------------------------------------------------------------ + # Approval prompt content (design section 31.16). The blocking handshake + # lives in ApprovalGate; these only append the prompt block to the pane, + # mirroring the render block of TerminalUI.ask_approval / ask_plan_approval + # MINUS the DiffReveal animation — the pane scroll replaces the Rich Live. + # ------------------------------------------------------------------ + + def show_approval(self, request: ApprovalRequest) -> None: + if request.diff: + # A stateful, click/Ctrl-O-collapsible diff (§31.16) — rendered at the + # pane width by _render_diff each frame, so it tracks resizes and its + # collapse toggle. total_rows is captured once here (not per frame). + self._close_open_response() + diff = _Diff(request.diff, total_rows=diff_row_count(request.diff, self._glyphs)) + self._renderables.append(diff) + self._latest_diff = diff + self._cache = None + # One risk-bordered card (§31.5): badge + WHY/EFFECT + CWD as a single + # decision zone, matching the dock border's approval color. + self._add_renderable(approval_card(request)) + + def show_plan_approval(self, plan: TaskPlan, path: str) -> None: + self._add_renderable(plan_panel(plan, self._glyphs)) + # The path is user-visible state reaching the pane → sanitize it. + self._add_renderable(Text(_sanitize_line(path), style="sp.faint")) diff --git a/shellpilot/cli/commands.py b/shellpilot/cli/commands.py index 2d4c85e..017c0a3 100644 --- a/shellpilot/cli/commands.py +++ b/shellpilot/cli/commands.py @@ -37,6 +37,11 @@ def build_parser() -> argparse.ArgumentParser: metavar="NAME", help="Model to use for this session (skips the boot picker).", ) + parser.add_argument( + "--legacy-ui", + action="store_true", + help="Use the classic line-based REPL instead of the full-screen UI.", + ) subparsers = parser.add_subparsers(dest="command") subparsers.add_parser( "doctor", help="Check local prerequisites (Python, Ollama, models, paths)." @@ -63,7 +68,12 @@ def run_cli(argv: Sequence[str] | None = None) -> int: from shellpilot.cli.terminal import run_interactive - return run_interactive(workspace, resume=args.resume, model_override=args.model) + return run_interactive( + workspace, + resume=args.resume, + model_override=args.model, + legacy_ui=args.legacy_ui, + ) def _run_config(workspace: Path, action: str) -> int: diff --git a/shellpilot/cli/doctor.py b/shellpilot/cli/doctor.py index 867f1c4..8aa95bb 100644 --- a/shellpilot/cli/doctor.py +++ b/shellpilot/cli/doctor.py @@ -112,6 +112,8 @@ def run_doctor( workspace: Path, paths: AppPaths | None = None, client: OllamaClient | None = None, + *, + console: Console | None = None, ) -> int: owns_client = client is None resolved_client = client or OllamaClient(timeout_seconds=3.0) @@ -120,5 +122,5 @@ def run_doctor( finally: if owns_client: resolved_client.close() - render(results, Console()) + render(results, console or Console()) return 0 if all(result.ok for result in results) else 1 diff --git a/shellpilot/cli/manual_shell.py b/shellpilot/cli/manual_shell.py index bc06c62..084b573 100644 --- a/shellpilot/cli/manual_shell.py +++ b/shellpilot/cli/manual_shell.py @@ -25,21 +25,49 @@ EXIT_COMMAND = "/exit-shell" -def run_manual_command(command: str, cwd: Path, audit: AuditLogger | None) -> int: - """Run one user-typed command with shell=True, streaming to the terminal.""" - completed = subprocess.run( # noqa: S602 - raw shell is this mode's explicit contract - command, shell=True, cwd=cwd, check=False - ) +def _audit_manual_command(audit: AuditLogger | None, command: str, exit_code: int) -> None: + """Single source of truth for the manual-shell audit event shape.""" if audit is not None: audit.write( "manual_shell_command", command=command, - exit_code=completed.returncode, + exit_code=exit_code, risk="raw_shell", ) + + +def run_manual_command(command: str, cwd: Path, audit: AuditLogger | None) -> int: + """Run one user-typed command with shell=True, streaming to the terminal.""" + completed = subprocess.run( # noqa: S602 - raw shell is this mode's explicit contract + command, shell=True, cwd=cwd, check=False + ) + _audit_manual_command(audit, command, completed.returncode) return completed.returncode +def run_manual_command_captured( + command: str, cwd: Path, audit: AuditLogger | None +) -> tuple[int, str]: + """Run one ``!`` with shell=True, CAPTURING its combined output. + + Same audit as :func:`run_manual_command`, but stdout+stderr are captured + (stderr interleaved into stdout in real order) instead of streaming to the + inherited terminal, so the full-screen app can render the output into its + pane rather than flashing the real terminal (§31.17). + """ + completed = subprocess.run( # noqa: S602 - raw shell is this mode's explicit contract + command, + shell=True, + cwd=cwd, + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + _audit_manual_command(audit, command, completed.returncode) + return completed.returncode, completed.stdout or "" + + def manual_shell_loop( console: Console, cwd: Path, diff --git a/shellpilot/cli/model_completion.py b/shellpilot/cli/model_completion.py new file mode 100644 index 0000000..166dd03 --- /dev/null +++ b/shellpilot/cli/model_completion.py @@ -0,0 +1,57 @@ +"""Model-name completion for `/model use` arguments.""" + +from __future__ import annotations + +import shlex +from dataclasses import dataclass + +from shellpilot.config.model import is_tested_model +from shellpilot.llm.ollama import LocalModel + + +@dataclass(frozen=True) +class ModelCompletionMatch: + label: str + fill: str + hint: str + + +_MODEL_USE_PREFIX = "/model use " + + +def model_completion_matches( + text: str, models: list[LocalModel], *, limit: int = 20 +) -> list[ModelCompletionMatch]: + lowered = text.lower() + if not lowered.startswith(_MODEL_USE_PREFIX): + return [] + raw_prefix = text[len(_MODEL_USE_PREFIX) :] + if "\n" in raw_prefix: + return [] + try: + parsed = shlex.split(raw_prefix) + except ValueError: + parsed = [] + name_prefix = parsed[0] if len(parsed) == 1 else raw_prefix + + matches: list[ModelCompletionMatch] = [] + for model in models: + if not model.name.startswith(name_prefix): + continue + hint = f"{model.size_bytes / 1e9:.1f} GB" + if not is_tested_model(model.name): + hint += " untested" + matches.append( + ModelCompletionMatch( + label=model.name, + fill=_MODEL_USE_PREFIX + _escape_model_name(model.name), + hint=hint, + ) + ) + if len(matches) >= limit: + break + return matches + + +def _escape_model_name(name: str) -> str: + return name.replace("\\", "\\\\").replace(" ", "\\ ") diff --git a/shellpilot/cli/model_completion_cache.py b/shellpilot/cli/model_completion_cache.py new file mode 100644 index 0000000..74b6cc4 --- /dev/null +++ b/shellpilot/cli/model_completion_cache.py @@ -0,0 +1,41 @@ +"""Thread-safe model list cache for prompt completion.""" + +from __future__ import annotations + +import threading +from typing import Protocol + +from shellpilot.llm.ollama import LocalModel + + +class _ModelLister(Protocol): + def list_models(self) -> list[LocalModel]: ... + + +class ModelCompletionCache: + def __init__(self, models: list[LocalModel]) -> None: + self._lock = threading.Lock() + self._models = list(models) + + def snapshot(self) -> list[LocalModel]: + with self._lock: + return list(self._models) + + def refresh_from(self, client: _ModelLister) -> None: + try: + models = client.list_models() + except Exception: # noqa: BLE001 - stale completions are better than a noisy prompt + return + with self._lock: + self._models = list(models) + + +def refresh_model_completion_cache_in_background( + cache: ModelCompletionCache, client: _ModelLister +) -> None: + thread = threading.Thread( + target=lambda: cache.refresh_from(client), + name="model-completion-refresh", + daemon=True, + ) + thread.start() diff --git a/shellpilot/cli/path_completion.py b/shellpilot/cli/path_completion.py new file mode 100644 index 0000000..246108e --- /dev/null +++ b/shellpilot/cli/path_completion.py @@ -0,0 +1,108 @@ +"""Filesystem path completion for slash-command arguments.""" + +from __future__ import annotations + +import shlex +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class PathCompletionMatch: + label: str + fill: str + + +_PATH_COMMAND_PREFIXES = ("/cwd set ", "/attach ", "/export ") + + +def path_completion_matches( + text: str, workspace: Path, *, limit: int = 20 +) -> list[PathCompletionMatch]: + parsed = _path_argument(text) + if parsed is None: + return [] + command_prefix, raw_path = parsed + if "\n" in raw_path: + return [] + + search_dir, name_prefix = _search_parts(raw_path, workspace) + try: + entries = list(search_dir.iterdir()) + except OSError: + return [] + + matches: list[PathCompletionMatch] = [] + for entry in sorted( + entries, + key=lambda path: (not path.is_dir(), path.name.lower(), path.name), + ): + if not entry.name.startswith(name_prefix): + continue + if entry.name.startswith(".") and not name_prefix.startswith("."): + continue + suffix = "/" if entry.is_dir() else "" + completed_label = _replace_leaf(_unescape_path(raw_path), entry.name + suffix) + completed_fill = _replace_leaf(raw_path, _escape_path(entry.name) + suffix) + matches.append( + PathCompletionMatch( + label=completed_label, + fill=command_prefix + completed_fill, + ) + ) + if len(matches) >= limit: + break + return matches + + +def _path_argument(text: str) -> tuple[str, str] | None: + lowered = text.lower() + for prefix in _PATH_COMMAND_PREFIXES: + if lowered.startswith(prefix): + return text[: len(prefix)], text[len(prefix) :] + return None + + +def _search_parts(raw_path: str, workspace: Path) -> tuple[Path, str]: + lookup_path = _unescape_path(raw_path) + if lookup_path == "" or lookup_path.endswith("/"): + return _expand_path(lookup_path, workspace), "" + path = _expand_path(lookup_path, workspace) + return path.parent, path.name + + +def _expand_path(raw_path: str, workspace: Path) -> Path: + if raw_path.startswith("~"): + try: + return Path(raw_path).expanduser() + except RuntimeError: + # Home cannot be resolved (no $HOME, no pwd entry) — this runs per + # keystroke, so degrade to no matches instead of crashing the app. + return workspace / raw_path + path = Path(raw_path) + if path.is_absolute(): + return path + return workspace / path + + +def _replace_leaf(raw_path: str, leaf: str) -> str: + if raw_path == "" or raw_path.endswith("/"): + return raw_path + leaf + parent, separator, _name = raw_path.rpartition("/") + if separator: + return parent + separator + leaf + return leaf + + +def _unescape_path(raw_path: str) -> str: + if raw_path == "": + return raw_path + try: + parts = shlex.split(raw_path) + except ValueError: + return raw_path + return parts[0] if len(parts) == 1 else raw_path + + +def _escape_path(raw_path: str) -> str: + return raw_path.replace("\\", "\\\\").replace(" ", "\\ ") diff --git a/shellpilot/cli/render.py b/shellpilot/cli/render.py index 93a6d71..f07e88f 100644 --- a/shellpilot/cli/render.py +++ b/shellpilot/cli/render.py @@ -8,17 +8,20 @@ from __future__ import annotations import re +from collections.abc import Callable from difflib import SequenceMatcher from pathlib import Path from rich import box from rich.cells import cell_len -from rich.console import Group +from rich.console import Group, RenderableType +from rich.markdown import Markdown from rich.panel import Panel from rich.text import Text from shellpilot.cli.theme import Glyphs from shellpilot.policy.approvals import ApprovalRequest +from shellpilot.policy.risk import RiskLevel from shellpilot.runtime.planner import PlanStep, TaskPlan WORD_HIGHLIGHT_RATIO = 0.5 @@ -61,7 +64,25 @@ def context_line( return Text(path_str + suffix, style="sp.dim") +# Syntax theme for fenced code in model responses (§31.7): ANSI colors on the +# terminal's own background — never a painted fill like monokai's. +MARKDOWN_CODE_THEME = "ansi_dark" + + +def response_markdown(text: str) -> Markdown: + """Model-response markdown (§31.7): sanitized, code on the terminal background. + + The one builder for every response sink (the app pane's committed and + in-progress responses, the legacy REPL's live stream and final render), so + code rendering can't drift between them. + """ + return Markdown(_sanitize_line(text), code_theme=MARKDOWN_CODE_THEME) + + def tool_call(name: str, args_summary: str, glyphs: Glyphs) -> Text: + # Generic fallback line for tools with no clean primary subject (§31.3): name + # bright, the key=value summary dim so routine plumbing doesn't shout. Tools + # with a subject route through tool_call_block (framed or inline) instead. return Text.assemble( (f"{glyphs.bullet} ", ""), (_sanitize_line(name), "sp.emph"), @@ -69,6 +90,106 @@ def tool_call(name: str, args_summary: str, glyphs: Glyphs) -> Text: ) +# Built-in tools whose primary action deserves a framed, high-contrast subject +# (§31.3): the command that runs, the URL that egresses. Their tool-call line is +# `⏺