Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2847,7 +2847,25 @@ The full-screen pane shows a **live frontier line** while a turn runs, so a long

**Reasoning gate.** The reasoning/total readout is gated on `settings.ui.show_reasoning_summary` (default true; previously a dead flag — this is its consumer), threaded into `AppUI(show_reasoning=…)` at both construction sites (`build_app`, and `terminal.py` app mode passing the setting). When false, the live line is `{plane} {phrase}… {N}s` and the done line is `✓ done · {N}s` (plane/phrase/timer only). This is display-only; audit capture of thinking is unaffected.

**Animation and caching.** `build_app` sets the `Application`'s `refresh_interval=0.1` so the timer ticks and the plane glides even between thinking chunks. While an indicator is active, `_render_ansi` bypasses the width cache entirely (elapsed changes every render, so it never reads or writes the cache); when idle, the per-tick render is a width-cache hit (a cheap no-op). `AppUI` stays pure state+render — the clock is injected and the repaint is the app's job, so it never calls `get_app()`/`invalidate()`. A turn that errors before `turn_finished` (or, later, a Ctrl-C cancel) leaves the indicator active; the full turn-failure/cancel robustness is a later branch. As a minimal cross-turn guard, `show_user_message` discards any such dangling indicator at the start of the next turn, so an errored turn never poisons the following turn's timer (without it, the next `begin_response` would be a no-op against the stale indicator and count from the old start).
**Animation and caching.** `build_app` sets the `Application`'s `refresh_interval=0.1` so the timer ticks and the plane glides even between thinking chunks. While an indicator is active, `_render_ansi` bypasses the width cache entirely (elapsed changes every render, so it never reads or writes the cache); when idle, the per-tick render is a width-cache hit (a cheap no-op). `AppUI` stays pure state+render — the clock is injected and the repaint is the app's job, so it never calls `get_app()`/`invalidate()`. A turn that errors before `turn_finished` (or a Ctrl-C cancel, §31.15) leaves the indicator active; as a minimal cross-turn guard, `show_user_message` discards any such dangling indicator at the start of the next turn, so an errored turn never poisons the following turn's timer (without it, the next `begin_response` would be a no-op against the stale indicator and count from the old start).

### 31.15 Model-stream cancellation (v2)

Ctrl-C aborts a model turn mid-stream — the motivating case is a long invisible thinking phase (§31.14) the user wants to stop. The cancellation contract is deterministic and history-safe.

**Ctrl-C is a key press, not a SIGINT.** The full-screen app runs in raw mode (ISIG disabled), so `c-c` is an ordinary key binding. `build_app` takes `on_interrupt: Callable[[], bool] | None`; the `c-c` handler is `if on_interrupt is not None and on_interrupt(): return` (a turn was cancelled) `else: show_status(_IDLE_HINT)` (idle). `run_app` wires `on_interrupt=runner.request_cancel`.

**Cancel event, set on the loop thread, observed on the worker.** `TurnRunner.start` creates a fresh `threading.Event` per turn and passes it into `run_turn(text, cancel=…)`. `request_cancel()` (loop thread) returns True and `Event.set()`s it when `_busy` and a cancel exists, else False — the only cross-thread signal (the app only ever *sets* the event; thread-safe). `ConversationRuntime.run_turn` reassigns `self._cancel = cancel` at the top (so a stale event can never leak across turns) and the tool loop passes it to every `self._llm.chat(...)`.

**In-thread stream close + typed exception.** `OllamaClient._stream_chat` checks the event at the top of the `for line in response.iter_lines():` loop and raises `GenerationCancelled` (defined in `llm/client.py`, deliberately NOT an `OllamaError` subclass). Raising INSIDE the `with self._client.stream(...)` block closes the response in-thread via the context manager — the safe close; `response.close()` is never called cross-thread. Because `GenerationCancelled` is neither `httpx.TransportError` nor `OllamaResponseError`, it propagates cleanly out through `chat()` (and through the think-retry, which threads `cancel` into the retried call) to the runtime.

**History integrity (load-bearing).** `run_turn` records the user message *before* the tool loop, then lets `GenerationCancelled` PROPAGATE — it never catches it, records a partial reply, or calls `turn_finished`. The model call raises at the `reply = self._llm.chat(...)` site, so the subsequent `self._record(reply)` is skipped: a cancelled turn leaves the user message in history but NO partial assistant reply. The partial is discarded, not truncated-and-stored.

**Worker: clean abort vs failure.** `TurnRunner._run` catches `GenerationCancelled` SEPARATELY from `Exception`: cancel → `schedule(inner_ui.abort_turn)` (a clean abort, not a "Turn failed" error); any other exception → the `show_error` path. The `finally` schedules `_mark_done` on EVERY path, so a cancelled turn clears `_busy` and the next submit works — the app never wedges.

**`AppUI.abort_turn` (app-side, not a RuntimeUI method).** Clears the dangling live indicator (`self._indicator = None`) then appends a `⏹ aborted` marker (ASCII fallback `Glyphs.cross`, style `sp.warn`). `_add_renderable` closes the open response first, so the partial streamed text stays visible — finalized — with the marker below it. This is display-only; the history discard is the runtime's.

**Out of scope (branch 6b — subprocess killpg).** A cancel requested while a tool *command* is running does NOT kill that subprocess; the command finishes/times out, then the next model call's stream-read sees the set event and aborts the turn. Killing the running subprocess on cancel is deferred.

## 32. Model Selection And Preload

Expand Down
12 changes: 10 additions & 2 deletions shellpilot/cli/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,9 @@
# (branch 4) shows whether it bites.
DOCK_MAX_ROWS = 10

# Idle Ctrl-C hint. NOTE: there is no turn to cancel in this inert shell; branch
# 6 replaces this with real turn cancellation / subprocess kill.
# Idle Ctrl-C hint, shown only when no turn is in flight. Branch 6 (§31.15) wired
# real turn cancellation through on_interrupt; this hint is the idle fallback.
# NOTE: subprocess-kill on cancel is branch 6b.
_IDLE_HINT = "(idle — type /exit to quit)"


Expand Down Expand Up @@ -149,6 +150,7 @@ def build_app(
output: Output | None = None,
ui: AppUI | None = None,
on_submit: Callable[[str], None] | None = None,
on_interrupt: Callable[[], bool] | None = None,
) -> Application[None]:
"""Build the full-screen app shell.

Expand Down Expand Up @@ -324,6 +326,12 @@ def _page_down(event: KeyPressEvent) -> None:

@kb.add("c-c")
def _interrupt(event: KeyPressEvent) -> None:
# Raw mode disables ISIG, so Ctrl-C is a normal key press, not a SIGINT
# (branch 6, §31.15). When a turn is in flight, on_interrupt cancels it and
# returns True (the worker aborts the stream and renders the marker), so we
# show nothing. Otherwise it is idle — fall back to the idle hint.
if on_interrupt is not None and on_interrupt():
return
_ui.show_status(_IDLE_HINT)

# NOTE: keyboard PageUp/PageDown drive the pane via the cursor-line model
Expand Down
1 change: 1 addition & 0 deletions shellpilot/cli/app_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ def run_app(
ctx_pct=ctx_pct,
ui=app_ui,
on_submit=runner.start,
on_interrupt=runner.request_cancel,
)
runner.app = app
runner.conversation = runtime
Expand Down
81 changes: 67 additions & 14 deletions shellpilot/cli/app_turn.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,12 @@
from collections.abc import Callable
from typing import TYPE_CHECKING

from shellpilot.llm.client import GenerationCancelled

if TYPE_CHECKING:
from prompt_toolkit.application import Application

from shellpilot.cli.app_ui import AppUI
from shellpilot.policy.approvals import ApprovalReply, ApprovalRequest
from shellpilot.runtime.conversation import ConversationRuntime
from shellpilot.runtime.events import RuntimeUI, TurnStats
Expand Down Expand Up @@ -138,13 +141,21 @@ class TurnRunner:
needs the conversation — is broken by deferred attribute assignment.
"""

def __init__(self, *, inner_ui: RuntimeUI, schedule: Schedule | None = None) -> None:
def __init__(self, *, inner_ui: AppUI, schedule: Schedule | None = None) -> None:
# The raw content UI (NOT the marshaling ThreadedUI): the worker calls
# show_error / abort_turn on it via schedule. Typed AppUI because the
# cancel path needs abort_turn, which is an app-side AppUI method (like
# show_user_message) and deliberately not on the RuntimeUI protocol.
self._inner_ui = inner_ui
# busy: only ever touched on the loop thread (see class docstring).
self._busy = False
# The worker handle, kept so a test (and, later, branch-6 cancellation)
# can join it. Spawned fresh per turn.
# The worker handle, kept so a test (and branch-6 cancellation) can join
# it. Spawned fresh per turn.
self._thread: threading.Thread | None = None
# Branch-6 per-turn cancel event (§31.15): created fresh in start(),
# signaled by request_cancel() on the loop thread, observed by the
# worker's model call. None when no turn is in flight.
self._cancel: threading.Event | None = None
# Set after construction, before app.run(), to break the build cycle;
# read only during/after run().
self.app: Application[None] | None = None
Expand Down Expand Up @@ -185,25 +196,61 @@ def start(self, text: str) -> None:
Ignores the call when a turn is already in flight — single model, single
conversation, single worker; no parallel turns. NOTE: branch 9 adds the
one-message queue + Up-arrow recall; until then a submit-while-busy is
dropped. NOTE: branch 6 adds real Ctrl-C turn cancellation; there is no
cancel path here.
dropped.

A fresh cancel ``Event`` is created here (before the worker spawns, so the
worker sees it) and passed into ``run_turn``; ``request_cancel`` signals
it from the loop thread to abort the turn (branch 6, §31.15).
"""
if self._busy:
return
self._busy = True
self._thread = threading.Thread(target=self._run, args=(text,), daemon=True)
cancel = threading.Event()
self._cancel = cancel
# Pass the event as a thread ARG (a captured local), not via self._cancel,
# so the worker never reads the mutable instance attribute. `request_cancel`
# sets the SAME object from the loop thread, so the worker's local sees it —
# but the threading invariant ("the worker only touches its args + schedule;
# self._cancel/_busy are loop-thread-only") then holds literally, without
# relying on the _busy guard to keep self._cancel from being reassigned.
self._thread = threading.Thread(target=self._run, args=(text, cancel), daemon=True)
self._thread.start()

def _run(self, text: str) -> None:
def request_cancel(self) -> bool:
"""Signal the in-flight turn to abort. Runs on the loop thread (Ctrl-C).

Returns True when a turn was in flight (the cancel event is now set, and
the worker's next model-call read boundary will raise
``GenerationCancelled``); False when idle (nothing to cancel — the caller
then shows the idle hint). ``_busy``/``_cancel`` are read on the loop
thread; ``Event.set()`` is the one thread-safe cross-thread signal to the
worker (raw mode disables ISIG, so Ctrl-C is a key press, not a SIGINT).
"""
if self._busy and self._cancel is not None:
self._cancel.set()
return True
return False

def _run(self, text: str, cancel: threading.Event) -> None:
"""Worker-thread body: run ONE synchronous turn off the loop thread.

Touches nothing on the loop thread directly — only ``conversation.run_turn``
``cancel`` is the per-turn event, received as an ARG (a captured local) so
the worker never reads ``self._cancel`` — keeping the threading invariant
literal. Touches nothing on the loop thread directly — only ``run_turn``
(its UI is the :class:`ThreadedUI`, so every UI call is marshaled) and
``schedule``. A ``run_turn`` exception (e.g. an approval-needing turn
raises ``NotImplementedError`` until branch 7, or an ``OllamaError``
surfaces) is rendered as a pane error instead of silently killing the
daemon thread; the ``finally`` clears ``busy`` on the loop thread so a
crashed turn never wedges the app.
``schedule``. A ``GenerationCancelled`` (the user hit Ctrl-C mid-stream)
is a CLEAN abort, routed to ``abort_turn`` — never the error path. Any
OTHER ``run_turn`` exception (an approval-needing turn raises
``NotImplementedError`` until branch 7, or an ``OllamaError`` surfaces) is
rendered as a pane error instead of silently killing the daemon thread.
The ``finally`` clears ``busy`` on the loop thread on EVERY path
(success/cancel/error) so a turn never wedges the app.

NOTE (branch 6b — subprocess killpg): a cancel requested while a tool
COMMAND is running does NOT kill that subprocess here; the command
finishes/times out, then the NEXT model call's stream-read sees the set
event and raises GenerationCancelled, aborting the turn. Killing the
running subprocess is branch 6b.
"""
try:
conversation = self.conversation
Expand All @@ -212,7 +259,13 @@ def _run(self, text: str) -> None:
# Raise INSIDE the try so it surfaces as a pane error and the
# finally still clears busy — never a silent daemon death + wedge.
raise RuntimeError("TurnRunner.conversation not set before start()")
conversation.run_turn(text)
conversation.run_turn(text, cancel=cancel)
except GenerationCancelled:
# Clean Ctrl-C abort, NOT a failure: clear the dangling indicator and
# mark the partial aborted. The partial assistant reply was never
# recorded in history (run_turn let the exception propagate before the
# record site), so this is a display-only finalization.
self._schedule(self._inner_ui.abort_turn)
except Exception as exc: # noqa: BLE001 - surface ANY worker failure to the pane
self._schedule(functools.partial(self._inner_ui.show_error, f"Turn failed: {exc}"))
finally:
Expand Down
16 changes: 16 additions & 0 deletions shellpilot/cli/app_ui.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,22 @@ def turn_finished(self, stats: TurnStats) -> None:
line.append(f" · {_fmt_count(stats.output_tokens)} total", style="sp.dim")
self._add_renderable(line)

def abort_turn(self) -> None:
# Branch 6 (§31.15): a turn was cancelled mid-stream (Ctrl-C). App-side
# (NOT a RuntimeUI protocol method) — TurnRunner calls it on the raw AppUI.
# Clear the dangling live indicator (stop the timer/plane) FIRST, then
# append the aborted marker. _add_renderable closes the open response
# first, so any partial streamed text stays visible — finalized — with
# the marker line below it. The partial assistant reply is never recorded
# in history; that discard is the runtime's (run_turn lets
# GenerationCancelled propagate before the record site). This is purely
# display: mark what the user already saw as stopped.
self._indicator = None
# `⏹` is not in the Glyphs set; gate it on the same unicode/ascii signal
# build_app uses, falling back to the ASCII cross glyph.
marker = "⏹" if self._glyphs == UNICODE_GLYPHS else self._glyphs.cross
self._add_renderable(Text(f"{marker} aborted", style="sp.warn"))

def stream_thinking(self, text: str) -> None:
# The reasoning count climbs ONLY while the model is thinking, so it
# freezes naturally when thinking stops and resumes if it thinks again. No
Expand Down
22 changes: 20 additions & 2 deletions shellpilot/llm/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,32 @@

from __future__ import annotations

import threading
from collections.abc import Callable, Sequence
from typing import Any, Protocol
from typing import TYPE_CHECKING, Any, Protocol

from shellpilot.llm.messages import Message, ToolDefinition
from shellpilot.llm.ollama import LocalModel

if TYPE_CHECKING:
# Only an annotation (list_models return type); a TYPE_CHECKING import keeps
# the protocol module free of a runtime dependency on the concrete Ollama
# client, so ollama.py can import GenerationCancelled from here without an
# import cycle.
from shellpilot.llm.ollama import LocalModel

TokenCallback = Callable[[str], None]


class GenerationCancelled(Exception):
"""A model turn was aborted mid-stream via the cancel event (branch 6, §31.15).

Deliberately NOT an ``OllamaError`` subclass: the think-retry in
``OllamaClient.chat`` and the worker in ``TurnRunner`` must distinguish a
user cancel from a provider failure, so it propagates past both as a clean,
typed signal rather than a generic error.
"""


class LLMClient(Protocol):
"""What the conversation runtime needs from a model provider."""

Expand All @@ -24,6 +41,7 @@ def chat(
options: dict[str, Any] | None = None,
on_token: TokenCallback | None = None,
on_thinking: TokenCallback | None = None,
cancel: threading.Event | None = None,
) -> Message: ...

def health(self) -> bool: ...
Expand Down
Loading