Terminal context, ,, repair, stdin piping, autostart, and per-tier model routing

README.md

@@ -54,7 +54,9 @@ From source: jump to [Install from source](#install-from-source).
 
 `,` and `?` are **conversational** — each terminal pane keeps its own thread. Type `, the same but with json output` and the model knows what "the same" means. After 30 min idle the thread auto-rotates so a forgotten tab doesn't bleed stale context.
 
-`?` has tools: read files (filesystem-gated), DuckDuckGo search, fetch URL. Force web-first with `? --web <q>`.
+`,,` (alias for `, --fix`) repairs whatever just failed: it sends the previous command, its exit status, and — at higher [terminal context](#terminal-context) levels — recent output, and proposes corrected commands through the same picker.
+
+`?` has tools: read files (filesystem-gated), DuckDuckGo search, fetch URL. Force web-first with `? --web <q>`. It's also pipe-friendly — `make 2>&1 | ? what broke` turns the piped output into context.
 
 `???` is the durable layer. Bare queries hit the archive of every past session (BM25; semantic if you've added embeddings). Flags pin long-term facts that get injected into every `?` system prompt.
 
@@ -85,6 +87,18 @@ huggingface-cli download unsloth/Qwen3-Coder-Next-GGUF
 
 `??` finds the GGUF inside your HuggingFace cache — no path config required.
 
+Each tier binds to its own port (`fast` :8091, `balanced` :8080, `smart` :8093), so tiers can serve **side by side** — and a single call can be routed to whichever fits:
+
+```sh
+?? --start fast                 # fast tier up, alongside balanced
+, --fast rename all .jpeg to .jpg    # this one call uses the fast tier
+? --smart why is my Makefile rebuilding everything   # this one, the smart tier
+?? --status                     # which tiers are up
+?? --stop fast                  # stop just that one
+```
+
+Don't want to babysit the server at all? `export SHELLLM_AUTOSTART=1` and `,` / `?` bring the default tier up on demand the first time you use them.
+
 ## Sessions
 
 Each pane × command gets a sticky JSONL session at `~/.cache/shellllm/sessions/`. Pane identity is `TERM_SESSION_ID` (Terminal.app / iTerm) → `TMUX_PANE` → `WINDOWID` → `$PPID`, first one that resolves.
@@ -102,6 +116,35 @@ When the conversation crosses 80% of `SHELLLM_CTX`, older turns are auto-summari
 
 Every expired or `--new`'d session flows into `~/.cache/shellllm/archive.db` (sqlite + FTS5) so `???` can search across panes and days.
 
+## Terminal context
+
+The terminal knows what just happened — shellllm uses it, at a level you control. One env var, a ladder of levels:
+
+```sh
+export SHELLLM_SHELL_CONTEXT=off        # capture nothing
+export SHELLLM_SHELL_CONTEXT=cmd        # previous command + exit status (default)
+export SHELLLM_SHELL_CONTEXT=history    # + last 10 commands
+export SHELLLM_SHELL_CONTEXT=output     # + recent pane output (tmux only)
+```
+
+References resolve themselves:
+
+```sh
+$ git push origin amin
+error: src refspec amin does not match any
+$ ,,                              # proposes: git push origin main
+$ ? why did that fail             # the model sees the command and its exit status
+```
+
+How it stays private:
+
+- **Local-first**: with a local model, nothing leaves the machine anyway. The ladder matters when you point `SHELLLM_BASE_URL` at a hosted API.
+- **Redaction**: captures pass through a secret scrubber (`KEY=…` assignments, `Bearer` headers, AWS/GitHub/Slack/Stripe-shaped tokens, JWTs) before the model sees them. Git SHAs survive — they're useful.
+- **Per-call env, never exported**: the zsh layer passes captures as one-shot environment for the single invocation; nothing lingers in your shell.
+- **Both sides enforce the ladder**: zsh won't capture above your level, and the Python side independently re-checks it.
+- **Ephemeral**: context blocks are rebuilt per turn and never persisted into sessions or the archive.
+- `--no-ctx` skips injection for one call; piped stdin (`cmd | ? …`) is its own explicit consent and works regardless of the ladder.
+
 ## Semantic recall (optional)
 
 Recall works in BM25-only mode out of the box. Adding a tiny embedding server upgrades it to **hybrid semantic + BM25** (fused with Reciprocal Rank Fusion):
@@ -146,21 +189,32 @@ Every file read through `?` goes through `safe_fs.safe_read`. Four rules, all en
 Reads cap at 1 MB and use `O_NOFOLLOW` as a belt against a resolve-then-open symlink race.
 
 ```sh
-pytest -v   # 197 tests; 38 dedicated to symlinks, traversal, denylist, lookalikes, truncation
+pytest -v   # 257 tests; 38 dedicated to symlinks, traversal, denylist, lookalikes, truncation
 ```
 
 ## Configuration
 
 | Variable | Default | Purpose |
 |---|---|---|
-| `SHELLLM_BASE_URL` | `http://127.0.0.1:8080` | llama-server endpoint |
-| `SHELLLM_PORT` | `8080` | Server port |
+| `SHELLLM_BASE_URL` | `http://127.0.0.1:8080` | llama-server (or hosted) endpoint |
+| `SHELLLM_API_KEY` | — | Bearer auth for chat — set when pointing at a hosted API |
+| `SHELLLM_MODEL` | `local` | Model id passed in chat requests (override per provider) |
+| `SHELLLM_PORT` | `8080` | Server port (default route + `balanced` tier) |
+| `SHELLLM_PORT_FAST` | `8091` | `fast` tier port |
+| `SHELLLM_PORT_BALANCED` | `$SHELLLM_PORT` | `balanced` tier port |
+| `SHELLLM_PORT_SMART` | `8093` | `smart` tier port |
+| `SHELLLM_SHELL_CONTEXT` | `cmd` (zsh layer) | `off` / `cmd` / `history` / `output` — terminal-context ladder |
+| `SHELLLM_AUTOSTART` | unset | `1` to auto-start the default tier when `,` / `?` find it down |
 | `SHELLLM_NGL` | `99` | GPU offload layers |
 | `SHELLLM_CTX` | `32768` | Context window (tokens) |
 | `SHELLLM_TIMEOUT` | `120` | HTTP timeout (seconds) |
 | `SHELLLM_LLAMA_MODEL` | — | Explicit GGUF path, overrides tier |
 | `SHELLLM_LOG` | `~/.cache/shellllm/llama-server.log` | Server log path |
-| `SHELLLM_EMBED_URL` | `http://127.0.0.1:8081` | Local embedding server endpoint |
+| `SHELLLM_RENDER_URL` | — | Firecrawl-compatible JS-render endpoint (enables) |
+| `SHELLLM_RENDER_API_KEY` | — | Bearer auth for the render service |
+| `SHELLLM_RENDER_TIMEOUT` | `30` | Render-service HTTP timeout (seconds) |
+| `SHELLLM_EMBED_URL` | `http://127.0.0.1:8081` | Local (or hosted) embedding endpoint |
+| `SHELLLM_EMBED_API_KEY` | — | Bearer auth for embeddings (falls back to `SHELLLM_API_KEY`) |
 | `SHELLLM_EMBED_PORT` | `8081` | Embedding server port |
 | `SHELLLM_EMBED_CTX` | `2048` | Embedding context window |
 | `SHELLLM_EMBED_MODEL` | `local-embed` | Model name passed to `/v1/embeddings` |
@@ -192,10 +246,53 @@ You don't need to ship every "what does git stash do" question to a frontier mod
 
 The bet: a local 27B model is roughly equivalent to a frontier model for the questions you ask between `git commit` and `make test`. The wins — privacy, latency, offline availability, $0 per question — are real, every day.
 
+## Use a hosted API instead of llama-server
+
+shellllm's chat and embedding paths are both OpenAI-compatible. Point them at any provider, BYOK:
+
+```sh
+# OpenAI
+export SHELLLM_BASE_URL="https://api.openai.com"
+export SHELLLM_API_KEY="sk-..."
+export SHELLLM_MODEL="gpt-4o-mini"
+
+# OpenRouter
+export SHELLLM_BASE_URL="https://openrouter.ai/api"
+export SHELLLM_API_KEY="sk-or-..."
+export SHELLLM_MODEL="anthropic/claude-3.5-sonnet"
+
+# Groq (very fast)
+export SHELLLM_BASE_URL="https://api.groq.com/openai"
+export SHELLLM_API_KEY="gsk-..."
+export SHELLLM_MODEL="llama-3.3-70b-versatile"
+```
+
+When `SHELLLM_API_KEY` is set, every chat request carries `Authorization: Bearer …`. Without it, requests stay anonymous (which is what the local `llama-server` wants). The same fall-through applies to embeddings: set `SHELLLM_EMBED_API_KEY` for a separate provider, or let it inherit `SHELLLM_API_KEY` when both endpoints share auth.
+
+Mix and match: local chat + hosted embeddings, hosted chat + local embeddings, or both hosted. The model decides; shellllm just plumbs.
+
+## JS rendering for `fetch_url`
+
+Static HTML works for most pages. SPAs (React/Vue/Svelte sites that paint after the initial response) come back empty. To handle them, point `fetch_url` at a Firecrawl-compatible scraper — hosted or self-hosted:
+
+```sh
+# Hosted: https://firecrawl.dev
+export SHELLLM_RENDER_URL="https://api.firecrawl.dev"
+export SHELLLM_RENDER_API_KEY="fc-..."
+
+# Self-hosted: https://github.com/mendableai/firecrawl
+export SHELLLM_RENDER_URL="http://localhost:3002"
+export SHELLLM_RENDER_API_KEY="any-string-if-disabled"
+```
+
+When configured, every `fetch_url` call tries `POST {url}/v1/scrape` first (Bearer auth, asks for markdown). On any failure — connection refused, 4xx/5xx, timeout — it falls back transparently to the static fetcher. Without the env vars, behavior is unchanged.
+
+Any service speaking the same `/v1/scrape` shape works. Other vendors can be bridged with a tiny proxy that translates between their API and Firecrawl's.
+
 ## What's deliberately not built
 
 - **GBNF prefix grammar for `,`.** JSON schema is enough for v1; the system prompt forbids the obvious destructive commands.
-- **JS rendering for `fetch_url`.** Pages are fetched as static HTML and reduced to text — SPAs that need JS to populate content will look empty.
+- **Local browser-based JS rendering.** Skipped on purpose — a 300MB Chromium dependency doesn't match the offline-by-default story. The BYOK `SHELLLM_RENDER_URL` path above is the supported alternative.
 
 ## Development
 
@@ -218,9 +315,11 @@ src/shellllm/
 ├── embed.py         client for a local llama-server in --embedding mode
 ├── compact.py       summary-buffer compaction over the same local model
 ├── claude_mem.py    optional adapter for claude-mem server-beta
+├── render.py        optional Firecrawl-compatible JS renderer for fetch_url
 ├── safe_fs.py       filesystem hard wall
-├── client.py        llama-server HTTP client (one-shot + streaming)
+├── client.py        OpenAI-compatible chat HTTP client (Bearer auth opt-in)
 ├── context.py       date/OS/timezone prelude
+├── shell_context.py opt-in terminal context: ladder, redaction, piped stdin
 └── web.py           stdlib DuckDuckGo scraper + fetch_url with SSRF guard
 ```
 

demo.tape

@@ -0,0 +1,146 @@
+# shellllm — flow demo
+#
+# Tells a 60-second story: you're working in a directory, you hit a
+# command you don't quite remember, you ask the model about it, you
+# refine, you move on, and later you recall a thing you asked earlier
+# — all without leaving your zsh prompt.
+#
+# Render
+#   brew install vhs                           # one-time, ~30s
+#   ?? --start balanced                        # llama-server must be running
+#   vhs demo.tape                              # writes demo.gif
+#
+# The Hide block at the top sets up a sandbox in /tmp/shellllm-demo with
+# a few sized files (so `du` has something to show) and seeds the
+# archive with one prior conversation about git stash (so the `???`
+# recall later finds a real, pre-existing hit). Everything else is the
+# tool acting in its native environment.
+
+Output demo.gif
+
+# ── Look ────────────────────────────────────────────────────────────────
+Require zsh
+Require llama-server
+Set Shell "zsh"
+Set FontSize 18
+Set FontFamily "JetBrainsMono Nerd Font"
+Set Theme "Catppuccin Mocha"
+Set Width 1200
+Set Height 720
+Set Padding 28
+Set BorderRadius 10
+Set WindowBar Colorful
+Set TypingSpeed 60ms
+Set PlaybackSpeed 1.0
+Set CursorBlink true
+
+# ── Setup (hidden) ──────────────────────────────────────────────────────
+# Fresh sandbox, isolated archive/memory paths so we don't touch the
+# user's real cache while recording.
+Hide
+Type "export SHELLLM_ARCHIVE_DB=/tmp/shellllm-demo/archive.db SHELLLM_MEMORY_FILE=/tmp/shellllm-demo/memory.jsonl SHELLLM_SESSIONS_DIR=/tmp/shellllm-demo/sessions"
+Enter
+Sleep 100ms
+Type "rm -rf /tmp/shellllm-demo && mkdir -p /tmp/shellllm-demo/sessions && cd /tmp/shellllm-demo"
+Enter
+Sleep 100ms
+# Some real-sized files so `du -ah | sort -hr | head -5` has output.
+Type "head -c 1500000 </dev/urandom > release-build.tar.gz"
+Enter
+Type "head -c 920000 </dev/urandom > node_modules.cache"
+Enter
+Type "head -c 410000 </dev/urandom > coverage-report.html"
+Enter
+Type "head -c 180000 </dev/urandom > webpack-stats.json"
+Enter
+Type "touch -d '2 hours ago' release-build.tar.gz coverage-report.html"
+Enter
+# Seed one prior shellllm session into the archive so `??? git` finds
+# something later in the demo. We use a heredoc to keep the quoting
+# manageable inside the Type command.
+Type@10ms "python3 - <<'PY'"
+Enter
+Type@10ms "from shellllm.archive import Archive"
+Enter
+Type@10ms "Archive().ingest_session(cmd='ask', terminal_id='demo', created_at=0, last_used=0, last_pwd='~/code/api', last_date='2026-06-08', turn_count=1, messages=[{'role':'user','content':'how does git stash work'},{'role':'assistant','content':'git stash temporarily shelves uncommitted changes so you can switch contexts, then `git stash pop` re-applies them on top of the current working tree.'}])"
+Enter
+Type@10ms "PY"
+Enter
+Sleep 400ms
+Type "clear"
+Enter
+Sleep 200ms
+Show
+
+# ── Beat 1 — `,` proposes a command ─────────────────────────────────────
+# You're in a directory full of build artifacts. You vaguely remember
+# `du` exists. Instead of googling, you ask the model in-band.
+Type ", find the five largest files in this directory"
+Sleep 600ms
+Enter
+# Wait for the model + fzf to render the candidate picker.
+Sleep 6s
+# Accept the highlighted proposal — `print -z` drops it on the prompt.
+Enter
+Sleep 800ms
+# Run it.
+Enter
+Sleep 3s
+
+# ── Beat 2 — refine via the sticky session ──────────────────────────────
+# Same `,` thread in this pane: the model remembers the prior proposal.
+# "the same but only ones modified today" works because it has
+# context, exactly like a real conversation.
+Type ", the same but only ones modified in the last hour"
+Sleep 600ms
+Enter
+Sleep 6s
+Enter
+Sleep 800ms
+Enter
+Sleep 3s
+
+# ── Beat 3 — `?` answers without breaking the flow ──────────────────────
+# Two files made the cut. Now you need to commit, but `git stash` came
+# to mind and you blanked. Ask the model in-band, get an answer.
+Type "? in markdown and 3 lines max, what does git stash do"
+Sleep 600ms
+Enter
+# Streaming markdown render — generous sleep because turn length varies
+# with the local model.
+Sleep 14s
+
+# ── Beat 4 — pin a long-term fact ───────────────────────────────────────
+# While you're here, pin something. Now every `?` carries it.
+Type "??? --add this project uses pnpm and ships via release-build.tar.gz"
+Sleep 500ms
+Enter
+Sleep 2s
+
+# ── Beat 5 — start a fresh `?` thread so the prior turn lands in the archive
+# `--new` archives the current session — perfect mid-flow segue.
+Type "? --new"
+Sleep 500ms
+Enter
+Sleep 2s
+Type "clear"
+Enter
+Sleep 400ms
+
+# ── Beat 6 — `???` recalls across past sessions ─────────────────────────
+# "What was that git thing I asked earlier?" Bare query → FTS5 search
+# across every archived conversation in this pane and others.
+Type "??? git stash"
+Sleep 500ms
+Enter
+Sleep 5s
+
+# ── Beat 7 — `??? --archives` shows what's in the vault ─────────────────
+Type "??? --archives 5"
+Sleep 500ms
+Enter
+Sleep 5s
+
+# ── Outro ───────────────────────────────────────────────────────────────
+# Let the final frame breathe so a viewer can read the last row.
+Sleep 3s

src/shellllm/archive.py

@@ -33,7 +33,8 @@
 import re
 import sqlite3
 import time
-from collections.abc import Callable, Iterable
+from collections.abc import Callable, Iterable, Iterator
+from contextlib import contextmanager
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any
@@ -396,11 +397,23 @@ def _search_cosine(
 
     # ── Internals ------------------------------------------------------
 
-    def _conn(self) -> sqlite3.Connection:
+    @contextmanager
+    def _conn(self) -> Iterator[sqlite3.Connection]:
+        """Yield a connection inside a transaction, then *close* it.
+
+        ``with sqlite3.connect(...)`` alone only scopes the transaction —
+        the underlying connection (3 fds in WAL mode: db, -wal, -shm)
+        stays open until GC. Under the test suite that exhausted the
+        default macOS fd limit, so we close deterministically.
+        """
         conn = sqlite3.connect(self.path)
-        conn.execute("PRAGMA journal_mode=WAL")
-        conn.execute("PRAGMA synchronous=NORMAL")
-        return conn
+        try:
+            conn.execute("PRAGMA journal_mode=WAL")
+            conn.execute("PRAGMA synchronous=NORMAL")
+            with conn:
+                yield conn
+        finally:
+            conn.close()
 
 
 # ── Helpers --------------------------------------------------------------