Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Everything beyond the 3-step install. Pick what you need.
- [The hard wall](#the-hard-wall)
- [Environment variables](#environment-variables)
- [Install from source](#install-from-source)
- [Using shellllm without zsh](#using-shellllm-without-zsh)
- [Use a hosted API instead of llama-server](#use-a-hosted-api-instead-of-llama-server)
- [JS rendering for `fetch_url`](#js-rendering-for-fetch_url)
- [What's deliberately not built](#whats-deliberately-not-built)
Expand Down Expand Up @@ -185,6 +186,58 @@ exec zsh

After a `git pull` you only need `exec zsh` to pick up updates to `zsh/shellllm.zsh`. Python entry-points reload automatically (editable install).

## Using shellllm without zsh

The zsh integration ships the punctuation glyphs (`,` `,,` `?` `??` `???`), which require zsh-specific features like `noglob` and aliasing characters that bash treats as wildcards. The Python CLIs underneath work in any POSIX shell.

### Option 1: bash adapter (recommended for bash users)

A small bash file exposes the same commands under short alphabetic names. Source it from `~/.bashrc`:

```sh
source "$(brew --prefix)/share/shellllm/shellllm.bash"
```

Then:

| Alias | Maps to | Example |
|---|---|---|
| `llmc` | propose commands (`,`) | `llmc find the five largest files here` |
| `llmcc` | propose with terminal context | `llmcc verify the file we just produced` |
| `llmf` | fix the previous command (`,, ` bare) | `llmf` |
| `llma` | ask (`?`) | `llma what does git stash do` |
| `llmm` | memory / recall (`???`) | `llmm docker volumes` |

For reliable previous-command capture in bash, install [bash-preexec](https://github.com/rcaloras/bash-preexec) first — without it, `llmcc` and `llmf` only see the exit status, not the command text.

### Option 2: call the CLIs directly

The three Python entry points work in any shell, no adapter required:

```sh
shellllm-comma "find the five largest files here"
shellllm-ask "what does git stash do"
shellllm-recall "docker volumes"
shellllm-recall --add "the project uses pnpm"
```

Add your own short aliases or shell functions as you prefer.

### Server control

Without the zsh layer, `??` is unavailable — start `llama-server` directly:

```sh
llama-server -m ~/.cache/huggingface/hub/.../Qwen3.6-27B-Q4_K_M.gguf \
-c 32768 -ngl 99 --host 127.0.0.1 --port 8080 &
```

Or set `SHELLLM_AUTOSTART=1` and let any CLI invoke a helper. The `shellllm-comma` and `shellllm-ask` binaries respect `SHELLLM_BASE_URL`, so per-call tier routing still works by switching the variable:

```sh
SHELLLM_BASE_URL=http://127.0.0.1:8091 shellllm-comma "explain this Makefile"
```

## Use a hosted API instead of llama-server

shellllm's chat and embedding paths are both OpenAI-compatible. Point them at any provider, BYOK:
Expand Down
2 changes: 2 additions & 0 deletions Formula/shellllm.rb
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def install

# Ship the zsh integration file at a stable, well-known location.
pkgshare.install "zsh/shellllm.zsh"
pkgshare.install "bash/shellllm.bash"
end

def caveats
Expand Down Expand Up @@ -121,5 +122,6 @@ def caveats

# Confirm the zsh integration shipped where caveats say it does.
assert_predicate pkgshare/"shellllm.zsh", :exist?
assert_predicate pkgshare/"shellllm.bash", :exist?
end
end
60 changes: 35 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,67 +4,77 @@
[![python](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/)
[![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

> Local LLM at your zsh prompt. Five glyphs, no API key, works offline.
> A local LLM at your shell prompt. Five glyphs, no account, works offline.

Drop English at your prompt and get a real shell command. Typo something and fix it with two keystrokes. Ask the model a question without breaking flow. Search every past conversation by content. The whole CLI is punctuation — `,` `,,` `?` `??` `???` — because the best terminal UI is the one that fits next to `cd` and `ls`.
`shellllm` puts a local language model behind a handful of punctuation commands in your shell. Describe what you want in English and get a real shell command back. Mistype something and recover with two keystrokes. Ask a question without leaving the terminal. Search every conversation you've ever had with the tool.

And it knows what just happened in your terminal: the previous command and its exit status ride along (redacted, local, [level-controlled](CONFIGURATION.md#terminal-context)), so "that" and "why did it fail" mean what you think they mean.
Everything runs against a local [`llama.cpp`](https://github.com/ggerganov/llama.cpp) server — no API keys, no data leaves your machine, no cost per question, available on a plane.

![shellllm demo](demo.gif)

No API key. No data leaves your machine. Works with WiFi off (except `? --web`).

## Install

Three steps. Local model, no account.
Three steps. macOS or Linux, local model, no account required.

```sh
# 1. Tool (pulls llama.cpp, fzf, and the CLIs)
brew install FrancoisChastel/shellllm/shellllm
echo 'source "$(brew --prefix)/share/shellllm/shellllm.zsh"' >> ~/.zshrc
exec zsh

# 2. Model (one-time; `pipx install huggingface_hub` if you lack the CLI)
# 2. Model (one-time download, ~16 GB)
huggingface-cli download unsloth/Qwen3.6-27B-GGUF
# (need huggingface-cli? `pipx install huggingface_hub`)

# 3. Go
?? # start the server (~10s once cached)
# 3. Run
?? # start the local server (~10s once cached)
, find the five largest files here
```

Zero babysitting: `export SHELLLM_AUTOSTART=1` and the first `,` or `?` starts the server for you.
Don't want to manage the server yourself? `export SHELLLM_AUTOSTART=1` and the first `,` or `?` starts it on demand.

Not on zsh? The Python CLIs (`shellllm-comma`, `shellllm-ask`, `shellllm-recall`) work in bash, fish, or any POSIX shell. See [CONFIGURATION.md#using-shellllm-without-zsh](CONFIGURATION.md#using-shellllm-without-zsh) for a minimal bash adapter.

## The five commands

| Cmd | What | Example |
| Command | What it does | Example |
|---|---|---|
| `, <english>` | Propose shell commands, pick one in fzf, drop on prompt. Never executes. | `, the five largest files here` |
| `,, [english]` | Same, but **with terminal context**. Bare `,,` = fix the previous command. | `,,` after a typo'd push |
| `? <q>` | Ask the model. Streams markdown. Sticky per-pane session. Pipe-friendly. | `? what does git stash do` |
| `???` | Memory & recall. Bare query searches the archive. Flags pin long-term facts. | `??? --add I prefer ripgrep` |
| `??` | Start / stop / status the local `llama-server`. | `?? --start fast` |
| `, <prompt>` | Propose 3–5 shell commands, pick one in fzf, drop on prompt. Never executes. | `, the five largest files here` |
| `,, [prompt]` | Same, but with terminal context attached. Bare `,,` repairs the previous command. | `,,` after a typo'd command |
| `? <question>` | Ask the model. Streams markdown, keeps a per-pane conversation. Reads piped input. | `? what does git stash do` |
| `???` | Memory and recall. Bare query searches the archive; flags pin long-term facts. | `??? --add I prefer ripgrep` |
| `??` | Start, stop, or check the local `llama-server`. | `?? --start fast` |

A few moves worth knowing:

```sh
make 2>&1 | ? what broke # pipe an error in, get a diagnosis
?? --start fast # multiple tiers run side by side
, --smart explain this Makefile # route one call to a specific tier
make 2>&1 | ? what broke # pipe an error, get a diagnosis
?? --start fast # two tiers can serve side by side
, --smart explain this Makefile # route one call to the bigger model
??? --add the project uses pnpm # pin a fact; every `?` carries it
??? docker volumes # bare query = recall across past sessions
??? docker volumes # bare query → search past sessions
```

That's the whole tour. **For everything else — model tiers, hosted-API setup, terminal-context ladder details, semantic recall, JS rendering for SPAs, the full env-var table, and the filesystem hard wall — see [CONFIGURATION.md](CONFIGURATION.md).**
That's the whole surface. **For model tiers, hosted-API setup, the terminal-context ladder, semantic recall, JS rendering, the full environment-variable table, and the filesystem hard wall, see [CONFIGURATION.md](CONFIGURATION.md).**

## Design

A few decisions are load-bearing:

- **`,` never executes.** The comma proposes commands and drops the chosen one on your prompt line. You confirm with Enter. The model never runs anything on its own.
- **The filesystem has a hard wall.** `?` can read files, but only inside `$HOME` or `$PWD`, never inside `.ssh`, `.aws`, `.gnupg`, or any other secret-bearing path. Symlinks are canonicalised before containment is checked. Reads cap at 1 MB.
- **Terminal context is a ladder.** What `shellllm` sees from your terminal — the previous command, its exit status, recent history, recent output — is gated by `SHELLLM_SHELL_CONTEXT`, redacted for secrets, rebuilt per call, and never persisted.
- **Sessions are sticky per pane, ephemeral per session.** Each terminal pane has its own conversation thread; idle sessions roll into a searchable archive automatically.

## Why this exists
## Why local

You don't need to ship every "what does git stash do" question to a frontier model. The wifi will be off on the plane and you'll still want a hand. Every `tar -czvf` answer has been in your model's training data for two years. Claude Code is great but it lives in its own window — `cd ~/project && ?` shouldn't require a browser tab.
You don't need a frontier model to remember `tar -czvf`. The questions you ask between `git commit` and `make test` — flag lookups, "what does this command do", "fix my typo" — are well within reach of a local 27B model. In exchange you get privacy, sub-second latency, no per-question cost, and a prompt that works on a flight.

The bet: a local 27B model is roughly equivalent to a frontier model for the questions you ask between `git commit` and `make test`. The wins — privacy, latency, offline availability, $0 per question — are real, every day.
For the harder questions, `shellllm` doesn't force a choice: point `SHELLLM_BASE_URL` at OpenAI, OpenRouter, Groq, or any OpenAI-compatible endpoint, and the same five glyphs route through a hosted model instead. Mix and match — local chat with hosted embeddings, hosted chat with local recall, whatever fits.

## For contributors and AI agents

Coding conventions, the load-bearing invariants (the **hard wall**, the **comma never executes**, the **terminal-context ladder**), test rules, and where to look: see [AGENTS.md](AGENTS.md).
Conventions, the load-bearing invariants in detail, test rules, and where to look first: see [AGENTS.md](AGENTS.md).

## License

Expand Down
48 changes: 48 additions & 0 deletions bash/shellllm.bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# shellllm — bash adapter
#
# Bash doesn't allow `?`, `??`, `???` as command names (they're globs),
# so this adapter exposes the same Python CLIs under short alphabetic
# aliases. Source from ~/.bashrc:
#
# source /path/to/shellllm/bash/shellllm.bash
#
# Provides:
# llmc propose commands llmc find five largest files
# llmcc propose with context llmcc verify the file just produced
# llmf fix the previous command llmf
# llma ask llma what does git stash do
# llmm memory / recall llmm --add I prefer ripgrep
# llmm docker volumes
#
# Server control (??) and per-call tier routing (--fast / --smart) are
# zsh-layer features; in bash, start llama-server yourself or call the
# CLIs with SHELLLM_BASE_URL=http://127.0.0.1:8091 to route per call.

: "${SHELLLM_COMMA:=shellllm-comma}"
: "${SHELLLM_ASK:=shellllm-ask}"
: "${SHELLLM_RECALL:=shellllm-recall}"
: "${SHELLLM_SHELL_CONTEXT:=cmd}"
export SHELLLM_SHELL_CONTEXT

# bash-preexec (https://github.com/rcaloras/bash-preexec) is the
# standard way to get reliable "last command + last exit status" in
# bash. If it's loaded we use it; otherwise we fall back to $? at call
# time (still useful) and an empty SHELLLM_LAST_CMD.
_shellllm_preexec() { _SHELLLM_PREV_CMD="$1"; }
_shellllm_precmd() { _SHELLLM_PREV_STATUS=$?; }
if declare -f precmd_functions preexec_functions >/dev/null 2>&1; then
preexec_functions+=(_shellllm_preexec)
precmd_functions+=(_shellllm_precmd)
fi

_shellllm_with_ctx() {
SHELLLM_LAST_STATUS="${_SHELLLM_PREV_STATUS:-$1}" \
SHELLLM_LAST_CMD="${_SHELLLM_PREV_CMD:-}" \
"${@:2}"
}

llmc() { local s=$?; "$SHELLLM_COMMA" "$@"; }
llmcc() { local s=$?; _shellllm_with_ctx "$s" "$SHELLLM_COMMA" --ctx "$@"; }
llmf() { local s=$?; _shellllm_with_ctx "$s" "$SHELLLM_COMMA" --fix; }
llma() { local s=$?; _shellllm_with_ctx "$s" "$SHELLLM_ASK" "$@"; }
llmm() { "$SHELLLM_RECALL" "$@"; }
Binary file modified demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
40 changes: 24 additions & 16 deletions demo.tape
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Hide
Type "export PATH=$PWD/.venv/bin:$PATH && source ./zsh/shellllm.zsh"
Enter
Sleep 200ms
Type "export SHELLLM_ARCHIVE_DB=/tmp/shellllm-demo/archive.db SHELLLM_MEMORY_FILE=/tmp/shellllm-demo/memory.jsonl SHELLLM_SESSIONS_DIR=/tmp/shellllm-demo/sessions SHELLLM_SHELL_CONTEXT=cmd"
Type "export SHELLLM_ARCHIVE_DB=/tmp/shellllm-demo/archive.db SHELLLM_MEMORY_FILE=/tmp/shellllm-demo/memory.jsonl SHELLLM_SESSIONS_DIR=/tmp/shellllm-demo/sessions SHELLLM_SHELL_CONTEXT=cmd SHELLLM_BASE_URL=http://127.0.0.1:8091"
Enter
Sleep 100ms
Type "rm -rf /tmp/shellllm-demo && mkdir -p /tmp/shellllm-demo/sessions && cd /tmp/shellllm-demo"
Expand Down Expand Up @@ -97,11 +97,20 @@ Enter
Sleep 200ms
Show

# ── Beat 1 — `, --fast` proposes a command via the fast tier ────────────
# You're in a directory full of build artifacts. You vaguely remember
# `du` exists. Ask the model — routed to the fast tier (MoE 3B-active,
# ~3× faster than balanced).
Type ", --fast find the five largest files in this directory"
# ── Beat 0 — spin up the local model ────────────────────────────────────
# The audience needs to know the LLM lives on your machine. `?? --start
# fast` puts a llama-server on :8091 with the MoE 3B-active tier — the
# rest of the demo uses it by default (SHELLLM_BASE_URL points there).
Type "?? --start fast"
Sleep 600ms
Enter
Sleep 2s

# ── Beat 1 — `,` proposes a command ─────────────────────────────────────
# You're in a directory of build artifacts. You vaguely remember `du`
# exists. Ask the model in-band — no flag needed; SHELLLM_BASE_URL is
# already pointing at the fast tier.
Type ", find the five largest files in this directory"
Sleep 600ms
Enter
Wait+Screen@30s /enter: drop on prompt/
Expand All @@ -111,20 +120,19 @@ Sleep 1s
Enter
Sleep 3s

# ── Beat 2 — `,,` fixes the previous command (still on fast) ────────────
# A real flag typo, the kind people make daily. `--grpe` instead of
# `--grep`. The shell prints "unknown option" with exit status 129.
# Bare `,,` brings the terminal context (command + status) and lets the
# fast tier propose the fix.
Type "git log --oneline -n 10 --grpe 'fix'"
# ── Beat 2 — `,,` diagnoses then fixes the previous command ─────────────
# A real flag typo on `find`. BSD and GNU find both accept `-type`, so
# the corrected command works on every platform — `,,` diagnoses the
# typo in one stderr line and proposes the corrected command first.
Type "find . -tpye f -name '*.py'"
Sleep 500ms
Enter
Sleep 1500ms
Type ",, --fast"
Type ",,"
Sleep 800ms
Enter
Wait+Screen@30s /enter: drop on prompt/
Sleep 600ms
Sleep 800ms
Enter
Sleep 1s
Enter
Expand All @@ -133,10 +141,10 @@ Type "clear"
Enter
Sleep 400ms

# ── Beat 3 — `? --fast` for a quick question ────────────────────────────
# ── Beat 3 — `?` answers a quick question ───────────────────────────────
# `git stash` came to mind and you blanked. The fast tier handles this
# in seconds.
Type "? --fast in markdown and 3 lines max, what does git stash do"
Type "? in markdown and 3 lines max, what does git stash do"
Sleep 600ms
Enter
Sleep 14s
Expand Down
Loading
Loading