diff --git a/README.md b/README.md index 54b3e3d..2e5c6c2 100644 --- a/README.md +++ b/README.md @@ -1,73 +1,66 @@ # Vibetty -WebSocket terminal server with voice input support and Claude AI intelligent interaction. +Voice-driven terminal for AI coding agents. Speak into a **VibeKeys Max** keyboard and Vibetty transcribes your speech straight into a coding agent like Claude Code, Codex, or Gemini CLI — or any terminal program — served over the web. -## Features +## How it works -- **WebSocket Terminal** - Real-time terminal web interface based on Axum framework -- **Voice Input** - Speech-to-text support for executing commands via voice -- **Claude AI Integration** - AI-assisted terminal interaction using `echokit_terminal` -- **Multiple ASR Support** - - OpenAI Whisper API - - Alibaba Cloud Paraformer real-time speech recognition (todo) +You speak into the VibeKeys Max keyboard's built-in microphone (push-to-talk or toggle). The keyboard streams the audio over a WebSocket to the Vibetty server. In the default **Whisper** mode, the server packages the audio as WAV and sends it to a Whisper-compatible API (Groq, OpenAI, GLM…), then injects the transcribed text into your terminal session and the AI agent running in it. A browser view shows the live terminal. -## Quick Start +``` +VibeKeys Max mic ──WebSocket──▶ Vibetty server ──WAV──▶ Whisper API ──text──▶ terminal + agent + (push-to-talk / toggle) +``` -### ASR Configuration +Prefer not to use a cloud API? **WebVosk** mode runs speech recognition locally in the browser instead, with no API key. See [WebVosk](#webvosk--offline-no-api-key). -Vibetty supports two speech recognition modes: +## Features -#### Interactive Setup (Recommended) +- **WebSocket Terminal** - Real-time terminal web interface based on the Axum framework +- **Voice Input** - Speak commands through the VibeKeys Max microphone; speech is transcribed to text +- **Agent-agnostic** - wraps any terminal-based coding agent (Claude Code, Codex, Gemini CLI, OpenCode, aider…) or any shell command — Vibetty has no hard dependency on any one agent +- **Multiple ASR Backends** + - Whisper API — OpenAI, Groq, GLM, ByteFuture, or any custom endpoint (default) + - WebVosk — offline, in-browser, no API key + - Alibaba Cloud Paraformer real-time recognition (todo) -Run the setup wizard to configure ASR interactively: +## Installation -```bash -vibetty setup -``` - -This launches a TUI where you can: -1. Select a platform: **Whisper** or **WebVosk** -2. If Whisper, choose a provider preset: **OpenAI**, **ByteFuture**, **Groq**, **GLM**, or **Custom** -3. Fill in API key and other settings -4. Configuration is saved to `~/.vibetty/config.toml` +### Option A: Download a pre-built binary -#### Manual Configuration +Download the latest release for your platform from the [releases page](https://github.com/second-state/vibetty/releases): -You can also configure ASR manually via environment variables. +| Platform | Asset | +|---|---| +| Linux | `vibetty-linux-x64` | +| macOS (Apple Silicon) | `vibetty-macos-arm64` | +| Windows | `vibetty-windows-x64.exe` | -##### Option 1: Whisper API (Server-side) +### Option B: Build from source -Create a `.env` file and configure the Whisper API (Groq recommended). Alternatively, you can set these environment variables directly in your shell configuration file (e.g., `~/.bashrc` or `~/.zshrc`): +Requires [Rust](https://rustup.rs/). ```bash -VIBECODE_ASR_API_KEY=your_api_key_here -VIBECODE_ASR_URL=https://api.groq.com/openai/v1/audio/transcriptions -VIBECODE_ASR_MODEL=whisper-large-v3 -VIBECODE_ASR_LANG=en -VIBECODE_ASR_PROMPT= +cargo build --release +# binary at ./target/release/vibetty (vibetty.exe on Windows) ``` -Then start the service: - -**Option A: Download pre-built binary** +### Add to your PATH (optional) -Download the latest release from the [releases page](https://github.com/second-state/vibetty/releases): +To run `vibetty` from any directory, place the binary in a directory on your `PATH`. We recommend `~/.cargo/bin`: ```bash -# After downloading -./vibetty -- claude -``` - -**Option B: Build from source** +# Pre-built binary +mv vibetty ~/.cargo/bin/ -```bash -# Build the release binary -cargo build --release -./target/release/vibetty -- claude +# Or self-compiled binary +mv target/release/vibetty ~/.cargo/bin/ ``` -**Tip:** To run `vibetty` from any directory, place the binary in a directory on your `PATH`. If it exists, we recommend `~/.cargo/bin`: +On Windows (PowerShell): +```powershell +move vibetty-windows-x64.exe $env:USERPROFILE\.cargo\bin\vibetty.exe +```
What is PATH? @@ -99,52 +92,134 @@ echo $PATH | grep -q "$HOME/.cargo/bin" && echo "Yes" || echo "No" ```
+## Quick Start -```bash -# For pre-built binary -mv vibetty ~/.cargo/bin/ +Vibetty defaults to **Whisper** mode (cloud transcription). The fastest path: -# Or for self-compiled binary -mv target/release/vibetty ~/.cargo/bin/ +**1. Get a Whisper API key.** Groq is recommended. You can also use OpenAI, GLM, ByteFuture, or any Whisper-compatible endpoint. + +**2. Configure ASR** with the interactive wizard (writes `~/.vibetty/config.toml`): + +```bash +vibetty setup ``` +Manual environment-variable configuration is covered in [Configuration](#configuration). + +**3. Start the server with your agent.** Anything after `--` is launched in the terminal, so Vibetty works with any coding agent CLI: + +| Agent | Launch command | +|---|---| +| Claude Code | `vibetty -- claude` | +| OpenAI Codex | `vibetty -- codex` | +| Gemini CLI | `vibetty -- gemini` | +| OpenCode | `vibetty -- opencode` | +| aider | `vibetty -- aider` | +| Plain shell | `vibetty -- bash` | + +The agent CLI must already be installed and on your `PATH`. + +**4. Pair your VibeKeys Max.** Open `http://localhost:3000/setup` and connect to the keyboard over Bluetooth. Set the **VibeKeys server WebSocket URL** to your Vibetty server (e.g. `ws://:3000/ws`) and choose a **Microphone Mode** (PushToTalk or Toggle). + +**5. Watch the terminal.** Open `http://localhost:3000` to see the live session. Speak into the keyboard and your words run as commands. + +For all options: ```bash -# For pre-built binary -mv vibetty ~/.cargo/bin/ +vibetty --help +``` -# Or for self-compiled binary -mv target/release/vibetty ~/.cargo/bin/ +## Configuration + +Vibetty supports two speech-recognition backends. Configure either interactively or via environment variables. + +### Interactive setup (recommended) + +```bash +vibetty setup ``` -##### Option 2: WebVosk (Browser-side) +A TUI walks you through: +1. Select a platform: **Whisper** or **WebVosk** +2. If Whisper, choose a provider preset: **OpenAI**, **ByteFuture**, **Groq**, **GLM**, or **Custom** +3. Fill in the API key and other settings +4. Settings are saved to `~/.vibetty/config.toml` -Speech recognition runs entirely in the browser using Vosk models. No API key required. +### Whisper (default) -**Option A: Download pre-built binary** +Server-side transcription via a Whisper-compatible API. Create a `.env` file (or set these in your shell profile, e.g. `~/.bashrc` / `~/.zshrc`): ```bash -# Set ASR platform and run -VIBECODE_ASR_PLATFORM=web_vosk ./vibetty -- claude +VIBECODE_ASR_API_KEY=your_api_key_here +VIBECODE_ASR_URL=https://api.groq.com/openai/v1/audio/transcriptions +VIBECODE_ASR_MODEL=whisper-large-v3 +VIBECODE_ASR_LANG=en +VIBECODE_ASR_PROMPT= ``` -**Option B: Build from source** +Then start the service: ```bash -# Set ASR platform and run -VIBECODE_ASR_PLATFORM=web_vosk ./vibetty -- -- claude +vibetty -- claude ``` -Then visit the WebVosk interface at: http://localhost:3000/vosk +### WebVosk — offline, no API key -**Note:** First-time use requires downloading Vosk model files (~40MB each). The models are cached in your browser. +Speech recognition runs entirely in the browser using Vosk models. No API key required, and no audio is sent to a cloud service. -For more options, use `--help`: ```bash -./vibetty --help +VIBECODE_ASR_PLATFORM=web_vosk vibetty -- claude +``` + +Then open the WebVosk interface at `http://localhost:3000/vosk`. + +**Note:** First-time use downloads Vosk model files (~40MB each). The models are cached in your browser. + +## Environment Variables + +| Variable | Description | Default | +|---|---|---| +| `VIBECODE_ASR_PLATFORM` | ASR platform to use: `whisper` or `web_vosk` | `whisper` | +| `VIBECODE_ASR_URL` | Whisper API endpoint URL | `https://api.openai.com/v1/audio/transcriptions` | +| `VIBECODE_ASR_API_KEY` | Whisper API key (Groq recommended) | _(empty)_ | +| `VIBECODE_ASR_MODEL` | Whisper model name | `whisper-1` | +| `VIBECODE_ASR_LANG` | ASR language (e.g. `en`, `zh`) | _(empty, auto-detected by API)_ | +| `VIBECODE_ASR_PROMPT` | Prompt passed to the Whisper API to guide transcription | _(empty)_ | +| `VIBECODE_ASR_DEBUG_WAV` | Set to any value to save recorded audio as `debug_.wav` for debugging | _(unset)_ | +| `VIBECODE_EXIT_COMMAND` | Custom voice exit command. When ASR result matches this value (case-insensitive), it is replaced with `/exit` | _(unset)_ | + +> **Note:** Legacy environment variables (e.g. `ASR_URL`, `ASR_API_KEY`, `VIBETTY_EXIT_COMMAND`, etc.) have been renamed with the `VIBECODE_` prefix. Using old names will trigger a warning but still work. Please migrate to the new names. + +## Platform Support + +Vibetty runs on **Linux**, **macOS**, and **Windows**. + +| Platform | PTY backend | Requirements | +|---|---|---| +| Linux | Unix PTY | — | +| macOS | Unix PTY | — | +| Windows | ConPTY (via [`portable-pty-psmux`](https://crates.io/crates/portable-pty-psmux)) | Windows 10 (1809+) or Windows 11 | + +### Running on Windows + +The quick-start commands above use Unix-style paths; on Windows, use the `.exe` and backslash paths from **PowerShell** or **Command Prompt**: + +```powershell +# Pre-built binary +.\vibetty-windows-x64.exe -- claude + +# Or build from source +cargo build --release +.\target\release\vibetty.exe -- claude ``` -Visit: http://localhost:3000 after starting the service. +Set environment variables with `$env:` in PowerShell: + +```powershell +$env:VIBECODE_ASR_API_KEY = "your_api_key_here" +$env:VIBECODE_ASR_URL = "https://api.groq.com/openai/v1/audio/transcriptions" +.\vibetty.exe -- claude +``` ## API Reference @@ -175,22 +250,3 @@ curl -X POST http://localhost:3000/api/change-dir \ ``` **Note:** This endpoint only accepts requests from localhost for security reasons. - -## Environment Variables - -| Variable | Description | Default | -|---|---|---| -| `VIBECODE_ASR_PLATFORM` | ASR platform to use: `whisper` or `web_vosk` | `whisper` | -| `VIBECODE_ASR_URL` | Whisper API endpoint URL | `https://api.openai.com/v1/audio/transcriptions` | -| `VIBECODE_ASR_API_KEY` | Whisper API key (Groq recommended) | _(empty)_ | -| `VIBECODE_ASR_MODEL` | Whisper model name | `whisper-1` | -| `VIBECODE_ASR_LANG` | ASR language (e.g. `en`, `zh`) | _(empty, auto-detected by API)_ | -| `VIBECODE_ASR_PROMPT` | Prompt passed to the Whisper API to guide transcription | _(empty)_ | -| `VIBECODE_ASR_DEBUG_WAV` | Set to any value to save recorded audio as `debug_.wav` for debugging | _(unset)_ | -| `VIBECODE_EXIT_COMMAND` | Custom voice exit command. When ASR result matches this value (case-insensitive), it is replaced with `/exit` | _(unset)_ | - -> **Note:** Legacy environment variables (e.g. `ASR_URL`, `ASR_API_KEY`, `VIBETTY_EXIT_COMMAND`, etc.) have been renamed with the `VIBECODE_` prefix. Using old names will trigger a warning but still work. Please migrate to the new names. - -## Platform Support - -Supports **Linux**, **macOS**, and **Windows** (via ConPTY). diff --git a/docs/README.zh.md b/docs/README.zh.md index 4ca8160..4859165 100644 --- a/docs/README.zh.md +++ b/docs/README.zh.md @@ -1,43 +1,123 @@ # Vibetty -WebSocket 终端服务器,支持语音输入和 Claude AI 智能交互。 +面向 AI 编程 agent 的语音终端。对着 **VibeKeys Max** 键盘说话,Vibetty 会把语音转写后直接送进你选择的编程 agent —— Claude Code、Codex、Gemini CLI —— 或任意终端程序,并通过网页提供访问。 + +## 工作原理 + +你对着 VibeKeys Max 键盘的内置麦克风说话(按住说话 / 切换模式)。键盘把音频通过 WebSocket 流式传到 Vibetty 服务端。在默认的 **Whisper** 模式下,服务端把音频打包成 WAV 发送给 Whisper 兼容 API(Groq、OpenAI、GLM…),再把转写出的文本注入终端会话以及其中运行的 AI agent。浏览器页面用于显示实时终端。 + +``` +VibeKeys Max 麦克风 ──WebSocket──▶ Vibetty 服务端 ──WAV──▶ Whisper API ──文本──▶ 终端 + agent + (按住说话 / 切换) +``` + +不想用云端 API?**WebVosk** 模式可以完全在浏览器本地做语音识别,无需 API Key,详见 [WebVosk](#webvosk离线无需-api-key)。 ## 功能特性 - **WebSocket 终端** - 基于 Axum 框架的实时终端 Web 接口 -- **语音输入** - 支持语音转文字,可通过语音执行命令 -- **Claude AI 集成** - 使用 `echokit_terminal` 实现 AI 辅助终端交互 -- **多种 ASR 支持** - - OpenAI Whisper API - - 阿里云 Paraformer 实时语音识别(todo) +- **语音输入** - 通过 VibeKeys Max 麦克风说话,语音转写为文本 +- **Agent 无关** - 可包裹任意基于终端的编程 agent(Claude Code、Codex、Gemini CLI、OpenCode、aider…)或任意 shell 命令 —— Vibetty 不绑定任何单一 agent +- **多种 ASR 后端** + - Whisper API —— OpenAI、Groq、GLM、ByteFuture 或任意自定义端点(默认) + - WebVosk —— 离线、浏览器内、无需 API Key + - 阿里云 Paraformer 实时语音识别(todo) + +## 安装 + +### 方式 A:下载预编译二进制 + +从 [releases 页面](https://github.com/second-state/vibetty/releases) 下载对应平台的最新版本: + +| 平台 | 文件 | +|---|---| +| Linux | `vibetty-linux-x64` | +| macOS(Apple Silicon) | `vibetty-macos-arm64` | +| Windows | `vibetty-windows-x64.exe` | + +### 方式 B:从源码编译 + +需要安装 [Rust](https://rustup.rs/)。 + +```bash +cargo build --release +# 二进制位于 ./target/release/vibetty(Windows 上为 vibetty.exe) +``` + +### 加入 PATH(可选) + +如需在任意目录下运行 `vibetty`,把二进制文件放进 `PATH` 中的目录,推荐 `~/.cargo/bin`: + +```bash +# 预编译二进制 +mv vibetty ~/.cargo/bin/ + +# 或自行编译的二进制 +mv target/release/vibetty ~/.cargo/bin/ +``` + +Windows(PowerShell): + +```powershell +move vibetty-windows-x64.exe $env:USERPROFILE\.cargo\bin\vibetty.exe +``` ## 快速开始 -### ASR 配置 +Vibetty 默认使用 **Whisper** 模式(云端转写)。最快路径: + +**1. 获取 Whisper API Key。** 推荐 Groq,也可使用 OpenAI、GLM、ByteFuture 或任意 Whisper 兼容端点。 + +**2. 配置 ASR**,运行交互式向导(会写入 `~/.vibetty/config.toml`): + +```bash +vibetty setup +``` + +手动通过环境变量配置见 [配置](#配置)。 + +**3. 启动服务并带上你的 agent。** `--` 后面的内容都会在终端里启动,所以 Vibetty 适配任意 coding agent CLI: -Vibetty 支持两种语音识别模式: +| Agent | 启动命令 | +|---|---| +| Claude Code | `vibetty -- claude` | +| OpenAI Codex | `vibetty -- codex` | +| Gemini CLI | `vibetty -- gemini` | +| OpenCode | `vibetty -- opencode` | +| aider | `vibetty -- aider` | +| 纯 shell | `vibetty -- bash` | -#### 交互式配置(推荐) +对应的 agent CLI 需要已经安装并在你的 `PATH` 中。 -运行配置向导来交互式配置 ASR: +**4. 配对 VibeKeys Max。** 打开 `http://localhost:3000/setup`,通过蓝牙连接键盘。把 **VibeKeys 服务器 WebSocket 地址** 设为你的 Vibetty 服务端(例如 `ws://<你的主机>:3000/ws`),并选择**麦克风模式**(PushToTalk 或 Toggle)。 + +**5. 查看终端。** 打开 `http://localhost:3000` 看到实时会话。对着键盘说话,你的话会作为命令执行。 + +查看全部参数: + +```bash +vibetty --help +``` + +## 配置 + +Vibetty 支持两种语音识别后端,可交互式配置,也可通过环境变量配置。 + +### 交互式配置(推荐) ```bash vibetty setup ``` -启动后会进入 TUI 界面,可以: +进入 TUI 界面,可以: 1. 选择平台:**Whisper** 或 **WebVosk** -2. 如果选择 Whisper,可以选择提供商预设:**OpenAI**、**ByteFuture**、**Groq**、**GLM** 或 **Custom** +2. 如果选择 Whisper,可选择提供商预设:**OpenAI**、**ByteFuture**、**Groq**、**GLM** 或 **Custom** 3. 填写 API Key 等配置项 4. 配置保存到 `~/.vibetty/config.toml` -#### 手动配置 +### Whisper(默认) -也可以通过环境变量手动配置 ASR。 - -##### 选项 1:Whisper API(服务器端) - -创建 `.env` 文件并配置 Whisper API(推荐使用 Groq): +服务端通过 Whisper 兼容 API 转写。创建 `.env` 文件(或写入 shell 配置文件,如 `~/.bashrc` / `~/.zshrc`): ```bash VIBECODE_ASR_API_KEY=your_api_key_here @@ -50,33 +130,66 @@ VIBECODE_ASR_PROMPT= 然后启动服务: ```bash -# 使用 cargo 直接运行 -cargo run -- -- claude - -# 或者先编译再运行 -cargo build --release -./target/release/vibetty -- claude +vibetty -- claude ``` -##### 选项 2:WebVosk(浏览器端) +### WebVosk(离线,无需 API Key) -语音识别完全在浏览器中使用 Vosk 模型运行,无需 API 密钥。 +语音识别完全在浏览器中使用 Vosk 模型运行,无需 API 密钥,音频也不会发送到云端。 ```bash -# 设置 ASR 平台为 WebVosk -VIBECODE_ASR_PLATFORM=web_vosk cargo run -- -- claude +VIBECODE_ASR_PLATFORM=web_vosk vibetty -- claude ``` -然后访问 WebVosk 界面:http://localhost:3000/vosk +然后访问 WebVosk 界面:`http://localhost:3000/vosk` **注意:** 首次使用需要下载 Vosk 模型文件(每个约 40MB)。模型会缓存在浏览器中。 -更多参数可以使用 `--help` 命令查看: -```bash -cargo run -- --help +## 环境变量 + +| 变量 | 说明 | 默认值 | +|---|---|---| +| `VIBECODE_ASR_PLATFORM` | 使用的 ASR 平台:`whisper` 或 `web_vosk` | `whisper` | +| `VIBECODE_ASR_URL` | Whisper API 端点 URL | `https://api.openai.com/v1/audio/transcriptions` | +| `VIBECODE_ASR_API_KEY` | Whisper API Key(推荐 Groq) | _(空)_ | +| `VIBECODE_ASR_MODEL` | Whisper 模型名 | `whisper-1` | +| `VIBECODE_ASR_LANG` | ASR 语言(如 `en`、`zh`) | _(空,由 API 自动检测)_ | +| `VIBECODE_ASR_PROMPT` | 传给 Whisper API 用于引导转写的提示词 | _(空)_ | +| `VIBECODE_ASR_DEBUG_WAV` | 设为任意值即可把录音保存为 `debug_.wav` 用于调试 | _(未设置)_ | +| `VIBECODE_EXIT_COMMAND` | 自定义语音退出命令。当 ASR 结果(不区分大小写)匹配该值时,替换为 `/exit` | _(未设置)_ | + +> **注意:** 旧版环境变量(如 `ASR_URL`、`ASR_API_KEY`、`VIBETTY_EXIT_COMMAND` 等)已统一加上 `VIBECODE_` 前缀。使用旧名称会有警告但仍可工作,请尽快迁移到新名称。 + +## 平台支持 + +Vibetty 支持 **Linux**、**macOS** 和 **Windows**。 + +| 平台 | PTY 后端 | 系统要求 | +|---|---|---| +| Linux | Unix PTY | — | +| macOS | Unix PTY | — | +| Windows | ConPTY(基于 [`portable-pty-psmux`](https://crates.io/crates/portable-pty-psmux)) | Windows 10(1809+)或 Windows 11 | + +### 在 Windows 上运行 + +上面的快速开始命令使用 Unix 风格路径;在 Windows 上请在 **PowerShell** 或**命令提示符**中使用 `.exe` 和反斜杠路径: + +```powershell +# 预编译二进制 +.\vibetty-windows-x64.exe -- claude + +# 或从源码编译 +cargo build --release +.\target\release\vibetty.exe -- claude ``` -服务启动后访问: http://localhost:3000 +在 PowerShell 中用 `$env:` 设置环境变量: + +```powershell +$env:VIBECODE_ASR_API_KEY = "your_api_key_here" +$env:VIBECODE_ASR_URL = "https://api.groq.com/openai/v1/audio/transcriptions" +.\vibetty.exe -- claude +``` ## API 参考