Port all Bash to Python — complete migration

.github/workflows/ci.yml

@@ -12,11 +12,16 @@ jobs:
     steps:
       - uses: actions/checkout@v6
 
-      - name: Install ShellCheck
-        run: sudo apt-get install -y shellcheck
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
 
-      - name: Run ShellCheck
-        run: shellcheck --severity=warning claudio lib/*.sh
+      - name: Install ruff
+        run: pip install ruff
+
+      - name: Run ruff check
+        run: ruff check lib/ tests/
 
   test:
     strategy:
@@ -26,8 +31,13 @@ jobs:
     steps:
       - uses: actions/checkout@v6
 
-      - name: Setup BATS
-        uses: bats-core/bats-action@3.0.1
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install pytest
+        run: pip install pytest
 
       - name: Run tests
-        run: bats tests/
+        run: python3 -m pytest tests/ -v

CLAUDE.md

@@ -8,30 +8,28 @@ Claudio is a messaging-to-Claude Code bridge. It supports both Telegram and What
 
 ## Architecture
 
-- `claudio` — Main CLI entry point, dispatches subcommands (`status`, `start`, `install [bot_id]`, `uninstall {<bot_id>|--purge}`, `update`, `restart`, `log`, `telegram setup`, `whatsapp setup`, `version`).
-- `lib/config.sh` — Multi-bot config management. Handles global config (`$HOME/.claudio/service.env`) and per-bot config (`$HOME/.claudio/bots/<bot_id>/bot.env`). Functions: `claudio_load_bot()`, `claudio_save_bot_env()`, `claudio_list_bots()`, `_migrate_to_multi_bot()` (auto-migrates single-bot installs).
-- `lib/server.sh` — Starts the Python HTTP server and cloudflared named tunnel together. Handles webhook registration with retry logic. `register_all_webhooks()` registers webhooks for all configured bots.
-- `lib/server.py` — Python HTTP server (stdlib `http.server`), listens on port 8421, routes POST `/telegram/webhook` and POST/GET `/whatsapp/webhook`. Multi-bot dispatch: matches Telegram webhooks via secret-token header, WhatsApp webhooks via HMAC-SHA256 signature verification. Supports dual-platform bots (same bot_id serving both Telegram and WhatsApp). Loads bot registry from `~/.claudio/bots/*/bot.env`. SIGHUP handler for hot-reload. Composite queue keys (`bot_id:chat_id` for Telegram, `bot_id:phone_number` for WhatsApp) for per-bot, per-user message isolation. `/reload` endpoint (requires `MANAGEMENT_SECRET` authentication). Logging includes bot_id via `log_msg()` helper.
-- `lib/telegram.sh` — Telegram Bot API integration (send messages, parse webhooks, image download/validation, document download, voice message handling). `telegram_setup()` accepts optional bot_id for per-bot configuration. Model commands (`/haiku`, `/sonnet`, `/opus`) save to bot.env when `CLAUDIO_BOT_DIR` is set.
-- `lib/whatsapp.sh` — WhatsApp Business API integration (send messages with 4096 char chunking, parse webhooks, image/document/audio download with magic byte validation, voice message transcription via ElevenLabs). Uses WhatsApp Cloud API v21.0. `whatsapp_setup()` accepts optional bot_id for per-bot configuration. Model commands (`/haiku`, `/sonnet`, `/opus`) save to bot.env when `CLAUDIO_BOT_DIR` is set. Security: HMAC-SHA256 signature verification, per-bot app secrets, authorized phone number enforcement.
-- `lib/claude.sh` — Claude Code CLI wrapper with conversation context injection. Uses global SYSTEM_PROMPT.md (from repo root). Supports per-bot CLAUDE.md (loaded from `$CLAUDIO_BOT_DIR` when set).
-- `lib/history.sh` — Conversation history wrapper, delegates to `lib/db.sh` for SQLite storage. Per-bot history stored in `$CLAUDIO_BOT_DIR/history.db`.
-- `lib/db.sh` — SQLite database layer for conversation storage.
-- `lib/log.sh` — Centralized logging with module prefix, optional bot_id (from `CLAUDIO_BOT_ID` env var), and file output. Format: `[timestamp] [module] [bot_id] message`.
-- `lib/health-check.sh` — Cron health-check script (runs every minute) that calls `/health` endpoint. Auto-restarts the service if unreachable (throttled to once per 3 minutes, max 3 attempts). Sends Telegram alert after exhausting retries. Additional checks when healthy: disk usage alerts, log rotation, backup freshness, and recent log analysis (errors, restart loops, slow API — configurable via `LOG_CHECK_WINDOW` and `LOG_ALERT_COOLDOWN`). State: `.last_restart_attempt`, `.restart_fail_count`, `.last_log_alert` in `$HOME/.claudio/`. Loads first bot's credentials for alerting.
-- `lib/tts.sh` — ElevenLabs text-to-speech integration for generating voice responses.
-- `lib/stt.sh` — ElevenLabs speech-to-text integration for transcribing incoming voice messages.
-- `lib/backup.sh` — Automated backup management: rsync-based hourly/daily rotating backups of `$HOME/.claudio/` with cron scheduling. Subcommands: `backup <dest>`, `backup status <dest>`, `backup cron install/uninstall`.
-- `lib/memory.sh` — Cognitive memory system (bash glue). Invokes `lib/memory.py` for embedding-based retrieval and ACT-R activation scoring. Consolidates conversation history into long-term memories. Degrades gracefully if fastembed is not installed.
+- `claudio` — Python CLI entry point, imports `lib/cli.py` and dispatches subcommands (`status`, `start`, `install [bot_id]`, `uninstall {<bot_id>|--purge}`, `update`, `restart`, `log`, `telegram setup`, `whatsapp setup`, `version`).
+- `lib/cli.py` — CLI dispatch logic. Uses `sys.argv` (not argparse) with lazy imports per command for fast startup.
+- `lib/config.py` — `ClaudioConfig` class for global config (`~/.claudio/service.env`), `BotConfig` class for per-bot config (`~/.claudio/bots/<bot_id>/bot.env`). Functions: `parse_env_file()`, `save_bot_env()`, `save_model()`. Auto-migrates single-bot to multi-bot layout.
+- `lib/server.py` — Python HTTP server (stdlib `http.server`), listens on port 8421, routes POST `/telegram/webhook` and POST/GET `/whatsapp/webhook`. Multi-bot dispatch: matches Telegram webhooks via secret-token header, WhatsApp webhooks via HMAC-SHA256 signature verification. Supports dual-platform bots (same bot_id serving both Telegram and WhatsApp). Loads bot registry from `~/.claudio/bots/*/bot.env`. SIGHUP handler for hot-reload. Composite queue keys (`bot_id:chat_id` for Telegram, `bot_id:phone_number` for WhatsApp) for per-bot, per-user message isolation. `/reload` endpoint (requires `MANAGEMENT_SECRET` authentication). Webhook processing delegates to `lib/handlers.py`.
+- `lib/handlers.py` — Webhook orchestrator: parses webhooks, runs unified message pipeline (media download, voice transcription, Claude invocation, response delivery). Entry point: `process_webhook()`.
+- `lib/telegram_api.py` — `TelegramClient` class: send messages (4096-char chunking with Markdown fallback), send voice, typing indicator, reactions, file downloads with magic byte validation. Retry on 429/5xx.
+- `lib/whatsapp_api.py` — `WhatsAppClient` class: send messages (4096-char chunking), send audio, mark read, media downloads (two-step URL resolution). Retry on 429/5xx.
+- `lib/elevenlabs.py` — ElevenLabs TTS (`tts_convert()`) and STT (`stt_transcribe()`). Stdlib only.
+- `lib/claude_runner.py` — Claude CLI invocation with `start_new_session=True`, MCP config, JSON output parsing, token usage persistence. Returns `ClaudeResult` namedtuple.
+- `lib/setup.py` — Interactive setup wizards: `telegram_setup()`, `whatsapp_setup()`, `bot_setup()`. Validates credentials via API calls, polls for Telegram `/start`, generates secrets, saves config.
+- `lib/service.py` — Service management: systemd/launchd unit generation, symlink install, cloudflared tunnel setup, webhook registration with retry, cron health-check install, Claude hooks install, `service_status()`, `service_restart()`, `service_update()`, `service_install()`, `service_uninstall()`.
+- `lib/backup.py` — Automated backup management: rsync-based hourly/daily rotating backups of `~/.claudio/` with hardlink deduplication. Functions: `backup_run()`, `backup_status()`, `backup_cron_install()`, `backup_cron_uninstall()`.
+- `lib/health_check.py` — Standalone cron health-check script (runs every minute). Calls `/health` endpoint, auto-restarts service if unreachable (throttled to once per 3 minutes, max 3 attempts), sends Telegram alert after exhausting retries. Additional checks when healthy: disk usage, log rotation, backup freshness, recent log analysis. Self-contained `_parse_env_file()` for cron's minimal PATH.
+- `lib/util.py` — Shared utilities: `sanitize_for_prompt()`, `summarize()`, filename validation, magic byte checks (image/audio/OGG), `MultipartEncoder`, `strip_markdown()`, logging helpers, CLI output helpers (`print_error`, `print_success`, `print_warning`).
+- `lib/db.py` — Python SQLite helper providing parameterized queries with retry logic.
+- `lib/memory.py` — Cognitive memory system: embedding generation (fastembed), SQLite-backed storage, ACT-R activation scoring for retrieval, and memory consolidation via Claude. Degrades gracefully if fastembed is not installed.
 - `lib/mcp_tools.py` — MCP stdio server exposing Claudio tools: Telegram notifications (`send_telegram_message`) and delayed service restart (`restart_service`). Pure stdlib, no external dependencies.
 - `lib/hooks/post-tool-use.py` — PostToolUse hook that appends compact tool usage summaries to `$CLAUDIO_TOOL_LOG`. Captures Read, Write, Edit, Bash, Glob, Grep, Task, WebSearch, WebFetch usage. Skips MCP tools (already tracked by the notifier system). Active only when `CLAUDIO_TOOL_LOG` is set.
-- `lib/memory.py` — Python backend for cognitive memory: embedding generation (fastembed), SQLite-backed storage, ACT-R activation scoring for retrieval, and memory consolidation via Claude.
-- `lib/db.py` — Python SQLite helper providing parameterized queries to eliminate SQL injection risk. Used by `db.sh`.
-- `lib/service.sh` — systemd (Linux) and launchd (macOS) service management. Also handles cloudflared installation and named tunnel setup during `claudio install`. `bot_setup()` wizard for interactive bot configuration. `service_install()` accepts optional bot_id (defaults to "claudio"). `service_uninstall()` can remove individual bots or purge all data. Enables loginctl linger on install/update (so the user service survives logout) and disables it on uninstall if no other user services remain.
 - Runtime config/state lives in `$HOME/.claudio/` (not in the repo). Multi-bot directory structure: `~/.claudio/bots/<bot_id>/` containing `bot.env`, `CLAUDE.md`, `history.db`, and SQLite WAL files.
 
 ## Development
 
-Run locally with `./claudio start`. Requires `jq`, `curl`, `python3`, `sqlite3`, `cloudflared`, and `claude` CLI. The memory system optionally requires the `fastembed` Python package (degrades gracefully without it).
+Run locally with `./claudio start`. Requires `python3`, `sqlite3`, `cloudflared`, and `claude` CLI. The memory system optionally requires the `fastembed` Python package (degrades gracefully without it).
 
-**Tests:** Run `bats tests/` (requires [bats-core](https://github.com/bats-core/bats-core)). Tests use an isolated `$CLAUDIO_PATH` to avoid touching production data.
+**Tests:** `python3 -m pytest tests/` — 640 tests covering all modules (config, util, setup, service, backup, health_check, server, handlers, telegram_api, whatsapp_api, elevenlabs, claude_runner, cli).

CONTRIBUTING.md

@@ -17,7 +17,7 @@ Contributions are welcome! Bug reports, feature requests, documentation fixes, a
 
 ## Development Setup
 
-Claudio is a shell/Python project with no build system. [ShellCheck](https://www.shellcheck.net/) is used for linting and runs automatically via a pre-commit hook.
+Claudio is a pure-Python project (stdlib only, no external dependencies except optional `fastembed`). No build system required.
 
 Run locally with:
 
@@ -29,23 +29,24 @@ Runtime configuration and state are stored in `$HOME/.claudio/` (not in the repo
 
 ## Project Structure
 
-- `claudio` — Main CLI entry point, dispatches subcommands
-- `lib/config.sh` — Multi-bot config management: global (`service.env`) and per-bot (`bots/<bot_id>/bot.env`) configuration, migration, loading, saving, listing, bot_id validation for security
-- `lib/server.sh` — Starts the Python HTTP server and cloudflared tunnel, multi-bot webhook registration
+- `claudio` — Python CLI entry point, dispatches subcommands via `lib/cli.py`
+- `lib/cli.py` — CLI dispatch logic with lazy imports per command for fast startup
+- `lib/config.py` — `ClaudioConfig` class for global config, `BotConfig` for per-bot config, env file parsing, bot_id validation
 - `lib/server.py` — Python HTTP server (stdlib `http.server`, port 8421), multi-bot dispatch via secret-token matching, SIGHUP hot-reload, `/reload` endpoint
-- `lib/telegram.sh` — Telegram Bot API integration (messages, webhooks, images, documents, voice), per-bot setup
-- `lib/claude.sh` — Claude Code CLI wrapper with conversation context, global SYSTEM_PROMPT.md and per-bot CLAUDE.md support
-- `lib/history.sh` — Conversation history management, delegates to `lib/db.sh`, per-bot history database
-- `lib/db.sh` — SQLite database layer for conversation storage
-- `lib/log.sh` — Centralized logging
-- `lib/health-check.sh` — Cron health-check script (every minute) for webhook monitoring; auto-restarts service if unreachable (throttled to once per 3 minutes, max 3 attempts), sends Telegram alert on failure
-- `lib/tts.sh` — ElevenLabs text-to-speech for voice responses
-- `lib/stt.sh` — ElevenLabs speech-to-text for voice message transcription
-- `lib/backup.sh` — Automated backup management: rsync-based hourly/daily rotating backups with cron scheduling
-- `lib/memory.sh` — Cognitive memory system (bash glue), invokes `lib/memory.py`
-- `lib/memory.py` — Python memory backend: embeddings, retrieval, consolidation
-- `lib/db.py` — Python SQLite helper with parameterized queries
-- `lib/service.sh` — systemd/launchd service management, cloudflared setup, `bot_setup()` wizard, per-bot uninstall
+- `lib/handlers.py` — Webhook orchestrator: unified pipeline for Telegram and WhatsApp
+- `lib/telegram_api.py` — `TelegramClient` class with retry logic (send, download, typing, reactions)
+- `lib/whatsapp_api.py` — `WhatsAppClient` class with retry logic (send, download, mark read)
+- `lib/elevenlabs.py` — ElevenLabs TTS/STT integration
+- `lib/claude_runner.py` — Claude CLI runner with JSON parsing
+- `lib/setup.py` — Interactive setup wizards for Telegram, WhatsApp, and multi-platform bots
+- `lib/service.py` — Service management: systemd/launchd, cloudflared tunnel, webhook registration, cron, hooks
+- `lib/backup.py` — Automated backup management: rsync-based hourly/daily rotating backups with cron scheduling
+- `lib/health_check.py` — Cron health-check script: auto-restart, disk/log/backup monitoring, Telegram alerts
+- `lib/util.py` — Shared utilities (sanitization, validation, multipart encoding, logging, CLI output)
+- `lib/db.py` — SQLite helper with parameterized queries and retry logic
+- `lib/memory.py` — Cognitive memory system: embeddings, retrieval, consolidation (optional fastembed)
+- `lib/mcp_tools.py` — MCP stdio server for Telegram notifications and service restart
+- `lib/hooks/post-tool-use.py` — PostToolUse hook for tool usage tracking
 
 **Multi-bot directory structure:**
 - `~/.claudio/service.env` — Global configuration
@@ -55,48 +56,41 @@ Runtime configuration and state are stored in `$HOME/.claudio/` (not in the repo
 
 ## Running Tests
 
-Claudio uses [BATS](https://github.com/bats-core/bats-core) for testing.
+Claudio uses [pytest](https://docs.pytest.org/) for testing.
 
 ```bash
-# Install BATS (macOS)
-brew install bats-core
-
-# Install BATS (Linux/Debian/Ubuntu)
-sudo apt-get install bats
-
 # Run all tests
-bats tests/
+python3 -m pytest tests/ -v
 
 # Run a specific test file
-bats tests/db.bats
-bats tests/multibot.bats
-```
+python3 -m pytest tests/test_handlers.py -v
 
-Tests are located in the `tests/` directory. Key test suites:
+# Run tests matching a pattern
+python3 -m pytest tests/ -k "test_webhook" -v
+```
 
-- `tests/multibot.bats` — Multi-bot config: migration, loading, saving, listing (19 tests)
-- `tests/db.bats` — SQLite conversation storage
-- `tests/telegram.bats` — Telegram API integration
-- `tests/claude.bats` — Claude Code CLI wrapper
-- `tests/health-check.bats` — Health check and monitoring
-- `tests/memory.bats` — Cognitive memory system
+Tests are located in the `tests/` directory:
+
+- `tests/test_config.py` — Config management, multi-bot migration, env file I/O
+- `tests/test_util.py` — Shared utilities (sanitization, validation, multipart encoder)
+- `tests/test_setup.py` — Setup wizards (Telegram, WhatsApp, bot selection)
+- `tests/test_service.py` — Service management (systemd, cron, webhooks, symlinks)
+- `tests/test_backup.py` — Backup operations (rsync, rotation, cron scheduling)
+- `tests/test_health_check.py` — Health check (restart throttling, disk/log/backup monitoring)
+- `tests/test_cli.py` — CLI dispatch, version, usage, argument parsing
+- `tests/test_server.py` — HTTP server routing and webhook dispatch
+- `tests/test_handlers.py` — Webhook orchestrator (integration tests)
+- `tests/test_telegram_api.py` — TelegramClient API calls
+- `tests/test_whatsapp_api.py` — WhatsAppClient API calls
+- `tests/test_elevenlabs.py` — ElevenLabs TTS/STT
+- `tests/test_claude_runner.py` — Claude CLI runner
 
 When contributing, please:
 
 - Run existing tests before submitting changes
 - Add tests for new functionality when possible (especially multi-bot behavior)
 - Ensure all tests pass
 
-## Git Hooks
-
-The project includes a pre-commit hook that runs ShellCheck and tests before each commit. To enable it:
-
-```bash
-git config core.hooksPath .githooks
-```
-
-This ensures tests pass before any commit is allowed.
-
 ## Making Changes
 
 - Check existing issues and PRs to avoid duplicate work