diff --git a/.gitignore b/.gitignore index 7f3c6e14..710873e3 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,7 @@ *.pid # Local Codex inspection artifacts (don't ship) +codedb.snapshot *.asar app-asar-work/ *-bsLDOISN.js diff --git a/CHANGELOG.md b/CHANGELOG.md index a9f14dd8..0454ba7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,48 @@ and this project does not yet follow semantic versioning (pre-1.0). ### Added +- `codex-shim doctor`, a read-only local diagnostics command covering Python, + dependencies, Codex CLI availability, settings, runtime files, daemon health, + passthrough readiness, proxy loopback bypass, and Codex config wiring with + stable OK/WARN/FAIL/INFO output and summary exit-code handling. +- `docs/subscription-integration.md`, covering ChatGPT/Codex and + Cursor/Composer subscription passthrough setup, troubleshooting, limitations, + and privacy notes. +- Auto Router (`codex_shim/router.py`): an optional `Auto (smart routing)` picker + entry (slug `codex-auto`) that routes each task to the cheapest configured + model that can handle it. A cheap classifier model scores every candidate + `0.0–1.0` from a capability card, the shim picks the cheapest candidate whose + score clears `threshold` (default `0.7`), caches the decision per task, and + falls back safely on any error. Configured via an optional `router` block in + `~/.codex-shim/models.json`; gated in `/health`, `/v1/models`, `/api/models`, + the generated catalog, and `codex-shim list`. Env knobs: + `CODEX_SHIM_DISABLE_ROUTER`, `CODEX_SHIM_ROUTER_TIMEOUT`, + `CODEX_SHIM_ROUTER_MAX_TOKENS`, `CODEX_SHIM_ROUTER_LOG`. Documented in + `docs/AUTO_ROUTER.md` with a runnable offline proof at + `examples/auto_router_demo.py` and 48 offline tests + (`tests/test_router.py`, `tests/test_router_integration.py`) covering + scoring/selection, streaming, compaction, the chat endpoint, the agent + tool-loop cache, OpenAI/Anthropic classifiers, the exact classifier HTTP, + fallbacks, availability gating, and concurrency. +- Cursor/Composer subscription passthrough for slug `composer-2-5`. When + `cursor-agent login` is active, the shim spawns `cursor-agent --print` with + CLI OAuth (no Dashboard API key). The slug is auth-gated in `/health`, + `/v1/models`, and the generated catalog like ChatGPT passthrough. +- `POST /v1/responses/compact` support. ChatGPT passthrough forwards to the + native ChatGPT compact endpoint; BYOK OpenAI/chat and Anthropic routes run a + non-streaming compact summarization request and return a Responses-shaped + compacted window for the next Codex turn. +- BYOK fallback schemas for native Responses-only tools: `computer_use`, + `web_search`, `apply_patch`, and `local_shell` now translate into ordinary + function tools for chat-completions / Anthropic providers instead of being + dropped. Codex MCP function tools continue to pass through unchanged. +- Streaming `response.completed` events now include upstream `usage` when chat + or Anthropic streams provide it, so Codex can track token counts and trigger + auto-compaction. +- BYOK visual feedback passthrough for computer-use loops: Responses + `input_image`, `computer_call_output` screenshots, and visual + `function_call_output` payloads now reach OpenAI chat providers as + `image_url` parts and Anthropic providers as image blocks. - GitHub Actions CI (`.github/workflows/ci.yml`) running pytest and `compileall` on Python 3.11 and 3.12. - `[project.optional-dependencies] dev` in `pyproject.toml` so @@ -17,6 +59,20 @@ and this project does not yet follow semantic versioning (pre-1.0). and what to include in bug reports. - `.github/ISSUE_TEMPLATE/` with structured bug and feature request templates. - `CHANGELOG.md` (this file). +- Web-based model picker at `GET /picker` (with `GET /api/models` and + `POST /api/switch`) so the active shim model can be swapped from a browser + without the CLI. Switching rewrites `model = "..."` and the + `[model_providers.codex_shim]` `name = "..."` in `~/.codex/config.toml` so + the Codex Desktop UI shows the selected model name (e.g. "Kimi K2.6") + instead of the generic "Codex Shim" label. Optional auto-restart of Codex + Desktop is cross-platform (`taskkill` + `Codex.exe` on Windows, + `osascript` + `open -a Codex` on macOS). All picker routes are behind the + existing `Host`-header allowlist, so a visited web page still cannot drive + them via DNS rebinding. +- Best-effort dump of the last forwarded chat-completions request body to + `.codex-shim/last_request.json` to make strict-provider tokenization / + schema errors easier to triage. Upstream error bodies are now logged with + the model slug before being forwarded back. ### Changed @@ -28,6 +84,31 @@ and this project does not yet follow semantic versioning (pre-1.0). while still accepting `customModels` and camelCase aliases for existing exports. +### Fixed + +- Protected the state-changing picker `/api/switch` endpoint with a + per-process picker token so third-party pages cannot trigger model switches + or Desktop restarts through the loopback server. +- Image detail normalization in `responses_to_chat`: Codex Desktop's + `detail: "original"` on `input_image` items is mapped to `"high"` for + OpenAI Chat Completions providers; unknown detail values fall back to `"auto"`. +- `codex-shim patch-app` regex needles now match both legacy inline picker + filters in `model-queries-*.js` and newer extracted helpers in + `models-and-reasoning-efforts-*.js`, with APPLIED markers for idempotent + re-runs. +- Anthropic route requests now send only `x-api-key` (plus `anthropic-version`) + for authentication and no longer also attach `Authorization: Bearer `. + Some Anthropic-compatible gateways reject requests that carry both headers. + Providers that genuinely require a bearer token can still supply one via + `extraHeaders`. +- `codex-shim patch-app` now also patches the Codex Desktop sidebar's recent + thread loader so native `openai` chats remain visible while Desktop is routed + through the `codex_shim` provider. Tested on Codex Desktop 26.519.41501 / + `codex-cli 0.133.0-alpha.1` on macOS arm64. +- `patch-app` now updates `ElectronAsarIntegrity` in `Info.plist` after + repacking `app.asar`, and `restore-app` restores or recomputes that metadata + before re-signing the app bundle. + ## 2026-05-25 — Auth-gated ChatGPT passthrough + docs hardening ### Added diff --git a/Launch Codex-Shim.bat b/Launch Codex-Shim.bat new file mode 100644 index 00000000..13888df2 --- /dev/null +++ b/Launch Codex-Shim.bat @@ -0,0 +1,3 @@ +@echo off +title Codex-Shim Launcher +powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0launch.ps1" diff --git a/README.md b/README.md index 965e9eb4..5f3841d4 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,14 @@ local: - **ChatGPT/Codex passthrough.** If `~/.codex/auth.json` has a valid Codex access token, the shim can route Codex's native `/v1/responses` traffic to ChatGPT's Codex backend under the `gpt-5.5` slug used by current Codex builds. +- **Cursor/Composer passthrough.** If `cursor-agent login` is active, the shim + exposes `composer-2-5` and routes through your Cursor subscription — no + Dashboard API key (`crsr_…`) required. See + [`docs/subscription-integration.md`](docs/subscription-integration.md). +- **Auto Router (optional).** Add an `Auto (smart routing)` picker entry that + uses a cheap classifier model to route each task to the cheapest configured + model that can handle it — trivial turns stay cheap, hard turns escalate. See + [`docs/AUTO_ROUTER.md`](docs/AUTO_ROUTER.md). - **Prompt-catching/proxy-friendly architecture.** Put a local proxy in front of the shim to dedupe boilerplate, inject stable instructions, repair pseudo-tool text, or route prompts by policy before they hit an upstream. @@ -349,13 +357,14 @@ Recommended schema: } ``` -The loader also accepts camelCase aliases (`baseUrl`, `apiKey`, `displayName`, -`maxContextLimit`, `maxOutputTokens`, `noImageSupport`, `extraHeaders`) and a -legacy top-level `customModels` array, so existing model config exports can be -used directly. +The loader also accepts camelCase aliases (`baseUrl`, `apiKey`, `apiKeyEnv`, +`displayName`, `maxContextLimit`, `maxOutputTokens`, `noImageSupport`, +`extraHeaders`) and a legacy top-level `customModels` array, so existing model +config exports can be used directly. -The shim **never copies your API keys** into the generated catalog. Keys stay -in your settings file and are read fresh on every request. +The shim **never writes your API keys** into the generated catalog. Put literal +keys in your settings file or reference them with `api_key_env`; credentials +are resolved when requests are handled. Supported `provider` values: @@ -366,16 +375,73 @@ Supported `provider` values: | `minimax` | MiniMax Token Plan `/v1/chat/completions` | | `anthropic` | Anthropic `/v1/messages` | +The shim also accepts Anthropic Messages requests at +`http://127.0.0.1:8765/v1/messages`. For `openai` and +`generic-chat-completion-api` models, it translates Messages requests to +OpenAI-shaped chat completions and converts responses back to Anthropic shape. +For `anthropic` models, it passes the request through to the upstream +`/messages` endpoint with the configured model name. The bridge supports text, +image inputs, basic function tools/tool results, and streaming SSE. Provider +features such as prompt caching, extended thinking signatures, files, and token +counting remain upstream-specific. + Useful model fields: | field | behavior | |---|---| | `display_name` | Human-readable picker label. | +| `api_key_env` | Name of an environment variable that contains the upstream API key. | | `max_context_limit` | Catalog context window and compaction limits. | | `max_output_tokens` | Default max output when translating to Anthropic. | | `no_image_support` | When true, catalog advertises text-only input. | | `extra_headers` | Optional upstream headers merged into requests. | +### OpenCode Go + +OpenCode Go adds and updates models over time. Refresh the local settings from +the live OpenCode Go catalog instead of copying a hard-coded model list: + +```bash +export OPENCODE_GO_API_KEY="..." +codex-shim opencode-go refresh +codex-shim generate +codex-shim start +``` + +The refresh command calls `https://opencode.ai/zen/go/v1/models`, probes each +model through both `/chat/completions` and `/messages`, and writes `ocgo-*` +entries into `~/.codex-shim/models.json`. Models that work through chat +completions are configured as `generic-chat-completion-api`; models that only +work through Messages are configured as `anthropic`. + +Use `--settings` to write a different file, `--api-key-env` to use a different +environment variable name, or `--prefer messages` if you want models that +support both routes to prefer Anthropic Messages: + +```bash +codex-shim --settings /path/to/models.json opencode-go refresh --prefer messages +``` + +If you need a minimal manual fallback, add one model with the same key env: + +```json +{ + "models": [ + { + "slug": "ocgo-glm-5-1", + "model": "glm-5.1", + "display_name": "OpenCode Go GLM 5.1", + "provider": "generic-chat-completion-api", + "base_url": "https://opencode.ai/zen/go/v1", + "api_key_env": "OPENCODE_GO_API_KEY" + } + ] +} +``` + +The current OpenCode Go model list and endpoint split are documented at +. + ### Ollama / local OpenAI-compatible chat endpoints Codex sends the Responses API. Ollama and many local servers expose @@ -422,7 +488,14 @@ that hides any model whose slug is not on a hardcoded list. Custom catalog entries fall into the hidden bucket and never render in the picker. A single-boolean ASAR patch flips the allowlist branch off so the picker only -checks the local `hidden` flag (which this catalog never sets). +checks the local `hidden` flag (which this catalog never sets). On recent +Codex Desktop builds, the patch also changes the local recent-thread loader +from `modelProviders: null` to `modelProviders: []` so the sidebar continues to +show existing native `openai` chats while Desktop is routed through the +`codex_shim` provider. + +The combined patch has been tested on Codex Desktop **26.519.41501** / +`codex-cli 0.133.0-alpha.1` on macOS arm64. > Back up `app.asar` and `Info.plist` before patching. @@ -440,7 +513,22 @@ sed -i.bak -E 's/let u=c\.useHiddenModels&&o!==`amazonBedrock`,d;/let u=!1,d;/' diff "$PATCH_FILE.bak" "$PATCH_FILE" || true rm "$PATCH_FILE.bak" -# 3. Repack +# 3. Patch the sidebar recent-thread provider filter (single occurrence) +SIDEBAR_FILE=$(grep -RIl 'listRecentThreads' extracted/webview/assets/app-server-manager-signals-*.js | head -n1) +python3 - "$SIDEBAR_FILE" <<'PY' +from pathlib import Path +import sys + +path = Path(sys.argv[1]) +text = path.read_text() +old = "listRecentThreads({cursor:e,limit:t}){return this.params.requestClient.sendRequest(`thread/list`,{limit:t,cursor:e,sortKey:this.recentConversationSortKey,modelProviders:null,archived:!1,sourceKinds:ke})}" +new = "listRecentThreads({cursor:e,limit:t}){return this.params.requestClient.sendRequest(`thread/list`,{limit:t,cursor:e,sortKey:this.recentConversationSortKey,modelProviders:[],archived:!1,sourceKinds:ke})}" +if text.count(old) != 1: + raise SystemExit("expected one sidebar provider filter occurrence") +path.write_text(text.replace(old, new, 1)) +PY + +# 4. Repack npx --yes @electron/asar pack extracted app.asar.new sudo cp app.asar.new "$APP/Contents/Resources/app.asar" ``` @@ -450,7 +538,7 @@ That alone can crash Codex on next launch with `EXC_BREAKPOINT`. Electron's header** of the ASAR archive (not the whole file). Recompute it and re-sign: ```bash -# 4. Compute new header hash +# 5. Compute new header hash HEADER_HASH=$(python3 - "$APP/Contents/Resources/app.asar" <<'PY' import struct, hashlib, sys with open(sys.argv[1], 'rb') as f: @@ -461,29 +549,30 @@ PY ) echo "new header hash: $HEADER_HASH" -# 5. Patch Info.plist (replaces the hash for Resources/app.asar) +# 6. Patch Info.plist (replaces the hash for Resources/app.asar) sudo /usr/libexec/PlistBuddy -c \ "Set :ElectronAsarIntegrity:Resources/app.asar:hash $HEADER_HASH" \ "$APP/Contents/Info.plist" -# 6. Ad-hoc re-sign +# 7. Ad-hoc re-sign sudo codesign --force --deep --sign - "$APP" -# 7. Launch +# 8. Launch open "$APP" ``` To roll back: `sudo rm -rf "$APP" && sudo mv "$APP.unpatched-…" "$APP"`. -The CLI also has helper commands for patching/restoring `app.asar`: +The CLI also has helper commands for patching/restoring `app.asar` and the +matching ASAR integrity metadata: ```bash codex-shim patch-app codex-shim restore-app ``` -If Codex still crashes after `patch-app`, use the manual hash-update steps above -or restore with `codex-shim restore-app`. +If Codex still crashes after `patch-app`, restore with `codex-shim restore-app` +and re-check the manual patch needles against the installed Desktop build. --- @@ -520,6 +609,31 @@ that prefix as an alias and routes it to the same passthrough. --- +## Cursor/Composer passthrough (subscription) + +If `cursor-agent status` shows you are logged in, the shim exposes +**Composer 2.5** as slug `composer-2-5` and routes each request by spawning +`cursor-agent` with your CLI OAuth session — the same pattern +[Open Design](https://github.com/nexu-io/open-design) uses for Cursor Agent. + +```bash +cursor-agent login +scripts/codex-shim-install-cursor-composer +codex-shim model use composer-2-5 +codex-app +``` + +The install helper is optional; it regenerates the local catalog/config and +sets `composer-2-5` as the active model when `cursor-agent status` reports an +active login. Troubleshoot with `cursor-agent status` and `/health`, which +reports `cursor_passthrough: true` when the shim can expose Composer. + +Do **not** configure Composer via `cursor-api.standardagents.ai` unless you +intentionally want Dashboard API-key billing (`crsr_…`). That path is BYOK, +not CLI subscription. + +--- + ## How routing works ```text @@ -545,6 +659,53 @@ GLM, etc.) round-trip through `reasoning.encrypted_content` items. --- +## Auto Router (smart routing) + +Optionally add one extra picker entry — **`Auto (smart routing)`** (slug +`codex-auto`) — that chooses the right model *per task*: trivial turns go to a +cheap model, hard turns escalate to your strongest one. It runs entirely on the +models you already configure. + +On each new task the shim asks a cheap **classifier** model you nominate to score +every candidate `0.0–1.0` (how likely it nails the task first try), reading a +short **capability card** per candidate. It then routes to the **cheapest +candidate whose score clears `threshold`** (default `0.7`), caches that decision +for the task's tool-call round-trips, and falls back safely on any error. The +classifier never sees price, so it can't be biased toward expensive models. + +Turn it on by adding a `router` block to `~/.codex-shim/models.json`: + +```jsonc +"router": { + "enabled": true, + "slug": "codex-auto", + "classifier": "minimax-m3", // slug of a cheap configured model + "threshold": 0.7, + "default": "minimax-m3", + "cache": true, + "candidates": [ + { "slug": "minimax-m3", "cost": 0.3, "supports_images": false, + "card": "Cheap, fast. Single-file edits, codegen, simple refactors." }, + { "slug": "opus", "cost": 5.0, "supports_images": true, + "card": "Frontier. Big multi-file refactors, hard debugging, images." } + ] +} +``` + +Prove it end to end with no keys and no network: + +```bash +python3 examples/auto_router_demo.py +``` + +It spins up a mock multi-backend server, starts the **real** shim with the router +on, and shows trivial→cheap, medium→mid, hard→strong, image→image-capable, and a +repeat served from cache. Full configuration, env knobs (`CODEX_SHIM_ROUTER_LOG`, +`CODEX_SHIM_DISABLE_ROUTER`, …), and failure behavior are in +[`docs/AUTO_ROUTER.md`](docs/AUTO_ROUTER.md). + +--- + ## Tool calls and agent loops Codex expects Responses-API output items. Most BYOK upstreams speak either @@ -562,12 +723,30 @@ This is the piece that makes the shim useful for real Codex runs instead of only text chat. A model can ask Codex to run tools, Codex sends the tool output back through the shim, and the upstream model continues the same loop. +Native Responses-only tools now have BYOK fallbacks: + +| Responses tool | Chat/Anthropic fallback | +|---|---| +| `computer_use` / `computer_use_preview` | `computer_use` function with `{action, x, y, text, ...}` | +| `web_search` / `web_search_preview` | `web_search` function with `{query, ...}` | +| `apply_patch` | `apply_patch` function with `{patch, ...}` | +| `local_shell` / `shell` | `local_shell` function with `{command, ...}` | +| Codex MCP functions | Passed through as normal function tools | + +That keeps BYOK models inside the Codex agent loop even when the upstream API is +chat-completions or Anthropic Messages instead of native Responses. ChatGPT +passthrough remains the highest-fidelity path for first-party hosted tool item +shapes, but BYOK routes no longer drop those tools. Visual feedback is preserved +for vision-capable BYOK providers: Responses `input_image`, `computer_call_output` +screenshots, and visual `function_call_output` payloads become OpenAI chat +`image_url` parts or Anthropic image blocks instead of being flattened to text. + Known edge cases: -- Native Responses-only tool types such as freeform `apply_patch`, hosted - `web_search`, or `computer_use` are only fully native on the ChatGPT - passthrough path. Chat-completions and Anthropic upstreams receive function - tools after translation. +- BYOK native-tool fallbacks depend on the Codex client/harness recognizing and + executing the fallback function call. The shim translates tool schemas and + round-trips tool outputs; it does not execute computer, shell, patch, or MCP + actions itself. - Some OpenAI-compatible providers advertise tool calls but stream malformed JSON arguments. The shim preserves deltas; the provider still has to emit valid JSON by the end of the call. @@ -576,6 +755,23 @@ Known edge cases: --- +## Compaction + +Codex can compact long sessions through `POST /v1/responses/compact`. + +| route | behavior | +|---|---| +| ChatGPT passthrough (`gpt-5.5` / `openai-gpt-5-5*`) | Forwards to ChatGPT's native `/backend-api/codex/responses/compact` endpoint and rewrites returned model metadata back to the requested shim slug. | +| BYOK OpenAI/chat-completions providers | Sends a non-streaming summarization request through `/chat/completions`, then returns a Responses-shaped compacted window whose `output` can be used as the next `input`. | +| BYOK Anthropic providers | Sends a non-streaming compact request through `/messages`, then returns the same Responses-shaped compacted window. | + +The BYOK path intentionally strips provider-hostile fields such as `stream` and +`service_tier` before forwarding. It preserves the practical Codex behavior — a +smaller next context window — without pretending third-party chat APIs can emit +OpenAI's opaque encrypted compaction items. + +--- + ## Computer use, shell commands, images, and MCP The generated catalog advertises the Codex-facing capabilities Codex needs to @@ -594,13 +790,15 @@ What that means in practice: - **Shell/file operations** are still executed by Codex Desktop/CLI. The shim only translates the model request and response stream. -- **Images/screenshots** can pass to providers that accept images. Set - `noImageSupport: true` for text-only upstreams so Codex does not send image - content they cannot parse. -- **Computer-use/native hosted tools** should use the ChatGPT passthrough path - for best fidelity. BYOK chat/Anthropic routes can still participate in Codex - tool loops, but hosted Responses-only tool item types are not equivalent to - normal function tools. +- **Images/screenshots** can pass to providers that accept images. Responses + `input_image` items, `computer_call_output` screenshots, and visual tool + outputs are preserved as OpenAI chat `image_url` parts or Anthropic image + blocks. Set `noImageSupport: true` for text-only upstreams so Codex does not + send image content they cannot parse. +- **Computer-use/native hosted tools** use native Responses item types on the + ChatGPT passthrough path. BYOK chat/Anthropic routes receive deterministic + function-tool fallbacks (`computer_use`, `web_search`, `apply_patch`, + `local_shell`) so they can stay in the same Codex tool loop. Codex Desktop forwards three generic MCP tools to every model: @@ -722,10 +920,13 @@ codex-shim generate regenerate catalog/config without starting daemon codex-shim start regenerate catalog and start local shim daemon codex-shim enable start daemon and write managed ~/.codex/config.toml block codex-shim status health check + model count +codex-shim doctor read-only local diagnostics report codex-shim stop stop daemon codex-shim disable remove managed config block and stop daemon codex-shim restart stop, regenerate, and start daemon codex-shim list list generated slugs and upstream routes +codex-shim opencode-go refresh + refresh OpenCode Go models into the settings file codex-shim model list list slugs currently usable in the picker codex-shim model use set the Desktop default model in managed config codex-shim codex -- exec `codex` CLI through inline shim overrides @@ -744,17 +945,52 @@ codex-model [list|] shortcut for `codex-shim model …` Global flags: -- `--settings `: used by catalog/model/start/app/codex flows. -- `--port `: used by daemon/provider flows. +- `--settings `: used by catalog/model/start/app/codex/doctor flows. +- `--port `: used by daemon/provider/doctor flows. `patch-app` and `restore-app` always target `/Applications/Codex.app`, do not use `--settings`, and exit with a clear error on Windows/Linux. --- +## Model picker (web UI) + +The shim exposes a small browser UI for switching the active model without +restarting the CLI: + +- `GET /picker` — self-contained HTML page (dark theme) listing every model + the shim currently knows about, with the active one highlighted. +- `GET /api/models` — JSON list backing the picker. +- `POST /api/switch` — `{"slug": "...", "restart_codex": true|false}`. The + shim rewrites `model = "..."` and the `[model_providers.codex_shim]` + `name = "..."` in `~/.codex/config.toml` so the Codex Desktop UI shows + the selected model's display name (e.g. "Kimi K2.6") instead of the + generic "Codex Shim" label, and optionally relaunches Codex Desktop + (`open -a Codex` on macOS, `taskkill` + `Codex.exe` on Windows). This + state-changing picker endpoint requires the per-process + `X-Codex-Shim-Picker-Token` header embedded in `/picker`. + +All picker routes are behind the same `Host`-header allowlist as the rest of +the shim, so a visited web page cannot drive them via DNS rebinding. The +state-changing `/api/switch` endpoint also requires a per-process picker token, +so third-party pages cannot trigger model switches just because the loopback +server is reachable. + +--- + ## Security and privacy - The shim binds to `127.0.0.1` by default. +- The shim validates the `Host` header on every request and rejects anything + that is not a loopback name (`127.0.0.1`, `localhost`, `::1`), the configured + bind host, or an entry in `CODEX_SHIM_ALLOWED_HOSTS`. This blocks DNS-rebinding + attacks where a web page you visit resolves its own domain to `127.0.0.1` and + drives the shim with your credentials. If you deliberately bind to a + non-loopback host, add the host(s) you reach it by to + `CODEX_SHIM_ALLOWED_HOSTS` (comma-separated). +- The model picker protects its state-changing `/api/switch` endpoint with a + per-process picker token, so cross-site pages cannot switch the active model + or request a Desktop restart without loading the picker page. - API keys stay in your settings file; the generated catalog does not contain them. - Request logs are summary-level by default and avoid full prompt/API-key dumps. @@ -785,10 +1021,18 @@ use `--settings`, and exit with a clear error on Windows/Linux. ### Shim will not start ```bash +codex-shim doctor codex-shim status tail -n 80 .codex-shim/shim.log ``` +`codex-shim doctor` prints a read-only diagnostics report grouped by section +(Python, dependencies, Codex CLI, settings, runtime files, daemon health, +passthrough availability, proxy bypass, and Codex config). It never writes +configuration, starts/stops the daemon, calls model providers, or prints API +keys/tokens. It exits 1 only when a hard `FAIL` is detected; warnings are meant +as local setup hints. + Common causes: - Python is older than 3.11. diff --git a/codex_shim/catalog.py b/codex_shim/catalog.py index 7f47e7ad..0a12639b 100644 --- a/codex_shim/catalog.py +++ b/codex_shim/catalog.py @@ -3,7 +3,18 @@ import json from pathlib import Path -from .settings import CHATGPT_MODEL_SLUG, PROVIDER_NAME, ShimModel, chatgpt_passthrough_available, default_model_slug +from . import router as router_module +from .settings import ( + CHATGPT_MODEL_SLUG, + PROVIDER_NAME, + ShimModel, + available_model_slugs, + chatgpt_passthrough_available, + default_model_slug, + load_chatgpt_passthrough_catalog_models, + usable_byok_models, +) +from .cursor_passthrough import cursor_catalog_entry, cursor_passthrough_available PLAN_TIERS = ["free", "plus", "pro", "team", "business", "enterprise"] @@ -61,60 +72,42 @@ def catalog_entry(model: ShimModel) -> dict: } +def chatgpt_passthrough_entries() -> list[dict]: + """Catalog entries for GPT models routed through ChatGPT passthrough.""" + entries: list[dict] = [] + for raw in load_chatgpt_passthrough_catalog_models(): + entry = dict(raw) + entry["visibility"] = "list" + entry.setdefault("available_in_plans", PLAN_TIERS) + entry.setdefault("minimal_client_version", "0.0.1") + entry.setdefault("supported_in_api", True) + if entry.get("slug") == CHATGPT_MODEL_SLUG: + entry["isDefault"] = True + entry["priority"] = max(int(entry.get("priority") or 0), 10000) + entries.append(entry) + return entries + + def chatgpt_passthrough_entry() -> dict: - """Catalog entry for the original GPT-5.5 routed through ChatGPT passthrough.""" - return { - "slug": CHATGPT_MODEL_SLUG, - "display_name": "GPT-5.5", - "description": "OpenAI GPT-5.5 — the default Codex model, routed through ChatGPT passthrough.", - "context_window": 400000, - "max_context_window": 400000, - "auto_compact_token_limit": 320000, - "truncation_policy": {"mode": "tokens", "limit": 64000}, - "default_reasoning_level": "medium", - "supported_reasoning_levels": [ - {"effort": "minimal", "description": "Minimal reasoning"}, - {"effort": "low", "description": "Faster, lighter reasoning"}, - {"effort": "medium", "description": "Balanced"}, - {"effort": "high", "description": "Deeper reasoning"}, - {"effort": "xhigh", "description": "Maximum reasoning"}, - ], - "default_reasoning_summary": "auto", - "reasoning_summary_format": "experimental", - "supports_reasoning_summaries": True, - "default_verbosity": "medium", - "support_verbosity": True, - "apply_patch_tool_type": "freeform", - "web_search_tool_type": "text_and_image", - "supports_search_tool": True, - "supports_parallel_tool_calls": True, - "experimental_supported_tools": [], - "input_modalities": ["text", "image"], - "supports_image_detail_original": True, - "shell_type": "shell_command", - "visibility": "list", - "minimal_client_version": "0.0.1", - "supported_in_api": True, - "availability_nux": None, - "upgrade": None, - "isDefault": True, - "priority": 10000, - "prefer_websockets": False, - "available_in_plans": PLAN_TIERS, - "base_instructions": "You are Codex, a coding agent powered by GPT-5.5.", - "model_messages": { - "instructions_template": "You are Codex, a coding agent powered by GPT-5.5.", - "instructions_variables": {"model_name": "GPT-5.5"}, - }, - } + """Catalog entry for the default GPT-5.5 ChatGPT passthrough model.""" + for entry in chatgpt_passthrough_entries(): + if entry.get("slug") == CHATGPT_MODEL_SLUG: + return entry + return chatgpt_passthrough_entries()[0] -def write_catalog(models: list[ShimModel], path: Path) -> Path: +def write_catalog(models: list[ShimModel], path: Path, router_config=None) -> Path: path.parent.mkdir(parents=True, exist_ok=True) entries: list[dict] = [] + if router_config is not None and router_module.router_is_active(router_config, available_model_slugs(models)): + entries.append(router_module.router_catalog_entry(router_config)) if chatgpt_passthrough_available(): - entries.append(chatgpt_passthrough_entry()) - entries.extend(catalog_entry(model) for model in models) + entries.extend(chatgpt_passthrough_entries()) + if cursor_passthrough_available(): + entry = cursor_catalog_entry() + entry["isDefault"] = not chatgpt_passthrough_available() + entries.append(entry) + entries.extend(catalog_entry(model) for model in usable_byok_models(models)) payload = {"models": entries} path.write_text(json.dumps(payload, indent=2, sort_keys=False) + "\n") return path diff --git a/codex_shim/cli.py b/codex_shim/cli.py index 9a4d6fb0..d7a2ff22 100644 --- a/codex_shim/cli.py +++ b/codex_shim/cli.py @@ -1,29 +1,53 @@ from __future__ import annotations import argparse +from collections import Counter from dataclasses import dataclass import getpass +import importlib.util import os from pathlib import Path import ctypes import signal +import shutil import subprocess import sys import time import hashlib import json +import plistlib +import re +import struct from urllib.request import urlopen +from . import router as router_module from .catalog import _toml_escape, codex_config_overrides, write_catalog, write_config +from .cursor_passthrough import ( + cursor_passthrough_available, + cursor_passthrough_display_names, + is_cursor_passthrough_slug, +) from .settings import ( CHATGPT_MODEL_SLUG, DEFAULT_SETTINGS, DEFAULT_HOST, DEFAULT_PORT, + DEFAULT_CODEX_AUTH, PROVIDER_NAME, ModelSettings, + available_model_slugs, chatgpt_passthrough_available, + chatgpt_passthrough_display_names, + chatgpt_passthrough_slugs, default_model_slug, + is_chatgpt_passthrough_slug, + usable_byok_models, + byok_model_has_credentials, +) +from .opencode_go import ( + OPENCODE_GO_API_KEY_ENV, + OPENCODE_GO_BASE_URL, + refresh_opencode_go_settings, ) @@ -42,6 +66,32 @@ WINDOWS_STILL_ACTIVE = 259 PREVIOUS_TOP_LEVEL_PREFIX = "# codex-shim previous-top-level = " MANAGED_TOP_LEVEL_KEYS = {"model", "model_provider", "model_catalog_json"} +APP_ASAR_BACKUP_NAME = "app.asar.before-codex-shim-model-picker-patch" +INFO_PLIST_BACKUP_NAME = "Info.plist.before-codex-shim-model-picker-patch" +SYSTEM_CODEX_APP = Path("/Applications/Codex.app") +USER_CODEX_APP = Path.home() / "Applications" / "Codex.app" +MODEL_PICKER_NEEDLE = re.compile( + r"(?P(?:let )?\w+=)" + r"(?:\w+\.useHiddenModels|\w+)" + r"&&\w+!==`amazonBedrock`" + r"(?P[,;])" +) +MODEL_PICKER_REPLACEMENT = r"\g!1\g" +MODEL_PICKER_APPLIED = re.compile( + r"(?:let )?\w+=!1[,;][^\n]{0,300}\.forEach" +) + +SIDEBAR_RECENT_THREADS_NEEDLE = re.compile( + r"listRecentThreads\(\{cursor:e,limit:t(?:,useStateDbOnly:\w+(?:=!\d)?)?\}\)\{return this\.params\.requestClient\.sendRequest\(`thread/list`," + r"\{limit:t,cursor:e,sortKey:this\.recentConversationSortKey,modelProviders:null,archived:!1,sourceKinds:(\w+)(?:,useStateDbOnly:\w+)?\}\)\}" +) +SIDEBAR_RECENT_THREADS_REPLACEMENT = ( + r"listRecentThreads({cursor:e,limit:t}){return this.params.requestClient.sendRequest(`thread/list`," + r"{limit:t,cursor:e,sortKey:this.recentConversationSortKey,modelProviders:[],archived:!1,sourceKinds:\1})}" +) +SIDEBAR_RECENT_THREADS_APPLIED = re.compile( + r"\.recentConversationSortKey,modelProviders:\[\],archived:!1,sourceKinds:\w+" +) @dataclass(frozen=True) @@ -105,9 +155,18 @@ def main(argv: list[str] | None = None) -> int: sub.add_parser("disable") sub.add_parser("restart") sub.add_parser("status") - sub.add_parser("patch-app", help="Patch Codex Desktop model dropdown to allow custom catalog models.") + sub.add_parser("doctor", help="Print a read-only local diagnostics report.") + sub.add_parser("patch-app", help="Patch Codex Desktop picker/sidebar handling for custom shim models.") sub.add_parser("restore-app", help="Restore Codex Desktop app.asar from the pre-patch backup.") + opencode_parser = sub.add_parser("opencode-go", help="Discover and configure OpenCode Go models.") + opencode_sub = opencode_parser.add_subparsers(dest="opencode_go_command", required=True) + refresh_parser = opencode_sub.add_parser("refresh", help="Refresh OpenCode Go models into the settings file.") + refresh_parser.add_argument("--api-key-env", default=OPENCODE_GO_API_KEY_ENV) + refresh_parser.add_argument("--base-url", default=OPENCODE_GO_BASE_URL) + refresh_parser.add_argument("--prefer", choices=["chat", "messages"], default="chat") + refresh_parser.add_argument("--timeout", type=float, default=30.0) + model_parser = sub.add_parser("model", help="List or set the active shim model in Codex config.") model_sub = model_parser.add_subparsers(dest="model_command", required=True) model_sub.add_parser("list") @@ -159,10 +218,15 @@ def main(argv: list[str] | None = None) -> int: return start(args.settings, port) if args.command == "status": return status(port) + if args.command == "doctor": + return doctor(args.settings, port) if args.command == "patch-app": return patch_codex_app() if args.command == "restore-app": return restore_codex_app_bundle() + if args.command == "opencode-go": + if args.opencode_go_command == "refresh": + return refresh_opencode_go(args.settings, args.api_key_env, args.base_url, args.prefer, args.timeout) if args.command == "model": if args.model_command == "list": return list_models(args.settings) @@ -388,23 +452,376 @@ def _load_models(settings_path: Path): raise SystemExit(f"Settings file is not valid JSON: {expanded}: {exc}") from exc +def _active_router(models, settings_path: Path): + """RouterConfig when the Auto Router is enabled and has a usable candidate.""" + config = router_module.load_router_config(Path(settings_path).expanduser()) + if config and router_module.router_is_active(config, available_model_slugs(models)): + return config + return None + + +@dataclass(frozen=True) +class DoctorCheck: + section: str + status: str # OK | WARN | FAIL | INFO + message: str + detail: str = "" + + +def doctor(settings_path: Path, port: int) -> int: + """Print a read-only diagnostics report for the local codex-shim setup.""" + expanded = Path(settings_path).expanduser() + checks: list[DoctorCheck] = [] + checks.extend(_doctor_python()) + checks.extend(_doctor_dependencies()) + checks.extend(_doctor_codex_cli()) + checks.extend(_doctor_settings(expanded)) + checks.extend(_doctor_runtime_files()) + checks.extend(_doctor_daemon(port)) + checks.extend(_doctor_chatgpt()) + checks.extend(_doctor_cursor()) + checks.extend(_doctor_proxy_env()) + checks.extend(_doctor_codex_config()) + _print_doctor_report(checks) + return 1 if any(check.status == "FAIL" for check in checks) else 0 + + +def _doctor_python() -> list[DoctorCheck]: + version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + status = "OK" if sys.version_info >= (3, 11) else "FAIL" + detail = "" if status == "OK" else "codex-shim requires Python 3.11+" + return [ + DoctorCheck("Python", status, f"version: {version}", detail), + DoctorCheck("Python", "OK", f"executable: {sys.executable}"), + ] + + +def _doctor_dependencies() -> list[DoctorCheck]: + if importlib.util.find_spec("aiohttp") is None: + return [ + DoctorCheck( + "Dependencies", + "FAIL", + "aiohttp is not importable", + "Try: python3 -m pip install -e .", + ) + ] + return [DoctorCheck("Dependencies", "OK", "aiohttp importable")] + + +def _doctor_codex_cli() -> list[DoctorCheck]: + found = shutil.which("codex") + if not found: + return [ + DoctorCheck( + "Codex CLI", + "WARN", + "codex not found on PATH", + "Install and authenticate Codex before using codex-shim app/codex flows.", + ) + ] + checks = [DoctorCheck("Codex CLI", "OK", f"found: {found}")] + try: + result = subprocess.run([found, "--version"], capture_output=True, text=True, timeout=5) + except (OSError, subprocess.TimeoutExpired) as exc: + checks.append(DoctorCheck("Codex CLI", "WARN", "could not run codex --version", str(exc))) + return checks + output = (result.stdout or result.stderr).strip().splitlines() + version = output[0].strip() if output else "unknown" + if len(version) > 200: + version = version[:197] + "..." + if result.returncode == 0: + checks.append(DoctorCheck("Codex CLI", "OK", f"version: {version}")) + else: + checks.append(DoctorCheck("Codex CLI", "WARN", "codex --version failed", version)) + return checks + + +def _doctor_settings(settings_path: Path) -> list[DoctorCheck]: + section = "Settings" + path = settings_path.expanduser() + if not path.exists(): + detail = "Create ~/.codex-shim/models.json or run codex login for ChatGPT passthrough-only use." + return [DoctorCheck(section, "WARN", f"settings file not found: {path}", detail)] + checks = [DoctorCheck(section, "OK", f"path: {path}")] + try: + models = _load_models(path) + except SystemExit as exc: + message = str(exc) + if "not valid JSON" in message or "invalid JSON" in message: + return [DoctorCheck(section, "FAIL", f"invalid JSON: {path}", message)] + return [DoctorCheck(section, "FAIL", f"could not load settings: {path}", message)] + except Exception as exc: + return [DoctorCheck(section, "FAIL", f"could not load settings: {path}", str(exc))] + + usable = usable_byok_models(models) + missing_count = len(models) - len(usable) + checks.append(DoctorCheck(section, "OK", f"configured models: {len(models)}")) + checks.append(DoctorCheck(section, "OK", f"usable BYOK models: {len(usable)}")) + if missing_count: + checks.append(DoctorCheck(section, "WARN", f"models missing API keys: {missing_count}")) + else: + checks.append(DoctorCheck(section, "OK", "models missing API keys: 0")) + providers = Counter(model.provider for model in models) + provider_text = ", ".join(f"{provider}={count}" for provider, count in sorted(providers.items())) or "none" + checks.append(DoctorCheck(section, "INFO", f"providers: {provider_text}")) + + router_config = router_module.load_router_config(path) + if router_config is None: + checks.append(DoctorCheck(section, "INFO", "auto router configured: false")) + else: + active = _active_router(models, path) + if active is not None: + checks.append(DoctorCheck(section, "OK", f"auto router active: {active.slug}")) + elif router_config.effective_enabled: + checks.append( + DoctorCheck( + section, + "WARN", + f"auto router configured but inactive: {router_config.slug}", + "Ensure at least one router candidate matches a usable model slug.", + ) + ) + else: + checks.append(DoctorCheck(section, "INFO", f"auto router configured but disabled: {router_config.slug}")) + return checks + + +def _doctor_runtime_files() -> list[DoctorCheck]: + checks: list[DoctorCheck] = [] + if CATALOG_PATH.exists(): + checks.append(DoctorCheck("Runtime files", "OK", f"catalog: {CATALOG_PATH}")) + try: + data = json.loads(CATALOG_PATH.read_text()) + models = data.get("models", []) if isinstance(data, dict) else [] + count = len(models) if isinstance(models, list) else 0 + checks.append(DoctorCheck("Runtime files", "OK", f"catalog models: {count}")) + except (OSError, json.JSONDecodeError) as exc: + checks.append( + DoctorCheck("Runtime files", "WARN", f"catalog JSON is not readable: {CATALOG_PATH}", str(exc)) + ) + else: + checks.append(DoctorCheck("Runtime files", "INFO", f"catalog missing: {CATALOG_PATH}")) + if CONFIG_PATH.exists(): + checks.append(DoctorCheck("Runtime files", "OK", f"config: {CONFIG_PATH}")) + else: + checks.append(DoctorCheck("Runtime files", "INFO", f"config missing: {CONFIG_PATH}")) + if PID_PATH.exists(): + checks.append(DoctorCheck("Runtime files", "INFO", f"pid file: {PID_PATH}")) + else: + checks.append(DoctorCheck("Runtime files", "INFO", f"pid file missing: {PID_PATH}")) + if LOG_PATH.exists(): + checks.append(DoctorCheck("Runtime files", "INFO", f"log file: {LOG_PATH}")) + else: + checks.append(DoctorCheck("Runtime files", "INFO", f"log file missing: {LOG_PATH}")) + return checks + + +def _doctor_daemon(port: int) -> list[DoctorCheck]: + checks = [DoctorCheck("Shim daemon", "INFO", f"health URL: http://{DEFAULT_HOST}:{port}/health")] + pid = _read_pid() + if pid is None: + checks.append(DoctorCheck("Shim daemon", "INFO", f"pid file missing or unreadable: {PID_PATH}")) + elif _pid_running(pid): + checks.append(DoctorCheck("Shim daemon", "OK", f"pid {pid} is running")) + else: + checks.append(DoctorCheck("Shim daemon", "WARN", f"pid {pid} is not running")) + + health = _health(port) + if health is None: + checks.append(DoctorCheck("Shim daemon", "WARN", "health endpoint unavailable")) + return checks + model_count = _health_model_count(health.get("models")) + if health.get("ok") is True: + checks.append(DoctorCheck("Shim daemon", "OK", f"health ok: {model_count} models")) + else: + checks.append(DoctorCheck("Shim daemon", "WARN", f"health not ok: {model_count} models")) + for key in ("chatgpt_passthrough", "cursor_passthrough", "auto_router"): + if key in health: + checks.append(DoctorCheck("Shim daemon", "INFO", f"{key}: {_bool_text(health.get(key))}")) + return checks + + +def _health_model_count(value) -> int: + if isinstance(value, int): + return value + if isinstance(value, list): + return len(value) + return 0 + + +def _doctor_chatgpt() -> list[DoctorCheck]: + if _env_flag("CODEX_SHIM_DISABLE_CHATGPT"): + return [DoctorCheck("ChatGPT passthrough", "INFO", "disabled via CODEX_SHIM_DISABLE_CHATGPT")] + auth_path = Path(DEFAULT_CODEX_AUTH).expanduser() + if chatgpt_passthrough_available(): + return [DoctorCheck("ChatGPT passthrough", "OK", f"available via {auth_path}")] + if auth_path.exists(): + detail = "Run `codex login` again if you want ChatGPT/Codex passthrough." + else: + detail = "Run `codex login` if you want ChatGPT/Codex passthrough." + return [DoctorCheck("ChatGPT passthrough", "WARN", "unavailable", detail)] + + +def _doctor_cursor() -> list[DoctorCheck]: + if _env_flag("CODEX_SHIM_DISABLE_CURSOR"): + return [DoctorCheck("Cursor passthrough", "INFO", "disabled via CODEX_SHIM_DISABLE_CURSOR")] + bin_override = os.environ.get("CURSOR_AGENT_BIN", "").strip() + agent_bin = bin_override or shutil.which("cursor-agent") + checks: list[DoctorCheck] = [] + if agent_bin: + checks.append(DoctorCheck("Cursor passthrough", "INFO", f"cursor-agent: {agent_bin}")) + else: + checks.append(DoctorCheck("Cursor passthrough", "WARN", "cursor-agent not found on PATH")) + if cursor_passthrough_available(): + checks.append(DoctorCheck("Cursor passthrough", "OK", "cursor-agent logged in")) + for slug in sorted(cursor_passthrough_display_names()): + checks.append(DoctorCheck("Cursor passthrough", "INFO", f"exposed model: {slug}")) + else: + checks.append( + DoctorCheck( + "Cursor passthrough", + "WARN", + "unavailable", + "Run `cursor-agent login` if you want Cursor passthrough.", + ) + ) + return checks + + +def _doctor_proxy_env() -> list[DoctorCheck]: + required = {"127.0.0.1", "localhost", "::1"} + values: set[str] = set() + for key in ("NO_PROXY", "no_proxy"): + raw = os.environ.get(key, "") + for part in raw.split(","): + value = part.strip().lower() + if value: + values.add(value) + if "*" in values or required <= values: + return [DoctorCheck("Proxy", "OK", "loopback hosts covered by NO_PROXY/no_proxy")] + return [ + DoctorCheck( + "Proxy", + "WARN", + "NO_PROXY/no_proxy does not include all loopback hosts", + "Recommended: 127.0.0.1,localhost,::1", + ) + ] + + +def _doctor_codex_config() -> list[DoctorCheck]: + path = Path(CODEX_CONFIG_PATH).expanduser() + if not path.exists(): + return [ + DoctorCheck( + "Codex config", + "INFO", + "shim provider is not currently installed", + "Run `codex-shim app .` or `codex-shim enable` to wire Codex to the shim.", + ) + ] + checks = [DoctorCheck("Codex config", "OK", f"config exists: {path}")] + try: + text = path.read_text() + except OSError as exc: + return [DoctorCheck("Codex config", "WARN", f"could not read config: {path}", str(exc))] + provider_configured = ( + f'model_provider = "{PROVIDER_NAME}"' in text or f"[model_providers.{PROVIDER_NAME}]" in text + ) + if provider_configured: + checks.append(DoctorCheck("Codex config", "OK", "shim provider configured")) + else: + checks.append( + DoctorCheck( + "Codex config", + "INFO", + "shim provider is not currently installed", + "Run `codex-shim app .` or `codex-shim enable` to wire Codex to the shim.", + ) + ) + current = _current_managed_model() + if current: + checks.append(DoctorCheck("Codex config", "OK", f"active shim model: {current}")) + else: + checks.append(DoctorCheck("Codex config", "INFO", "active shim model: none")) + return checks + + +def _print_doctor_report(checks: list[DoctorCheck]) -> None: + current_section = None + for check in checks: + if check.section != current_section: + if current_section is not None: + print() + print(check.section) + current_section = check.section + print(f" {check.status:<5} {check.message}") + if check.detail: + for line in check.detail.splitlines(): + print(f" {line}") + counts = Counter(check.status for check in checks) + summary_status = "FAIL" if counts["FAIL"] else "OK" + print() + print("Summary") + print( + f" {summary_status:<5} " + f"{counts['OK']} ok, {counts['WARN']} warn, {counts['FAIL']} fail, {counts['INFO']} info" + ) + + +def _env_flag(name: str) -> bool: + return os.environ.get(name, "").strip().lower() in {"1", "true", "yes", "on"} + + +def _bool_text(value) -> str: + return "true" if bool(value) else "false" + + def generate(settings_path: Path, port: int) -> None: models = _load_models(settings_path) try: default_model_slug(models) except ValueError as exc: raise SystemExit(str(exc)) from exc - write_catalog(models, CATALOG_PATH) + router_config = router_module.load_router_config(Path(settings_path).expanduser()) + write_catalog(models, CATALOG_PATH, router_config=router_config) write_config(models, CONFIG_PATH, CATALOG_PATH, port) print(f"Generated {len(models)} model entries:") + if _active_router(models, settings_path) is not None: + print(f" auto router: {router_config.slug} ({router_config.display_name})") print(f" catalog: {CATALOG_PATH}") print(f" config: {CONFIG_PATH}") print("No files under ~/.codex were modified.") +def refresh_opencode_go(settings_path: Path, api_key_env: str, base_url: str, prefer: str, timeout: float) -> int: + print(f"Refreshing OpenCode Go models from {base_url}...") + try: + result = refresh_opencode_go_settings( + settings_path, + api_key_env=api_key_env, + base_url=base_url, + prefer=prefer, + timeout=timeout, + ) + except RuntimeError as exc: + print(str(exc), file=sys.stderr) + return 1 + print(f"Refreshed {len(result.models)} OpenCode Go models into {result.settings_path}.") + if result.skipped: + print(f"Skipped {len(result.skipped)} models with no working probed endpoint:") + for model_id, chat_status, messages_status in result.skipped: + print(f" {model_id}: chat={chat_status}, messages={messages_status}") + for row in result.models: + print(f" {row['slug']} -> {row['model']} ({row['provider']}, {row['opencode_go_endpoint']})") + return 0 + + def install_codex_config(settings_path: Path, port: int, model_slug: str | None = None) -> None: models = _load_models(settings_path) - default_slug = _resolve_model_slug(models, model_slug) + router_config = _active_router(models, settings_path) + default_slug = _resolve_model_slug(models, model_slug, router_config) CODEX_CONFIG_PATH.parent.mkdir(parents=True, exist_ok=True) RUNTIME_DIR.mkdir(parents=True, exist_ok=True) original = CODEX_CONFIG_PATH.read_text() if CODEX_CONFIG_PATH.exists() else "" @@ -418,7 +835,10 @@ def install_codex_config(settings_path: Path, port: int, model_slug: str | None previous_top_level = _extract_top_level_key_lines(CODEX_CONFIG_BACKUP_PATH.read_text(), MANAGED_TOP_LEVEL_KEYS) cleaned = _remove_top_level_keys(cleaned, MANAGED_TOP_LEVEL_KEYS) cleaned = _remove_section(cleaned, f"model_providers.{PROVIDER_NAME}") - top_block, provider_block = _managed_config_blocks(default_slug, port, previous_top_level) + provider_name = _provider_display_name(models, default_slug, router_config) + top_block, provider_block = _managed_config_blocks( + default_slug, port, previous_top_level, provider_name=provider_name + ) CODEX_CONFIG_PATH.write_text(top_block + "\n" + cleaned.lstrip() + "\n" + provider_block) print(f"Installed shim config into {CODEX_CONFIG_PATH}.") @@ -426,13 +846,23 @@ def install_codex_config(settings_path: Path, port: int, model_slug: str | None def list_models(settings_path: Path) -> int: models = _load_models(settings_path) rows: list[tuple[str, str, str, str]] = [] + router_config = _active_router(models, settings_path) + if router_config is not None: + rows.append((router_config.slug, router_config.display_name, "per-task pick", "auto")) if chatgpt_passthrough_available(): - rows.append(("gpt-5.5", "GPT-5.5", "gpt-5.5", "chatgpt")) - rows.extend((model.slug, model.display_name, model.model, model.provider) for model in models) + for slug, display_name in chatgpt_passthrough_display_names().items(): + rows.append((slug, display_name, slug, "chatgpt")) + if cursor_passthrough_available(): + for slug, display_name in cursor_passthrough_display_names().items(): + rows.append((slug, display_name, "composer-2.5", "cursor-subscription")) + rows.extend((model.slug, model.display_name, model.model, model.provider) for model in usable_byok_models(models)) + for model in models: + if model not in usable_byok_models(models): + rows.append((model.slug, f"{model.display_name} (missing API key)", model.model, model.provider)) if not rows: print( "No models available. Create ~/.codex-shim/models.json, pass --settings /path/to/models.json, " - "or run `codex login` so ~/.codex/auth.json grants the gpt-5.5 passthrough.", + "run `codex login` for GPT passthrough, or run `cursor-agent login` for Composer passthrough.", file=sys.stderr, ) return 1 @@ -545,8 +975,12 @@ def exec_codex(settings_path: Path, port: int, codex_args: list[str]) -> None: def exec_codex_app(settings_path: Path, port: int, path: str) -> None: _quit_codex_app() - args = ["codex", "app", path] - subprocess.Popen(args, env=_with_loopback_no_proxy(os.environ.copy())) + codex_app = patched_codex_app_bundle() + if codex_app is not None: + subprocess.Popen(["open", "-a", str(codex_app)], env=_with_loopback_no_proxy(os.environ.copy())) + else: + args = ["codex", "app", path] + subprocess.Popen(args, env=_with_loopback_no_proxy(os.environ.copy())) _foreground_codex_app() @@ -575,14 +1009,19 @@ def patch_codex_app() -> int: if sys.platform != "darwin": print("patch-app is macOS-only; Windows MSIX Codex Desktop cannot be patched with this ASAR helper.", file=sys.stderr) return 1 - app_asar = Path("/Applications/Codex.app/Contents/Resources/app.asar") - backup = RUNTIME_DIR / "app.asar.before-codex-shim-model-picker-patch" - workdir = RUNTIME_DIR / "app-asar-work" - needle = "let u=c.useHiddenModels&&o!==`amazonBedrock`,d;" - replacement = "let u=!1,d;" + codex_app = _codex_app_bundle_for_patch() + app_asar = codex_app / "Contents/Resources/app.asar" + info_plist = codex_app / "Contents/Info.plist" + backup = RUNTIME_DIR / APP_ASAR_BACKUP_NAME + info_backup = RUNTIME_DIR / INFO_PLIST_BACKUP_NAME if not app_asar.exists(): - print(f"Codex app bundle not found at {app_asar}.", file=sys.stderr) + print(f"Codex app bundle not found at {codex_app}.", file=sys.stderr) + return 1 + if codex_app == USER_CODEX_APP: + print(f"Patching user Codex copy at {codex_app}.") + if not info_plist.exists(): + print(f"Codex Info.plist not found at {info_plist}.", file=sys.stderr) return 1 if not _has_command("npx"): print("npx is required to patch the Electron asar bundle.", file=sys.stderr) @@ -596,8 +1035,12 @@ def patch_codex_app() -> int: if not versioned_backup.exists(): versioned_backup.write_bytes(app_asar.read_bytes()) print(f"Backed up current app.asar to {versioned_backup}.") + if not info_backup.exists(): + info_backup.write_bytes(info_plist.read_bytes()) + print(f"Backed up original Info.plist to {info_backup}.") _quit_codex_app() + workdir = RUNTIME_DIR / "app-asar-work-user" if workdir.exists(): import shutil @@ -605,24 +1048,13 @@ def patch_codex_app() -> int: workdir.mkdir(parents=True) subprocess.run(["npx", "--yes", "asar", "extract", str(app_asar), str(workdir)], check=True) - bundle_file = _find_model_queries_bundle(workdir, needle, replacement) - if bundle_file is None: - print("Could not find the expected model picker filter in Codex Desktop.", file=sys.stderr) - return 1 - text = bundle_file.read_text() - changed = False - if replacement in text: - print("Codex Desktop model picker patch is already applied.") - elif needle in text: - bundle_file.write_text(text.replace(needle, replacement)) - subprocess.run(["npx", "--yes", "asar", "pack", str(workdir), str(app_asar)], check=True) - changed = True - print("Patched Codex Desktop model picker allowlist filter.") - else: - print("Could not find the expected model picker filter in Codex Desktop.", file=sys.stderr) + changed = _patch_codex_desktop_bundles(workdir) + if changed is None: return 1 if changed: - _resign_codex_app() + subprocess.run(["npx", "--yes", "asar", "pack", str(workdir), str(app_asar)], check=True) + _update_app_asar_integrity(app_asar, info_plist) + _resign_codex_app(codex_app) return 0 @@ -630,13 +1062,22 @@ def restore_codex_app_bundle() -> int: if sys.platform != "darwin": print("restore-app is macOS-only; Windows MSIX Codex Desktop cannot be restored with this ASAR helper.", file=sys.stderr) return 1 - app_asar = Path("/Applications/Codex.app/Contents/Resources/app.asar") - backup = RUNTIME_DIR / "app.asar.before-codex-shim-model-picker-patch" + codex_app = patched_codex_app_bundle() or _codex_app_bundle_for_patch() + app_asar = codex_app / "Contents/Resources/app.asar" + info_plist = codex_app / "Contents/Info.plist" + backup = RUNTIME_DIR / APP_ASAR_BACKUP_NAME + info_backup = RUNTIME_DIR / INFO_PLIST_BACKUP_NAME if not backup.exists(): print(f"No app.asar backup found at {backup}.") return 0 _quit_codex_app() app_asar.write_bytes(backup.read_bytes()) + if info_backup.exists(): + info_plist.write_bytes(info_backup.read_bytes()) + print(f"Restored {info_plist} from {info_backup}.") + elif info_plist.exists(): + _update_app_asar_integrity(app_asar, info_plist) + _resign_codex_app(codex_app) print(f"Restored {app_asar} from {backup}.") return 0 @@ -655,31 +1096,160 @@ def _app_asar_hash(path: Path) -> str: return h.hexdigest() -def _find_model_queries_bundle(workdir: Path, needle: str, replacement: str) -> Path | None: +def _app_asar_header_hash(path: Path) -> str: + with path.open("rb") as f: + _, _, _, json_size = struct.unpack("<4I", f.read(16)) + header_json = f.read(json_size) + return hashlib.sha256(header_json).hexdigest() + + +def _update_app_asar_integrity(app_asar: Path, info_plist: Path) -> None: + header_hash = _app_asar_header_hash(app_asar) + data = plistlib.loads(info_plist.read_bytes()) + try: + data["ElectronAsarIntegrity"]["Resources/app.asar"]["hash"] = header_hash + except KeyError as exc: + raise RuntimeError(f"Could not update ElectronAsarIntegrity in {info_plist}") from exc + info_plist.write_bytes(plistlib.dumps(data)) + print("Updated ElectronAsarIntegrity for app.asar.") + + +def _patch_codex_desktop_bundles(workdir: Path) -> bool | None: + patches = [ + ( + "model picker allowlist filter", + [ + "models-and-reasoning-efforts-*.js", + "model-queries-*.js", + "*.js", + ], + MODEL_PICKER_NEEDLE, + MODEL_PICKER_REPLACEMENT, + MODEL_PICKER_APPLIED, + ), + ( + "shim-mode sidebar provider filter", + ["app-server-manager-signals-*.js", "*.js"], + SIDEBAR_RECENT_THREADS_NEEDLE, + SIDEBAR_RECENT_THREADS_REPLACEMENT, + SIDEBAR_RECENT_THREADS_APPLIED, + ), + ] + changed = False + for label, globs, needle, replacement, applied in patches: + bundle_file = _find_js_bundle(workdir, globs, needle, applied) + if bundle_file is None: + print(f"Could not find the expected {label} in Codex Desktop.", file=sys.stderr) + return None + result = _replace_once(bundle_file, needle, replacement, applied) + if result is None: + print(f"Could not patch the expected {label} in Codex Desktop.", file=sys.stderr) + return None + if result: + changed = True + print(f"Patched Codex Desktop {label}.") + else: + print(f"Codex Desktop {label} patch is already applied.") + return changed + + +def _find_js_bundle( + workdir: Path, + globs: list[str], + needle: re.Pattern[str], + applied: re.Pattern[str], +) -> Path | None: assets_dir = workdir / "webview" / "assets" if not assets_dir.exists(): return None - candidates = sorted(assets_dir.glob("model-queries-*.js")) - candidates.extend(p for p in sorted(assets_dir.glob("*.js")) if p not in candidates) + candidates: list[Path] = [] + for pattern in globs: + candidates.extend(p for p in sorted(assets_dir.glob(pattern)) if p not in candidates) for path in candidates: - try: - text = path.read_text() - except UnicodeDecodeError: - text = path.read_text(errors="ignore") - if needle in text or replacement in text: + text = _read_text_lossy(path) + if needle.search(text) or applied.search(text): return path return None -def _resign_codex_app() -> None: +def _replace_once( + path: Path, + needle: re.Pattern[str], + replacement: str, + applied: re.Pattern[str], +) -> bool | None: + text = _read_text_lossy(path) + matches = needle.findall(text) + if not matches: + if applied.search(text): + return False + return None + if len(matches) != 1: + return None + path.write_text(needle.sub(replacement, text, count=1)) + return True + + +def _read_text_lossy(path: Path) -> str: + try: + return path.read_text() + except UnicodeDecodeError: + return path.read_text(errors="ignore") + + +def patched_codex_app_bundle() -> Path | None: + for codex_app in (USER_CODEX_APP, SYSTEM_CODEX_APP): + app_asar = codex_app / "Contents/Resources/app.asar" + if app_asar.exists() and _app_asar_is_patched(app_asar): + return codex_app + return None + + +def _codex_app_bundle_for_patch() -> Path: + system_asar = SYSTEM_CODEX_APP / "Contents/Resources/app.asar" + if system_asar.exists() and _path_is_writable(system_asar): + return SYSTEM_CODEX_APP + return _ensure_user_codex_app() + + +def _ensure_user_codex_app() -> Path: + user_asar = USER_CODEX_APP / "Contents/Resources/app.asar" + if user_asar.exists(): + return USER_CODEX_APP + system_asar = SYSTEM_CODEX_APP / "Contents/Resources/app.asar" + if not system_asar.exists(): + raise SystemExit(f"Codex Desktop not found at {SYSTEM_CODEX_APP}.") + USER_CODEX_APP.parent.mkdir(parents=True, exist_ok=True) + subprocess.run(["ditto", str(SYSTEM_CODEX_APP), str(USER_CODEX_APP)], check=True) + print(f"Copied Codex Desktop to {USER_CODEX_APP} for patching.") + return USER_CODEX_APP + + +def _path_is_writable(path: Path) -> bool: + try: + with path.open("r+b"): + return True + except OSError: + return False + + +def _app_asar_is_patched(app_asar: Path) -> bool: + try: + text = app_asar.read_bytes().decode("utf-8", errors="ignore") + except OSError: + return False + return MODEL_PICKER_APPLIED.search(text) is not None and SIDEBAR_RECENT_THREADS_APPLIED.search(text) is not None + + +def _resign_codex_app(codex_app: Path = SYSTEM_CODEX_APP) -> None: # Electron validates app.asar through the bundle signature metadata at # startup. Re-sign after patching so the modified archive does not trip the # asar integrity check. subprocess.run( - ["codesign", "--force", "--deep", "--sign", "-", "/Applications/Codex.app"], + ["codesign", "--force", "--deep", "--sign", "-", str(codex_app)], check=True, ) - print("Re-signed Codex.app after patch.") + print(f"Re-signed {codex_app} after patch.") def _foreground_codex_app() -> None: @@ -708,7 +1278,29 @@ def _foreground_codex_app() -> None: pass -def _managed_config_blocks(default_slug: str, port: int, previous_top_level: dict[str, str] | None = None) -> tuple[str, str]: +def _provider_display_name(models, slug: str, router_config=None) -> str: + if router_config is not None and slug == router_config.slug: + return router_config.display_name + if chatgpt_passthrough_available(): + display_name = chatgpt_passthrough_display_names().get(slug) + if display_name: + return display_name + if cursor_passthrough_available(): + display_name = cursor_passthrough_display_names().get(slug) + if display_name: + return display_name + for model in models: + if model.slug == slug: + return model.display_name + return "Codex Shim" + + +def _managed_config_blocks( + default_slug: str, + port: int, + previous_top_level: dict[str, str] | None = None, + provider_name: str = "Codex Shim", +) -> tuple[str, str]: metadata = "" if previous_top_level: metadata = PREVIOUS_TOP_LEVEL_PREFIX + json.dumps(previous_top_level, sort_keys=True) + "\n" @@ -721,7 +1313,7 @@ def _managed_config_blocks(default_slug: str, port: int, previous_top_level: dic provider_block = f'''{MANAGED_BEGIN} [model_providers.{PROVIDER_NAME}] -name = "Codex Shim" +name = "{_toml_escape(provider_name)}" base_url = "http://127.0.0.1:{port}/v1" wire_api = "responses" experimental_bearer_token = "dummy" @@ -860,28 +1452,51 @@ def _override_args(settings_path: Path, port: int) -> list[str]: return args -def _resolve_model_slug(models, requested: str | None) -> str: +def _resolve_model_slug(models, requested: str | None, router_config=None) -> str: if requested is None: current = _current_managed_model() - if current in _valid_model_slugs(models): + if current in _valid_model_slugs(models, router_config): return current try: return default_model_slug(models) except ValueError as exc: raise SystemExit(str(exc)) from exc - if requested in {CHATGPT_MODEL_SLUG, "openai-gpt-5-5"}: + if router_config is not None and requested == router_config.slug: + return requested + if is_chatgpt_passthrough_slug(requested): if not chatgpt_passthrough_available(): raise SystemExit( - "gpt-5.5 passthrough requires a Codex login. " + "ChatGPT passthrough requires a Codex login. " "Run `codex login` so ~/.codex/auth.json contains tokens.access_token." ) - return CHATGPT_MODEL_SLUG + if requested.startswith("openai-gpt-"): + return CHATGPT_MODEL_SLUG + return requested + if is_cursor_passthrough_slug(requested): + if not cursor_passthrough_available(): + raise SystemExit( + "Composer passthrough requires Cursor CLI login. " + "Run `cursor-agent login`, then `cursor-agent status`." + ) + return requested if requested in cursor_passthrough_display_names() else "composer-2-5" by_slug = {model.slug: model.slug for model in models} - by_model = {} + by_model: dict[str, list[str]] = {} for model in models: by_model.setdefault(model.model, []).append(model.slug) if requested in by_slug: return requested + configured = {model.slug: model for model in models} + if requested in configured and not byok_model_has_credentials(configured[requested]): + if is_cursor_passthrough_slug(requested): + raise SystemExit( + f"Model {requested!r} is configured for BYOK but has no API key. " + "Remove it from ~/.codex-shim/models.json to use Cursor subscription passthrough, " + "or set CURSOR_API_KEY / ~/.codex-shim/cursor-api-key." + ) + raise SystemExit( + f"Model {requested!r} is configured but has no API key. " + "Set the provider API key in ~/.codex-shim/models.json or the matching env var." + ) if requested in by_model and len(by_model[requested]) == 1: return by_model[requested][0] matches = [model.slug for model in models if requested.lower() in model.display_name.lower()] @@ -909,10 +1524,14 @@ def _current_managed_model() -> str | None: return None -def _valid_model_slugs(models) -> set[str]: - slugs = {model.slug for model in models} +def _valid_model_slugs(models, router_config=None) -> set[str]: + slugs = {model.slug for model in usable_byok_models(models)} + if router_config is not None: + slugs.add(router_config.slug) if chatgpt_passthrough_available(): - slugs.add(CHATGPT_MODEL_SLUG) + slugs.update(chatgpt_passthrough_slugs()) + if cursor_passthrough_available(): + slugs.update(cursor_passthrough_display_names()) return slugs diff --git a/codex_shim/cursor_passthrough.py b/codex_shim/cursor_passthrough.py new file mode 100644 index 00000000..7da80c4b --- /dev/null +++ b/codex_shim/cursor_passthrough.py @@ -0,0 +1,361 @@ +from __future__ import annotations + +import asyncio +import json +import os +import shutil +import subprocess +import time +from collections.abc import AsyncIterator +from typing import Any + +from .translate import responses_to_chat, strip_think + +CURSOR_MODEL_SLUG = "composer-2-5" +CURSOR_UPSTREAM_MODEL = "composer-2.5" +CURSOR_DISPLAY_NAME = "Composer 2.5" +CURSOR_PASSTHROUGH_SLUGS = frozenset({CURSOR_MODEL_SLUG, "composer-2.5"}) +_AUTH_PROBE_TTL_SEC = 30.0 +_auth_probe_cache: tuple[float, bool] | None = None + + +def cursor_spawn_env() -> dict[str, str]: + """Environment for cursor-agent child processes. + + Following open-design's subscription-first pattern: a stale + ``CURSOR_API_KEY`` in the shell must not override ``cursor-agent login``. + """ + env = os.environ.copy() + env.pop("CURSOR_API_KEY", None) + bin_override = env.get("CURSOR_AGENT_BIN", "").strip() + if bin_override: + env["PATH"] = f"{os.path.dirname(bin_override)}:{env.get('PATH', '')}" + return env + + +def _cursor_agent_bin() -> str: + override = os.environ.get("CURSOR_AGENT_BIN", "").strip() + if override: + return override + return shutil.which("cursor-agent") or "cursor-agent" + + +def _is_cursor_auth_failure(text: str) -> bool: + value = text.strip() + if not value: + return False + lowered = value.lower() + return any( + marker in lowered + for marker in ( + "authentication required", + "not authenticated", + "not logged in", + "please run", + "agent login", + "cursor_api_key", + ) + ) + + +def _probe_cursor_auth() -> bool: + if os.environ.get("CODEX_SHIM_DISABLE_CURSOR", "").lower() in {"1", "true", "yes", "on"}: + return False + if not shutil.which(_cursor_agent_bin()) and not os.environ.get("CURSOR_AGENT_BIN"): + return False + try: + result = subprocess.run( + [_cursor_agent_bin(), "status"], + capture_output=True, + text=True, + timeout=15, + env=cursor_spawn_env(), + ) + except (OSError, subprocess.TimeoutExpired): + return False + output = f"{result.stdout}\n{result.stderr}" + if _is_cursor_auth_failure(output): + return False + return "logged in" in output.lower() + + +def cursor_passthrough_available(*, force_refresh: bool = False) -> bool: + """Return True when cursor-agent is installed and logged in.""" + global _auth_probe_cache + now = time.monotonic() + if not force_refresh and _auth_probe_cache is not None: + cached_at, cached = _auth_probe_cache + if now - cached_at < _AUTH_PROBE_TTL_SEC: + return cached + available = _probe_cursor_auth() + _auth_probe_cache = (now, available) + return available + + +def is_cursor_passthrough_slug(slug: str) -> bool: + return slug in CURSOR_PASSTHROUGH_SLUGS + + +def cursor_upstream_model(_slug: str) -> str: + return CURSOR_UPSTREAM_MODEL + + +def cursor_workspace() -> str: + override = os.environ.get("CODEX_SHIM_CURSOR_WORKSPACE", "").strip() + return override or os.getcwd() + + +def cursor_passthrough_display_names() -> dict[str, str]: + return {CURSOR_MODEL_SLUG: CURSOR_DISPLAY_NAME} + + +def cursor_catalog_entry() -> dict[str, Any]: + return { + "slug": CURSOR_MODEL_SLUG, + "display_name": CURSOR_DISPLAY_NAME, + "description": "Cursor Composer 2.5 routed through your Cursor subscription (cursor-agent login).", + "context_window": 272_000, + "max_context_window": 272_000, + "auto_compact_token_limit": 217_600, + "truncation_policy": {"mode": "tokens", "limit": 64_000}, + "default_reasoning_level": "medium", + "supported_reasoning_levels": [ + {"effort": "low", "description": "Faster, lighter reasoning"}, + {"effort": "medium", "description": "Balanced speed and reasoning"}, + {"effort": "high", "description": "Deeper reasoning"}, + ], + "default_reasoning_summary": "none", + "reasoning_summary_format": "none", + "supports_reasoning_summaries": False, + "default_verbosity": "low", + "support_verbosity": False, + "apply_patch_tool_type": "freeform", + "web_search_tool_type": "text_and_image", + "supports_search_tool": False, + "supports_parallel_tool_calls": True, + "experimental_supported_tools": [], + "input_modalities": ["text", "image"], + "supports_image_detail_original": True, + "shell_type": "shell_command", + "visibility": "list", + "minimal_client_version": "0.0.1", + "supported_in_api": True, + "availability_nux": None, + "upgrade": None, + "priority": 11000, + "prefer_websockets": False, + "available_in_plans": ["free", "plus", "pro", "team", "business", "enterprise"], + "base_instructions": "You are Codex, a coding agent powered by Composer 2.5.", + "model_messages": { + "instructions_template": "You are Codex, a coding agent powered by Composer 2.5.", + "instructions_variables": {"model_name": CURSOR_DISPLAY_NAME}, + }, + } + + +def build_cursor_prompt(body: dict[str, Any]) -> str: + """Convert a Codex Responses payload into a cursor-agent prompt.""" + chat = responses_to_chat(body, cursor_upstream_model(str(body.get("model") or ""))) + sections: list[str] = [] + for message in chat.get("messages") or []: + role = str(message.get("role") or "user").upper() + content = _message_content(message) + if not content: + continue + if role in {"SYSTEM", "DEVELOPER"}: + sections.append(f"[{role}]\n{content}") + elif role == "ASSISTANT": + tool_calls = message.get("tool_calls") or [] + if tool_calls: + rendered_calls = [] + for call in tool_calls: + fn = call.get("function") or {} + rendered_calls.append( + f"{fn.get('name') or 'tool'}({fn.get('arguments') or ''})" + ) + sections.append(f"[ASSISTANT]\n{content}\nTool calls: {', '.join(rendered_calls)}") + else: + sections.append(f"[ASSISTANT]\n{content}") + elif role == "TOOL": + sections.append(f"[TOOL {message.get('tool_call_id', '')}]\n{content}") + else: + sections.append(f"[USER]\n{content}") + prompt = "\n\n".join(sections).strip() + return prompt or "Continue." + + +def _message_content(message: dict[str, Any]) -> str: + content = message.get("content") + if isinstance(content, str): + return strip_think(content) + if isinstance(content, list): + parts: list[str] = [] + for item in content: + if isinstance(item, str): + parts.append(item) + elif isinstance(item, dict): + if item.get("type") in {"text", "input_text", "output_text"}: + parts.append(str(item.get("text") or "")) + elif item.get("type") in {"input_image", "image_url"}: + parts.append("[image omitted for cursor-agent bridge]") + return strip_think("\n".join(part for part in parts if part)) + return "" + + +def _extract_cursor_assistant_text(message: Any) -> str: + if not isinstance(message, dict): + return "" + content = message.get("content") + if not isinstance(content, list): + return "" + parts = [ + str(block.get("text") or "") + for block in content + if isinstance(block, dict) and block.get("type") == "text" and block.get("text") + ] + return "".join(parts) + + +class CursorStreamParser: + """Parse cursor-agent ``stream-json`` lines into text deltas and usage.""" + + def __init__(self) -> None: + self.text_so_far = "" + self.final_text = "" + self.usage: dict[str, Any] | None = None + self.error: str | None = None + + def feed_line(self, line: str) -> str | None: + line = line.strip() + if not line: + return None + try: + obj = json.loads(line) + except json.JSONDecodeError: + return None + if not isinstance(obj, dict): + return None + + obj_type = obj.get("type") + if obj_type == "assistant" and obj.get("message"): + text = _extract_cursor_assistant_text(obj.get("message")) + if not text: + return None + delta = self._delta_for_text(text) + self.final_text = text + return delta + if obj_type == "result": + if obj.get("subtype") == "error" or obj.get("is_error"): + self.error = str(obj.get("result") or obj.get("error") or "cursor-agent failed") + elif isinstance(obj.get("result"), str) and obj.get("result"): + self.final_text = str(obj["result"]) + usage = obj.get("usage") + if isinstance(usage, dict): + self.usage = { + "input_tokens": usage.get("inputTokens"), + "output_tokens": usage.get("outputTokens"), + "cache_read_input_tokens": usage.get("cacheReadTokens"), + "cache_creation_input_tokens": usage.get("cacheWriteTokens"), + } + return None + if obj_type == "error": + self.error = str(obj.get("message") or obj.get("error") or "cursor-agent error") + return None + + def _delta_for_text(self, text: str) -> str | None: + if not self.text_so_far: + self.text_so_far = text + return text + if text == self.text_so_far: + return None + if text.startswith(self.text_so_far): + delta = text[len(self.text_so_far) :] + self.text_so_far = text + return delta or None + self.text_so_far += text + return text + + +async def iter_cursor_agent_events(prompt: str, model: str) -> AsyncIterator[dict[str, Any]]: + """Spawn cursor-agent and yield normalized stream events.""" + cmd = [ + _cursor_agent_bin(), + "--print", + "--output-format", + "stream-json", + "--stream-partial-output", + "--force", + "--trust", + "--workspace", + cursor_workspace(), + "--model", + model, + ] + proc = await asyncio.create_subprocess_exec( + *cmd, + stdin=asyncio.subprocess.PIPE, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + env=cursor_spawn_env(), + ) + assert proc.stdout is not None + stderr_chunks: list[bytes] = [] + + async def _drain_stderr() -> None: + if proc.stderr is None: + return + while True: + chunk = await proc.stderr.read(4096) + if not chunk: + break + stderr_chunks.append(chunk) + + stderr_task = asyncio.create_task(_drain_stderr()) + parser = CursorStreamParser() + stdout_complete = False + try: + if proc.stdin is not None: + proc.stdin.write(prompt.encode("utf-8")) + await proc.stdin.drain() + proc.stdin.close() + + buffer = "" + while True: + chunk = await proc.stdout.read(4096) + if not chunk: + break + buffer += chunk.decode("utf-8", errors="replace") + while "\n" in buffer: + line, buffer = buffer.split("\n", 1) + delta = parser.feed_line(line) + if delta: + yield {"type": "text_delta", "delta": delta} + if buffer.strip(): + delta = parser.feed_line(buffer) + if delta: + yield {"type": "text_delta", "delta": delta} + stdout_complete = True + finally: + if not stdout_complete and proc.returncode is None: + proc.kill() + await proc.wait() + try: + await stderr_task + except asyncio.CancelledError: + pass + + stderr = b"".join(stderr_chunks).decode("utf-8", errors="replace") + if parser.error: + yield {"type": "error", "message": parser.error} + elif proc.returncode not in (0, None) and not parser.final_text: + message = stderr.strip() or f"cursor-agent exited with code {proc.returncode}" + if _is_cursor_auth_failure(message): + message = ( + "Cursor Agent is not authenticated. Run `cursor-agent login`, " + "then `cursor-agent status`, and retry." + ) + yield {"type": "error", "message": message} + if parser.usage: + yield {"type": "usage", "usage": parser.usage} + if parser.final_text: + yield {"type": "completed", "text": parser.final_text} diff --git a/codex_shim/hostguard.py b/codex_shim/hostguard.py new file mode 100644 index 00000000..59d5a741 --- /dev/null +++ b/codex_shim/hostguard.py @@ -0,0 +1,66 @@ +"""Host-header allowlist for the loopback shim server. + +Binding to 127.0.0.1 keeps the shim off the network, but it does not stop a +web page the user is already viewing from reaching it. A browser resolves an +attacker-controlled domain to 127.0.0.1 (DNS rebinding) and then issues +same-origin requests to the shim; the loopback socket happily accepts them. +Because the shim forwards each request upstream with the user's BYOK API keys +or ChatGPT access token, an unguarded shim lets any visited page spend the +user's model credits and read the responses. + +The browser still sends the attacker's domain in the ``Host`` header, so a +Host allowlist is the standard defense: accept loopback names plus whatever +bind host the operator configured, reject everything else. +""" + +from __future__ import annotations + +import os + +from aiohttp import web + +DEFAULT_ALLOWED_HOSTS = frozenset({"127.0.0.1", "localhost", "::1"}) +ALLOWED_HOSTS_ENV = "CODEX_SHIM_ALLOWED_HOSTS" +_WILDCARD_BINDS = frozenset({"", "0.0.0.0", "::"}) + + +def host_only(host_header: str) -> str: + """Return the hostname from a ``Host`` header value, port stripped.""" + value = (host_header or "").strip() + if not value: + return "" + if value.startswith("["): # bracketed IPv6 literal, e.g. [::1]:8765 + end = value.find("]") + if end != -1: + return value[1:end] + return value + if value.count(":") == 1: # host:port (bare IPv6 has 2+ colons) + return value.rsplit(":", 1)[0] + return value + + +def build_allowed_hosts(bind_host: str) -> set[str]: + """Loopback names + the configured bind host + the env allowlist.""" + allowed = {host.lower() for host in DEFAULT_ALLOWED_HOSTS} + bind = (bind_host or "").strip().lower() + if bind and bind not in _WILDCARD_BINDS: + allowed.add(bind) + for part in os.environ.get(ALLOWED_HOSTS_ENV, "").split(","): + part = part.strip().lower() + if part: + allowed.add(part) + return allowed + + +def host_guard_middleware(allowed_hosts: set[str]): + """aiohttp middleware that rejects requests with a non-allowlisted Host.""" + allowed = {host.lower() for host in allowed_hosts} + + @web.middleware + async def _guard(request: web.Request, handler): + hostname = host_only(request.headers.get("Host", "")).lower() + if hostname not in allowed: + raise web.HTTPForbidden(text="Forbidden: Host header not allowed") + return await handler(request) + + return _guard diff --git a/codex_shim/opencode_go.py b/codex_shim/opencode_go.py new file mode 100644 index 00000000..f7c904e5 --- /dev/null +++ b/codex_shim/opencode_go.py @@ -0,0 +1,238 @@ +from __future__ import annotations + +from dataclasses import dataclass +import json +import os +from pathlib import Path +from typing import Any +from urllib.error import HTTPError, URLError +from urllib.request import Request, urlopen + +from .settings import slugify + + +OPENCODE_GO_BASE_URL = "https://opencode.ai/zen/go/v1" +OPENCODE_GO_API_KEY_ENV = "OPENCODE_GO_API_KEY" +OPENCODE_GO_GENERATED_BY = "codex-shim opencode-go refresh" + + +@dataclass(frozen=True) +class OpenCodeGoRefreshResult: + settings_path: Path + models: list[dict[str, Any]] + skipped: list[tuple[str, int, int]] + + +def refresh_opencode_go_settings( + settings_path: Path, + *, + api_key_env: str = OPENCODE_GO_API_KEY_ENV, + base_url: str = OPENCODE_GO_BASE_URL, + prefer: str = "chat", + timeout: float = 30.0, +) -> OpenCodeGoRefreshResult: + api_key = os.environ.get(api_key_env, "").strip() + if not api_key: + raise RuntimeError(f"Set {api_key_env} before refreshing OpenCode Go models.") + + base_url = base_url.rstrip("/") + model_ids = fetch_opencode_go_model_ids(base_url, api_key, timeout=timeout) + generated: list[dict[str, Any]] = [] + skipped: list[tuple[str, int, int]] = [] + for model_id in model_ids: + chat_status = probe_chat_model(base_url, api_key, model_id, timeout=timeout) + messages_status = probe_messages_model(base_url, api_key, model_id, timeout=timeout) + row = opencode_go_model_row( + model_id, + chat_status=chat_status, + messages_status=messages_status, + api_key_env=api_key_env, + base_url=base_url, + prefer=prefer, + ) + if row is None: + skipped.append((model_id, chat_status, messages_status)) + else: + generated.append(row) + + write_opencode_go_models(settings_path, generated, base_url=base_url) + return OpenCodeGoRefreshResult(settings_path=Path(settings_path).expanduser(), models=generated, skipped=skipped) + + +def fetch_opencode_go_model_ids(base_url: str, api_key: str, *, timeout: float = 30.0) -> list[str]: + status, payload = _request_json( + "GET", + f"{base_url.rstrip('/')}/models", + {"Authorization": f"Bearer {api_key}"}, + timeout=timeout, + ) + if status < 200 or status >= 300: + raise RuntimeError(f"OpenCode Go /models returned HTTP {status}.") + data = payload.get("data") if isinstance(payload, dict) else None + if not isinstance(data, list): + raise RuntimeError("OpenCode Go /models did not return a model list.") + ids = [str(item.get("id") or "").strip() for item in data if isinstance(item, dict)] + return [model_id for model_id in ids if model_id] + + +def probe_chat_model(base_url: str, api_key: str, model_id: str, *, timeout: float = 30.0) -> int: + status, _payload = _request_json( + "POST", + f"{base_url.rstrip('/')}/chat/completions", + { + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + }, + { + "model": model_id, + "messages": [{"role": "user", "content": "hi"}], + "max_tokens": 1, + }, + timeout=timeout, + ) + return status + + +def probe_messages_model(base_url: str, api_key: str, model_id: str, *, timeout: float = 30.0) -> int: + status, _payload = _request_json( + "POST", + f"{base_url.rstrip('/')}/messages", + { + "x-api-key": api_key, + "anthropic-version": "2023-06-01", + "Content-Type": "application/json", + }, + { + "model": model_id, + "messages": [{"role": "user", "content": "hi"}], + "max_tokens": 1, + }, + timeout=timeout, + ) + return status + + +def opencode_go_model_row( + model_id: str, + *, + chat_status: int, + messages_status: int, + api_key_env: str, + base_url: str, + prefer: str, +) -> dict[str, Any] | None: + chat_ok = 200 <= chat_status < 300 + messages_ok = 200 <= messages_status < 300 + if chat_ok and (prefer == "chat" or not messages_ok): + provider = "generic-chat-completion-api" + endpoint = "chat" + elif messages_ok: + provider = "anthropic" + endpoint = "messages" + else: + return None + return { + "slug": f"ocgo-{slugify(model_id)}", + "model": model_id, + "display_name": f"OpenCode Go {display_name_from_model_id(model_id)}", + "provider": provider, + "base_url": base_url.rstrip("/"), + "api_key_env": api_key_env, + "no_image_support": True, + "generated_by": OPENCODE_GO_GENERATED_BY, + "opencode_go_endpoint": endpoint, + } + + +def write_opencode_go_models(settings_path: Path, rows: list[dict[str, Any]], *, base_url: str = OPENCODE_GO_BASE_URL) -> None: + path = Path(settings_path).expanduser() + existing = _read_settings_json(path) + payload = dict(existing) if isinstance(existing, dict) else {} + target = next((k for k in ("models", "customModels", "launchModels", "launch_models") if k in payload), "models") + preserved = [row for row in _settings_rows(existing) if not _is_opencode_go_row(row, base_url)] + payload[target] = [*preserved, *rows] + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(payload, indent=2) + "\n") + + +def display_name_from_model_id(model_id: str) -> str: + aliases = { + "glm": "GLM", + "kimi": "Kimi", + "deepseek": "DeepSeek", + "qwen": "Qwen", + "minimax": "MiniMax", + "mimo": "MiMo", + } + parts = model_id.replace("-", " ").split() + titled = [] + for part in parts: + lower = part.lower() + if lower in aliases: + titled.append(aliases[lower]) + elif lower.startswith("qwen"): + titled.append("Qwen" + part[4:]) + elif part[0].isalpha() and len(part) > 1 and part[1:].isdigit(): + titled.append(part[0].upper() + part[1:]) + else: + titled.append(part.capitalize()) + return " ".join(titled) + + +def _request_json( + method: str, + url: str, + headers: dict[str, str], + body: dict[str, Any] | None = None, + *, + timeout: float = 30.0, +) -> tuple[int, dict[str, Any]]: + data = json.dumps(body).encode("utf-8") if body is not None else None + request_headers = {"User-Agent": "codex-shim", "Accept": "application/json", **headers} + request = Request(url, data=data, headers=request_headers, method=method) + try: + with urlopen(request, timeout=timeout) as response: + raw = response.read().decode("utf-8", errors="replace") + return int(response.status), _json_payload(raw) + except HTTPError as exc: + raw = exc.read().decode("utf-8", errors="replace") + return int(exc.code), _json_payload(raw) + except URLError as exc: + raise RuntimeError(f"Could not reach OpenCode Go: {exc.reason}") from exc + + +def _json_payload(text: str) -> dict[str, Any]: + try: + payload = json.loads(text or "{}") + except json.JSONDecodeError: + return {} + return payload if isinstance(payload, dict) else {} + + +def _read_settings_json(path: Path) -> Any: + expanded = Path(path).expanduser() + if not expanded.exists(): + return {} + try: + return json.loads(expanded.read_text()) + except json.JSONDecodeError as exc: + raise RuntimeError(f"Settings file is not valid JSON: {expanded}: {exc}") from exc + + +def _settings_rows(data: Any) -> list[dict[str, Any]]: + rows: Any + if isinstance(data, list): + rows = data + elif isinstance(data, dict): + rows = data.get("models") + if rows is None: + rows = data.get("customModels") + if rows is None: + rows = data.get("launchModels", data.get("launch_models", [])) + else: + rows = [] + return [dict(row) for row in rows if isinstance(row, dict)] if isinstance(rows, list) else [] + + +def _is_opencode_go_row(row: dict[str, Any], base_url: str) -> bool: + return row.get("generated_by") == OPENCODE_GO_GENERATED_BY diff --git a/codex_shim/router.py b/codex_shim/router.py new file mode 100644 index 00000000..152eadd9 --- /dev/null +++ b/codex_shim/router.py @@ -0,0 +1,538 @@ +"""Auto Router — pick the right configured model for each task, automatically. + +The Auto Router adds one virtual model (default slug ``codex-auto``) to the +shim. When Codex sends a request for that slug, the shim asks a small, cheap +*classifier* model — one of your own configured BYOK models — to score every +candidate model on how likely it is to complete *this* task correctly on the +first try (0.0–1.0). The shim then routes the real request to the **cheapest** +candidate whose score clears a quality bar (default ``0.7``), so trivial turns +go to a cheap model and hard turns escalate to your strongest one. + +Design choices worth calling out: + +* The classifier scores on **capability only** — it never sees price. Cost is + applied afterward by the "cheapest among those good enough" tie-break, so the + classifier can't be biased toward expensive models. +* Each decision is **cached per task** (keyed by the latest user message), so a + task's follow-up tool-call round-trips reuse one classification instead of + paying the classifier tax on every request. +* It **degrades safely** at every step: unknown candidate slugs are dropped, a + missing/unavailable classifier falls back to the cheapest candidate + deterministically, and any error falls back to the configured default. The + request never breaks. + +This module is intentionally network-free and pure: the actual classifier HTTP +call is injected as a callable by ``server.py`` so the scoring/selection logic +is fully unit-testable offline. +""" + +from __future__ import annotations + +import json +import os +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Awaitable, Callable, Optional + +DEFAULT_ROUTER_SLUG = "codex-auto" +DEFAULT_ROUTER_DISPLAY_NAME = "Auto (smart routing)" +DEFAULT_THRESHOLD = 0.7 +DEFAULT_TIMEOUT = 12.0 +DEFAULT_MAX_TOKENS = 600 + +_CACHE_MAX = 256 +_cache: dict[str, str] = {} + +# An async callable that takes (system_prompt, user_content) and returns the +# classifier model's raw reply text. +ClassifyFn = Callable[[str, str], Awaitable[str]] + + +# --------------------------------------------------------------------------- +# Env knobs +# --------------------------------------------------------------------------- +def _env_flag(name: str) -> bool: + return os.environ.get(name, "").lower() in {"1", "true", "yes", "on"} + + +def router_disabled_via_env() -> bool: + """``CODEX_SHIM_DISABLE_ROUTER=1`` turns the router off even if enabled.""" + return _env_flag("CODEX_SHIM_DISABLE_ROUTER") + + +def router_log_enabled() -> bool: + """``CODEX_SHIM_ROUTER_LOG=1`` logs every routing decision + raw scores.""" + return _env_flag("CODEX_SHIM_ROUTER_LOG") + + +# --------------------------------------------------------------------------- +# Config +# --------------------------------------------------------------------------- +@dataclass(frozen=True) +class RouterCandidate: + slug: str + cost: float = 1.0 + card: str = "" + supports_images: bool = False + + +@dataclass(frozen=True) +class RouterConfig: + enabled: bool + slug: str + display_name: str + classifier: Optional[str] + threshold: float + default: Optional[str] + cache: bool + candidates: tuple[RouterCandidate, ...] + timeout: float + max_tokens: int + + @property + def effective_enabled(self) -> bool: + return self.enabled and not router_disabled_via_env() + + +def _as_float(value: Any, default: float) -> float: + try: + return float(value) + except (TypeError, ValueError): + return default + + +def load_router_config(settings_path: Path | str) -> Optional[RouterConfig]: + """Read the optional top-level ``router`` block from the settings JSON. + + Returns ``None`` when the file is missing/invalid or has no ``router`` key, + so the rest of the shim treats "no router configured" as the default. + """ + path = Path(settings_path).expanduser() + if not path.exists(): + return None + try: + data = json.loads(path.read_text()) + except (OSError, json.JSONDecodeError): + return None + if not isinstance(data, dict): + return None + raw = data.get("router") + if not isinstance(raw, dict): + return None + + candidates: list[RouterCandidate] = [] + for entry in raw.get("candidates") or []: + if not isinstance(entry, dict): + continue + slug = str(entry.get("slug") or entry.get("id") or "").strip() + if not slug: + continue + candidates.append( + RouterCandidate( + slug=slug, + cost=_as_float(entry.get("cost"), 1.0), + card=str(entry.get("card") or "").strip(), + supports_images=bool(entry.get("supports_images", False)), + ) + ) + + timeout = _as_float(raw.get("timeout"), DEFAULT_TIMEOUT) + env_timeout = os.environ.get("CODEX_SHIM_ROUTER_TIMEOUT") + if env_timeout: + timeout = _as_float(env_timeout, timeout) + + max_tokens = int(_as_float(raw.get("max_tokens"), DEFAULT_MAX_TOKENS)) + env_max = os.environ.get("CODEX_SHIM_ROUTER_MAX_TOKENS") + if env_max: + max_tokens = int(_as_float(env_max, max_tokens)) + + return RouterConfig( + enabled=bool(raw.get("enabled", False)), + slug=str(raw.get("slug") or raw.get("id") or DEFAULT_ROUTER_SLUG).strip() or DEFAULT_ROUTER_SLUG, + display_name=str(raw.get("display_name") or DEFAULT_ROUTER_DISPLAY_NAME), + classifier=(str(raw["classifier"]).strip() or None) if raw.get("classifier") else None, + threshold=_as_float(raw.get("threshold"), DEFAULT_THRESHOLD), + default=(str(raw["default"]).strip() or None) if raw.get("default") else None, + cache=bool(raw.get("cache", True)), + candidates=tuple(candidates), + timeout=timeout, + max_tokens=max_tokens, + ) + + +def filter_available(config: RouterConfig, available_slugs) -> list[RouterCandidate]: + """Candidates whose backend is actually usable right now (route among the + models that exist), never including the router's own virtual slug.""" + avail = set(available_slugs) + return [c for c in config.candidates if c.slug in avail and c.slug != config.slug] + + +def router_is_active(config: Optional[RouterConfig], available_slugs) -> bool: + return bool(config) and config.effective_enabled and len(filter_available(config, available_slugs)) >= 1 + + +# --------------------------------------------------------------------------- +# Catalog / discovery entries for the virtual model +# --------------------------------------------------------------------------- +def router_models_entry(config: RouterConfig, created: int) -> dict[str, Any]: + return {"id": config.slug, "object": "model", "created": created, "owned_by": "codex-shim-auto"} + + +def router_catalog_entry(config: RouterConfig) -> dict[str, Any]: + return { + "slug": config.slug, + "display_name": config.display_name, + "description": "Automatically routes each task to the cheapest configured model that can handle it.", + "context_window": 400_000, + "max_context_window": 400_000, + "auto_compact_token_limit": 320_000, + "truncation_policy": {"mode": "tokens", "limit": 64_000}, + "default_reasoning_level": "medium", + "supported_reasoning_levels": [ + {"effort": "low", "description": "Faster, lighter reasoning"}, + {"effort": "medium", "description": "Balanced speed and reasoning"}, + {"effort": "high", "description": "Deeper reasoning"}, + {"effort": "xhigh", "description": "Maximum reasoning where supported"}, + ], + "default_reasoning_summary": "none", + "reasoning_summary_format": "none", + "supports_reasoning_summaries": False, + "default_verbosity": "low", + "support_verbosity": False, + "apply_patch_tool_type": "freeform", + "web_search_tool_type": "text_and_image", + "supports_search_tool": False, + "supports_parallel_tool_calls": True, + "experimental_supported_tools": [], + "input_modalities": ["text", "image"], + "supports_image_detail_original": True, + "shell_type": "shell_command", + "visibility": "list", + "minimal_client_version": "0.0.1", + "supported_in_api": True, + "availability_nux": None, + "upgrade": None, + "priority": 12000, + "prefer_websockets": False, + "available_in_plans": ["free", "plus", "pro", "team", "business", "enterprise"], + "base_instructions": "You are Codex, a coding agent. The active model is chosen automatically per task.", + "model_messages": { + "instructions_template": "You are Codex, a coding agent. The active model is chosen automatically per task.", + "instructions_variables": {"model_name": config.display_name}, + }, + } + + +# --------------------------------------------------------------------------- +# Task signal extraction (works on Responses bodies and chat-completions bodies) +# --------------------------------------------------------------------------- +def _content_text(content: Any) -> str: + if content is None: + return "" + if isinstance(content, str): + return content + if isinstance(content, list): + parts = [] + for part in content: + if isinstance(part, dict): + parts.append(str(part.get("text") or part.get("content") or "")) + elif isinstance(part, str): + parts.append(part) + return "\n".join(p for p in parts if p) + if isinstance(content, dict): + return str(content.get("text") or content.get("content") or "") + return str(content) + + +def _latest_from_input(inp: Any) -> str: + if isinstance(inp, str): + return inp.strip() + if not isinstance(inp, list): + return "" + for item in reversed(inp): + if isinstance(item, str): + return item.strip() + if not isinstance(item, dict): + continue + if item.get("role") == "user": + text = _content_text(item.get("content")) + if text.strip(): + return text.strip() + elif item.get("type") in {"input_text", "text"}: + text = _content_text(item) + if text.strip(): + return text.strip() + return "" + + +def _latest_from_messages(messages: Any) -> str: + if not isinstance(messages, list): + return "" + for message in reversed(messages): + if isinstance(message, dict) and message.get("role") == "user": + text = _content_text(message.get("content")) + if text.strip(): + return text.strip() + return "" + + +def latest_user_text(body: dict[str, Any]) -> str: + """The latest real user instruction — the task to classify. + + Skips tool round-trip items (``function_call_output`` etc.) so a task's + follow-up turns keep the same cache key as the original ask. + """ + text = _latest_from_input(body.get("input")) + if text: + return text + return _latest_from_messages(body.get("messages")) + + +def _value_has_image(value: Any) -> bool: + if isinstance(value, dict): + vtype = str(value.get("type") or "") + if vtype in {"input_image", "image_url"} or "image_url" in value: + return True + return any(_value_has_image(v) for v in value.values()) + if isinstance(value, list): + return any(_value_has_image(v) for v in value) + return False + + +def has_images(body: dict[str, Any]) -> bool: + return _value_has_image(body.get("input")) or _value_has_image(body.get("messages")) + + +def task_signal(body: dict[str, Any]) -> dict[str, Any]: + inp = body.get("input") + if isinstance(inp, list): + input_items = len(inp) + elif inp: + input_items = 1 + else: + input_items = len(body.get("messages") or []) if isinstance(body.get("messages"), list) else 0 + tools = body.get("tools") or [] + return { + "task": latest_user_text(body), + "has_images": has_images(body), + "tool_count": len(tools) if isinstance(tools, list) else 0, + "input_items": input_items, + } + + +# --------------------------------------------------------------------------- +# Classifier prompt + score parsing + selection +# --------------------------------------------------------------------------- +def build_system_prompt(candidates: list[RouterCandidate]) -> str: + lines = [ + "You are a task-routing classifier for an AI coding agent.", + "You are given a describing the user's current task and a list", + "of candidate models. For EACH candidate, output a score from 0.0 to 1.0:", + "the probability that the model completes THIS task correctly on its first", + "attempt, without errors or rework.", + "", + "You are NOT choosing a winner. A downstream system combines your scores", + "with cost data you do not see to make the final pick. Be an accurate,", + "well-calibrated, independent probability estimator for each model.", + "", + "Scoring guide:", + " 0.0 cannot attempt (e.g. images required but unsupported) -- exact 0.0", + " 0.1-0.3 will almost certainly fail; lacks the capability", + " 0.4-0.6 real chance of failure; touches a known weakness or is uncertain", + " 0.7-0.8 likely success; handles this category well", + " 0.9-1.0 near-certain success; well within demonstrated ability", + "Use the full range. A short prompt is NOT necessarily an easy task -- hidden", + "complexity (multi-file edits, debugging, niche domains, strict correctness)", + "should pull scores down for weaker models. Default to ~0.5-0.6 when unsure.", + "", + "Candidate models:", + ] + for c in candidates: + card = c.card or "General-purpose model. No capability card provided." + lines.append("- slug: %s" % c.slug) + lines.append(" images: %s" % ("yes" if c.supports_images else "no")) + lines.append(" capability: %s" % card) + schema = {"scores": {c.slug: 0.0 for c in candidates}, "reasoning": "one short sentence"} + lines += [ + "", + "Respond with ONE JSON object, no prose, no code fence, exactly this shape:", + json.dumps(schema, ensure_ascii=False), + 'Every candidate slug above MUST appear in "scores". Each value in [0.0, 1.0].', + ] + return "\n".join(lines) + + +def build_user_content(signal: dict[str, Any], candidates: list[RouterCandidate]) -> str: + task = signal["task"] or "(no explicit instruction; infer from context)" + if len(task) > 6000: # head+tail, keep the classifier call cheap + task = task[:3000] + "\n...\n" + task[-3000:] + return "\n".join( + [ + "", + " images_present: %s" % ("yes" if signal["has_images"] else "no"), + " tools_available: %d" % signal["tool_count"], + " input_items: %d" % signal["input_items"], + " current_task: |", + "\n".join(" " + ln for ln in task.splitlines()) or " (empty)", + "", + "", + "Score these candidate slugs: %s" % ", ".join(c.slug for c in candidates), + ] + ) + + +def _clamp01(value: Any) -> float: + try: + value = float(value) + except (TypeError, ValueError): + return 0.0 + return 0.0 if value < 0 else (1.0 if value > 1 else value) + + +def parse_scores(text: str, candidate_slugs: list[str]) -> dict[str, float]: + """Pull the ``{"scores": {...}}`` object out of the classifier's reply.""" + if not text: + return {} + start, end = text.find("{"), text.rfind("}") + if start == -1 or end == -1 or end <= start: + return {} + for stop in (end, text.find("}", start)): # greedy object, then first object + if stop == -1 or stop <= start: + continue + try: + obj = json.loads(text[start : stop + 1]) + except Exception: + continue + scores = obj.get("scores") if isinstance(obj, dict) else None + if isinstance(scores, dict): + return {slug: _clamp01(scores.get(slug, 0)) for slug in candidate_slugs} + return {} + + +def pick_candidate( + scores: dict[str, float], + candidates: list[RouterCandidate], + threshold: float, + has_image_task: bool, +) -> tuple[Optional[str], float, str]: + """Cheapest candidate whose score clears the bar; image-incapable models are + hard-zeroed when the task has images; if none clear the bar, take the best.""" + scored: list[tuple[RouterCandidate, float]] = [] + for c in candidates: + score = scores.get(c.slug, 0.0) + if has_image_task and not c.supports_images: + score = 0.0 + scored.append((c, score)) + viable = [(c, s) for (c, s) in scored if s >= threshold] + if viable: + winner = min(viable, key=lambda cs: (float(cs[0].cost or 0), -cs[1])) + return winner[0].slug, winner[1], "score>=%.2f, cheapest" % threshold + best = max(scored, key=lambda cs: cs[1], default=None) + if best and best[1] > 0: + return best[0].slug, best[1], "below bar; highest score" + return None, 0.0, "no usable score" + + +def fallback_slug( + config: RouterConfig, + candidates: list[RouterCandidate], + has_image_task: bool = False, +) -> Optional[str]: + """Deterministic, classifier-free choice. + + When ``has_image_task`` is False (default), pick the configured default + if it is among the candidates, otherwise the cheapest candidate. This is + the original behaviour and is preserved so existing call sites do not + need to change semantics. + + When ``has_image_task`` is True, ignore candidates whose + ``supports_images`` is False — sending an image task to a text-only + upstream produces a confusing 400 (`unknown variant ``image_url`` `). + Among the remaining vision-capable candidates, prefer the configured + default if it qualifies, otherwise the cheapest. If no candidate + supports images, return None so the caller can surface the failure + explicitly rather than silently routing into an upstream that will + reject the body. + """ + pool = ( + [c for c in candidates if c.supports_images] if has_image_task else list(candidates) + ) + if not pool: + return None + if config.default and any(c.slug == config.default for c in pool): + return config.default + return min(pool, key=lambda c: float(c.cost or 0)).slug + + +def _cache_key(signal: dict[str, Any]) -> str: + return "%s|%s" % (signal["has_images"], hash(signal["task"])) + + +def reset_cache() -> None: + _cache.clear() + + +# --------------------------------------------------------------------------- +# Orchestration +# --------------------------------------------------------------------------- +async def resolve_auto( + config: RouterConfig, + candidates: list[RouterCandidate], + body: dict[str, Any], + classify: Optional[ClassifyFn], + *, + log: Optional[Callable[[str], None]] = None, +) -> tuple[Optional[str], dict[str, Any]]: + """Return the concrete candidate slug the Auto Router selects for this + request (or ``None`` if nothing is routable). Never raises.""" + + def _log(message: str) -> None: + if log: + log(message) + + try: + if not candidates: + return None, {"reason": "no candidates"} + if len(candidates) == 1: + return candidates[0].slug, {"reason": "single candidate", "scores": {}} + + signal = task_signal(body) + key = _cache_key(signal) + if config.cache: + cached = _cache.get(key) + if cached and any(c.slug == cached for c in candidates): + _log("[router] cache-hit -> %s" % cached) + return cached, {"reason": "cache", "scores": {}} + + if classify is None: + pick = fallback_slug(config, candidates, has_image_task=signal["has_images"]) + _log("[router] no classifier -> deterministic %s" % pick) + return pick, {"reason": "no classifier", "scores": {}} + + system_prompt = build_system_prompt(candidates) + user_content = build_user_content(signal, candidates) + try: + raw = await classify(system_prompt, user_content) + except Exception as exc: # noqa: BLE001 - any classifier failure is non-fatal + pick = fallback_slug(config, candidates, has_image_task=signal["has_images"]) + _log("[router] classifier failed (%s); falling back to %s" % (exc, pick)) + return pick, {"reason": "classifier error", "scores": {}} + + scores = parse_scores(raw, [c.slug for c in candidates]) + pick, score, why = pick_candidate(scores, candidates, config.threshold, signal["has_images"]) + if not pick: + pick = fallback_slug(config, candidates, has_image_task=signal["has_images"]) + why = "empty scores; fallback" + score = 0.0 + if config.cache and pick: + if len(_cache) >= _CACHE_MAX: + _cache.clear() + _cache[key] = pick + _log( + "[router] -> %s (score=%.2f; %s) scores=%s" + % (pick, score, why, json.dumps({c.slug: round(scores.get(c.slug, 0.0), 2) for c in candidates})) + ) + return pick, {"reason": why, "score": score, "scores": scores} + except Exception as exc: # noqa: BLE001 - routing must never break a request + _log("[router] unexpected error: %s" % exc) + return None, {"reason": "error", "scores": {}} diff --git a/codex_shim/server.py b/codex_shim/server.py index bb906ab2..787b0d08 100644 --- a/codex_shim/server.py +++ b/codex_shim/server.py @@ -2,68 +2,217 @@ import argparse import json +import re +import secrets +import sys import time +import uuid from pathlib import Path from typing import Any from urllib.parse import urljoin from aiohttp import ClientSession, ClientTimeout, web +from .cursor_passthrough import ( + CURSOR_MODEL_SLUG, + build_cursor_prompt, + cursor_passthrough_available, + cursor_passthrough_display_names, + cursor_upstream_model, + is_cursor_passthrough_slug, + iter_cursor_agent_events, +) +from . import router as router_module +from .hostguard import build_allowed_hosts, host_guard_middleware from .settings import ( CHATGPT_MODEL_SLUG, DEFAULT_CODEX_AUTH, DEFAULT_SETTINGS, DEFAULT_HOST, DEFAULT_PORT, + PROVIDER_NAME, ModelSettings, ShimModel, + available_model_slugs, chatgpt_passthrough_available, + chatgpt_passthrough_display_names, + chatgpt_passthrough_slugs, + byok_model_has_credentials, + chatgpt_upstream_model, + is_chatgpt_passthrough_slug, + usable_byok_models, ) from .translate import ( SHIM_ENCRYPTED_CONTENT_PREFIX, + anthropic_messages_to_chat, anthropic_to_chat_response, anthropic_to_response, + chat_completion_to_anthropic_message, chat_completion_to_response, chat_to_anthropic, + normalize_responses_usage, responses_to_anthropic, responses_to_chat, + _chat_finish_to_anthropic_stop, + _responses_usage_to_anthropic_usage, ) +DEBUG_DIR = Path(__file__).resolve().parents[1] / ".codex-shim" +CODEX_CONFIG_PATH = Path.home() / ".codex" / "config.toml" +PICKER_TOKEN_HEADER = "X-Codex-Shim-Picker-Token" + class ShimServer: - def __init__(self, settings_path: Path = DEFAULT_SETTINGS): + def __init__(self, settings_path: Path = DEFAULT_SETTINGS, host: str = DEFAULT_HOST): self.settings = ModelSettings(settings_path) + self.host = host self.timeout = ClientTimeout(total=None, sock_connect=120, sock_read=None) + self.picker_token = secrets.token_urlsafe(32) def app(self) -> web.Application: - app = web.Application(client_max_size=64 * 1024 * 1024) + allowed_hosts = build_allowed_hosts(self.host) + app = web.Application( + client_max_size=64 * 1024 * 1024, + middlewares=[host_guard_middleware(allowed_hosts)], + ) app.router.add_get("/health", self.health) app.router.add_get("/v1/models", self.models) app.router.add_post("/v1/chat/completions", self.chat_completions) + app.router.add_post("/v1/messages", self.anthropic_messages) app.router.add_post("/v1/responses", self.responses) + app.router.add_post("/v1/responses/compact", self.responses_compact) + app.router.add_get("/picker", self.picker_page) + app.router.add_get("/api/models", self.api_models) + app.router.add_post("/api/switch", self.switch_model) return app + async def picker_page(self, _request: web.Request) -> web.Response: + return web.Response(text=_picker_html(self.picker_token), content_type="text/html") + + async def api_models(self, _request: web.Request) -> web.Response: + current = _current_managed_model() + data: list[dict[str, Any]] = [] + router_config = self._active_router() + if router_config is not None: + data.append( + { + "slug": router_config.slug, + "display_name": router_config.display_name, + "provider": "auto", + "active": current == router_config.slug, + } + ) + if chatgpt_passthrough_available(): + for slug, display_name in chatgpt_passthrough_display_names().items(): + data.append( + { + "slug": slug, + "display_name": display_name, + "provider": "chatgpt", + "active": current == slug, + } + ) + if cursor_passthrough_available(): + for slug, display_name in cursor_passthrough_display_names().items(): + data.append( + { + "slug": slug, + "display_name": display_name, + "provider": "cursor", + "active": current == slug, + } + ) + for m in usable_byok_models(self.settings.load()): + data.append( + { + "slug": m.slug, + "display_name": m.display_name, + "provider": m.provider, + "active": current == m.slug, + } + ) + return web.json_response(data) + + def _valid_picker_token(self, request: web.Request) -> bool: + token = request.headers.get(PICKER_TOKEN_HEADER, "") + return secrets.compare_digest(token, self.picker_token) + + async def switch_model(self, request: web.Request) -> web.Response: + if not self._valid_picker_token(request): + return web.json_response({"error": "forbidden"}, status=403) + try: + body = await request.json() + except json.JSONDecodeError: + return web.json_response({"error": "invalid JSON body"}, status=400) + slug = str(body.get("slug") or "").strip() + if not slug: + return web.json_response({"error": "slug is required"}, status=400) + models = usable_byok_models(self.settings.load()) + valid = {m.slug for m in models} + display_for: dict[str, str] = {m.slug: m.display_name for m in models} + router_config = self._active_router() + if router_config is not None: + valid.add(router_config.slug) + display_for[router_config.slug] = router_config.display_name + if chatgpt_passthrough_available(): + valid.update(chatgpt_passthrough_slugs()) + display_for.update(chatgpt_passthrough_display_names()) + if cursor_passthrough_available(): + valid.update(cursor_passthrough_display_names()) + display_for.update(cursor_passthrough_display_names()) + if slug not in valid: + return web.json_response({"error": f"unknown model: {slug}"}, status=404) + _set_active_model(slug, display_for.get(slug, slug)) + restart = bool(body.get("restart_codex")) + if restart: + _restart_codex_app() + return web.json_response({"ok": True, "model": slug, "restarted": restart}) + async def health(self, _request: web.Request) -> web.Response: - models = self.settings.load() - count = len(models) + (1 if chatgpt_passthrough_available() else 0) + models = usable_byok_models(self.settings.load()) + chatgpt_ok = chatgpt_passthrough_available() + cursor_ok = cursor_passthrough_available() + passthrough_count = len(chatgpt_passthrough_slugs()) if chatgpt_ok else 0 + if cursor_ok: + passthrough_count += len(cursor_passthrough_display_names()) + count = len(models) + passthrough_count return web.json_response( { "ok": True, "models": count, - "chatgpt_passthrough": chatgpt_passthrough_available(), + "chatgpt_passthrough": chatgpt_ok, + "cursor_passthrough": cursor_ok, + "auto_router": self._active_router() is not None, } ) async def models(self, _request: web.Request) -> web.Response: now = int(time.time()) data: list[dict[str, Any]] = [] + router_config = self._active_router() + if router_config is not None: + data.append(router_module.router_models_entry(router_config, now)) if chatgpt_passthrough_available(): - data.append({"id": CHATGPT_MODEL_SLUG, "object": "model", "created": now, "owned_by": "chatgpt"}) - data.extend({"id": model.slug, "object": "model", "created": now, "owned_by": "codex-shim"} for model in self.settings.load()) + data.extend( + {"id": slug, "object": "model", "created": now, "owned_by": "chatgpt"} + for slug in sorted(chatgpt_passthrough_slugs()) + ) + if cursor_passthrough_available(): + data.extend( + { + "id": slug, + "object": "model", + "created": now, + "owned_by": "cursor", + } + for slug in sorted(cursor_passthrough_display_names()) + ) + data.extend({"id": model.slug, "object": "model", "created": now, "owned_by": "codex-shim"} for model in usable_byok_models(self.settings.load())) return web.json_response({"object": "list", "data": data}) async def chat_completions(self, request: web.Request) -> web.StreamResponse: body = await request.json() + body = await self._maybe_apply_auto_router(body) route = self._route(body) if route.is_openai_chat: forwarded = dict(body) @@ -76,12 +225,39 @@ async def chat_completions(self, request: web.Request) -> web.StreamResponse: return await self._post_anthropic(request, route, forwarded, as_responses=False) raise web.HTTPBadGateway(text=f"Unsupported model provider: {route.provider}") + async def anthropic_messages(self, request: web.Request) -> web.StreamResponse: + body = await request.json() + route = self._route(body) + if route.is_openai_chat: + forwarded = anthropic_messages_to_chat(body, route.model, route.max_output_tokens) + return await self._post_openai_chat_as_anthropic(request, route, forwarded) + if route.is_anthropic: + forwarded = dict(body) + forwarded["model"] = route.model + return await self._post_anthropic_messages(request, route, forwarded) + raise web.HTTPBadGateway(text=f"Unsupported model provider: {route.provider}") + async def responses(self, request: web.Request) -> web.StreamResponse: body = await request.json() _log_incoming_request("/v1/responses", body) + body = await self._maybe_apply_auto_router(body) model = str(body.get("model") or "") - if model == CHATGPT_MODEL_SLUG or model.startswith("openai-gpt-5-5"): - return await self._chatgpt_passthrough(request, body) + if is_chatgpt_passthrough_slug(model): + upstream = chatgpt_upstream_model(model) + override = model if model != upstream else None + return await self._chatgpt_passthrough( + request, + body, + response_model_override=override, + upstream_model=upstream, + ) + if is_cursor_passthrough_slug(model): + return await self._cursor_passthrough( + request, + body, + response_model_override=model, + upstream_model=cursor_upstream_model(model), + ) if self._needs_image_gen(body) or self._needs_image_followup(body): return await self._chatgpt_passthrough(request, body, response_model_override=model) route = self._route(body) @@ -93,6 +269,42 @@ async def responses(self, request: web.Request) -> web.StreamResponse: return await self._post_anthropic(request, route, forwarded, as_responses=True) raise web.HTTPBadGateway(text=f"Unsupported model provider: {route.provider}") + async def responses_compact(self, request: web.Request) -> web.StreamResponse: + body = await request.json() + _log_incoming_request("/v1/responses/compact", body) + body = await self._maybe_apply_auto_router(body) + model = str(body.get("model") or "") + if is_chatgpt_passthrough_slug(model): + upstream = chatgpt_upstream_model(model) + return await self._chatgpt_compact_passthrough(request, body, upstream_model=upstream) + if is_cursor_passthrough_slug(model): + compact_body = dict(body) + compact_body["input"] = body.get("input") or [] + compact_body["instructions"] = ( + f"{body.get('instructions') or ''}\n\nSummarize the conversation above into a compact " + "context window suitable for continuing the task." + ).strip() + return await self._cursor_passthrough( + request, + compact_body, + response_model_override=model, + upstream_model=cursor_upstream_model(model), + force_non_stream=True, + ) + route = self._route(body) + compact_body = _compact_request_body(body, route.model) + if route.is_openai_chat: + forwarded = responses_to_chat(compact_body, route.model) + forwarded["stream"] = False + response = await self._post_openai_chat(request, route, forwarded, as_responses=True) + return await _as_compact_response(response, route.slug) + if route.is_anthropic: + forwarded = responses_to_anthropic(compact_body, route.model, route.max_output_tokens) + forwarded["stream"] = False + response = await self._post_anthropic(request, route, forwarded, as_responses=True) + return await _as_compact_response(response, route.slug) + raise web.HTTPBadGateway(text=f"Unsupported model provider: {route.provider}") + def _needs_image_gen(self, body: dict[str, Any]) -> bool: tools = body.get("tools") or [] image_tool_names: set[str] = set() @@ -251,12 +463,16 @@ def _content_to_debug_text(self, content: Any) -> str: return str(content) async def _chatgpt_passthrough( - self, request: web.Request, body: dict[str, Any], response_model_override: str | None = None + self, + request: web.Request, + body: dict[str, Any], + response_model_override: str | None = None, + upstream_model: str | None = None, ) -> web.StreamResponse: """Forward a Responses request to chatgpt.com using the user's Codex auth. - Lets the picker expose OpenAI's real GPT-5.5 (ChatGPT subscription) as a - first-class model alongside configured BYOK entries. + Lets the picker expose OpenAI GPT models (ChatGPT subscription) as + first-class models alongside configured BYOK entries. """ auth_path = DEFAULT_CODEX_AUTH.expanduser() try: @@ -269,7 +485,7 @@ async def _chatgpt_passthrough( if not access_token: raise web.HTTPUnauthorized(text="auth.json has no access_token") forwarded = _sanitize_chatgpt_passthrough_body(body) - forwarded["model"] = CHATGPT_MODEL_SLUG + forwarded["model"] = upstream_model or CHATGPT_MODEL_SLUG headers = { "Authorization": f"Bearer {access_token}", "Content-Type": "application/json", @@ -316,11 +532,219 @@ async def _chatgpt_passthrough( pass return response + async def _chatgpt_compact_passthrough( + self, + request: web.Request, + body: dict[str, Any], + upstream_model: str | None = None, + ) -> web.StreamResponse: + auth_path = DEFAULT_CODEX_AUTH.expanduser() + try: + auth = json.loads(auth_path.read_text()) + except FileNotFoundError: + raise web.HTTPUnauthorized(text="~/.codex/auth.json not found") + tokens = auth.get("tokens") or {} + access_token = tokens.get("access_token") + account_id = tokens.get("account_id") or "" + if not access_token: + raise web.HTTPUnauthorized(text="auth.json has no access_token") + forwarded = _sanitize_chatgpt_passthrough_body(body) + original_model = str(forwarded.get("model") or "") + forwarded["model"] = upstream_model or CHATGPT_MODEL_SLUG + forwarded.pop("stream", None) + headers = { + "Authorization": f"Bearer {access_token}", + "Content-Type": "application/json", + "Accept": "application/json", + "OpenAI-Beta": "responses=2026-02-06", + "originator": "codex_cli_rs", + "chatgpt-account-id": account_id, + "session_id": request.headers.get("session_id", ""), + } + url = "https://chatgpt.com/backend-api/codex/responses/compact" + async with ClientSession(timeout=self.timeout) as session: + upstream = await session.post(url, json=forwarded, headers=headers) + if upstream.status >= 400: + return await _error_response(upstream) + payload = await upstream.json(content_type=None) + _rewrite_response_model(payload, original_model or None) + return web.json_response(payload) + + async def _cursor_passthrough( + self, + request: web.Request, + body: dict[str, Any], + response_model_override: str | None = None, + upstream_model: str | None = None, + force_non_stream: bool = False, + ) -> web.StreamResponse: + """Route Composer through cursor-agent using Cursor subscription login.""" + if not cursor_passthrough_available(): + raise web.HTTPUnauthorized( + text="Cursor subscription auth unavailable. Run `cursor-agent login`, then retry." + ) + slug = response_model_override or CURSOR_MODEL_SLUG + upstream = upstream_model or cursor_upstream_model(slug) + prompt = build_cursor_prompt(body) + stream = bool(body.get("stream")) and not force_non_stream + + if not stream: + text = "" + usage: dict[str, Any] | None = None + async for event in iter_cursor_agent_events(prompt, upstream): + if event["type"] == "completed": + text = str(event.get("text") or text) + elif event["type"] == "usage": + usage = event.get("usage") if isinstance(event.get("usage"), dict) else None + elif event["type"] == "error": + raise web.HTTPBadGateway(text=str(event.get("message") or "cursor-agent failed")) + payload: dict[str, Any] = { + "id": f"resp_{int(time.time() * 1000)}", + "object": "response", + "model": slug, + "status": "completed", + "output": [ + { + "id": "msg_0", + "type": "message", + "status": "completed", + "role": "assistant", + "content": [{"type": "output_text", "text": text, "annotations": []}], + } + ], + } + normalized_usage = normalize_responses_usage(usage) + if normalized_usage: + payload["usage"] = normalized_usage + return web.json_response(payload) + + response = _sse_response() + await response.prepare(request) + tool_types = _build_tool_types(body) + state = ResponsesStreamState(slug, tool_types) + try: + await state.start(response) + async for event in iter_cursor_agent_events(prompt, upstream): + if event["type"] == "text_delta": + await state.write_chat_delta( + response, + {"choices": [{"delta": {"content": event["delta"]}}]}, + ) + elif event["type"] == "usage": + normalized_usage = normalize_responses_usage(event.get("usage")) + if normalized_usage: + state.usage = normalized_usage + elif event["type"] == "error": + message = str(event.get("message") or "cursor-agent failed") + await state.write_chat_delta( + response, + {"choices": [{"delta": {"content": message}}]}, + ) + break + await state.finish(response) + except ClientDisconnected: + pass + except Exception as exc: + print(f"[err] cursor passthrough {slug}: {exc}", flush=True) + raise web.HTTPBadGateway(text=str(exc)) from exc + try: + await response.write_eof() + except Exception: + pass + return response + + # ------------------------------------------------------------------ + # Auto Router + # ------------------------------------------------------------------ + def _active_router(self): + """Return the RouterConfig only when enabled and at least one candidate + backend is usable, so discovery never advertises a dead Auto entry.""" + config = self.settings.load_router() + if config and router_module.router_is_active(config, available_model_slugs(self.settings.load())): + return config + return None + + async def _maybe_apply_auto_router(self, body: dict[str, Any]) -> dict[str, Any]: + """If the request targets the Auto Router slug, classify the task and + rewrite ``model`` to the concrete backend that should handle it. Any + failure leaves the body untouched so the request still routes normally.""" + config = self.settings.load_router() + if not config or not config.effective_enabled: + return body + if str(body.get("model") or "") != config.slug: + return body + resolved = await self._resolve_auto_model(config, body) + if resolved and resolved != config.slug: + if router_module.router_log_enabled(): + print(f"[router] {config.slug} -> {resolved}", flush=True) + new_body = dict(body) + new_body["model"] = resolved + return new_body + return body + + async def _resolve_auto_model(self, config, body: dict[str, Any]) -> str | None: + models = self.settings.load() + candidates = router_module.filter_available(config, available_model_slugs(models)) + if not candidates: + return None + classify = None + if config.classifier: + classifier_model = self.settings.by_slug_or_model(config.classifier) + if ( + classifier_model is not None + and byok_model_has_credentials(classifier_model) + and (classifier_model.is_openai_chat or classifier_model.is_anthropic) + ): + classify = self._make_classifier(classifier_model, config) + log = (lambda message: print(message, flush=True)) if router_module.router_log_enabled() else None + resolved, _info = await router_module.resolve_auto(config, candidates, body, classify, log=log) + return resolved or router_module.fallback_slug( + config, candidates, has_image_task=router_module.has_images(body) + ) + + def _make_classifier(self, model: ShimModel, config): + timeout = ClientTimeout(total=config.timeout + 5, sock_connect=config.timeout, sock_read=config.timeout) + + async def classify(system_prompt: str, user_content: str) -> str: + async with ClientSession(timeout=timeout) as session: + if model.is_anthropic: + url = _join_url(model.base_url, "/messages") + payload = { + "model": model.model, + "max_tokens": config.max_tokens, + "system": system_prompt, + "messages": [{"role": "user", "content": user_content}], + } + upstream = await session.post(url, json=payload, headers=_anthropic_headers(model)) + upstream.raise_for_status() + data = await upstream.json(content_type=None) + return _anthropic_text(data) + url = _join_url(model.base_url, "/chat/completions") + payload = { + "model": model.model, + "stream": False, + "temperature": 0, + "max_tokens": config.max_tokens, + "messages": [ + {"role": "system", "content": system_prompt}, + {"role": "user", "content": user_content}, + ], + } + upstream = await session.post(url, json=payload, headers=_openai_headers(model)) + upstream.raise_for_status() + data = await upstream.json(content_type=None) + message = (data.get("choices") or [{}])[0].get("message") or {} + return str(message.get("content") or "") + + return classify + def _route(self, body: dict[str, Any]) -> ShimModel: requested = str(body.get("model") or "") route = self.settings.by_slug_or_model(requested) if route is None: raise web.HTTPNotFound(text=f"Unknown model slug/model: {requested}") + if not byok_model_has_credentials(route): + raise web.HTTPUnauthorized(text=_missing_api_key_message(route)) return route async def _post_openai_chat( @@ -328,17 +752,36 @@ async def _post_openai_chat( ) -> web.StreamResponse: url = _join_url(route.base_url, "/chat/completions") headers = _openai_headers(route) + _dump_debug_request(route.slug, url, body) async with ClientSession(timeout=self.timeout) as session: upstream = await session.post(url, json=body, headers=headers) if upstream.status >= 400: - return await _error_response(upstream) + return await _error_response(upstream, slug=route.slug) if body.get("stream"): - return await self._stream_openai_chat(request, upstream, route, as_responses) + return await self._stream_openai_chat(request, upstream, route, as_responses, body) payload = await upstream.json(content_type=None) if as_responses: - return web.json_response(chat_completion_to_response(payload, route.slug)) + tool_types = _build_tool_types(body) + payload = chat_completion_to_response(payload, route.slug, tool_types) + intercepted = _maybe_intercept_web_search(payload) + return web.json_response(intercepted or payload) return web.json_response(payload) + async def _post_openai_chat_as_anthropic( + self, request: web.Request, route: ShimModel, body: dict[str, Any] + ) -> web.StreamResponse: + url = _join_url(route.base_url, "/chat/completions") + headers = _openai_headers(route) + _dump_debug_request(route.slug, url, body) + async with ClientSession(timeout=self.timeout) as session: + upstream = await session.post(url, json=body, headers=headers) + if upstream.status >= 400: + return await _anthropic_error_response(upstream) + if body.get("stream"): + return await self._stream_openai_chat_as_anthropic(request, upstream, route) + payload = await upstream.json(content_type=None) + return web.json_response(chat_completion_to_anthropic_message(payload, route.slug)) + async def _post_anthropic( self, request: web.Request, route: ShimModel, body: dict[str, Any], as_responses: bool ) -> web.StreamResponse: @@ -349,19 +792,39 @@ async def _post_anthropic( if upstream.status >= 400: return await _error_response(upstream) if body.get("stream"): - return await self._stream_anthropic(request, upstream, route, as_responses) + return await self._stream_anthropic(request, upstream, route, as_responses, body) payload = await upstream.json(content_type=None) if as_responses: - return web.json_response(anthropic_to_response(payload, route.slug)) + tool_types = _build_tool_types(body) + payload = anthropic_to_response(payload, route.slug, tool_types) + intercepted = _maybe_intercept_web_search(payload) + return web.json_response(intercepted or payload) return web.json_response(anthropic_to_chat_response(payload, route.slug)) + async def _post_anthropic_messages( + self, request: web.Request, route: ShimModel, body: dict[str, Any] + ) -> web.StreamResponse: + url = _join_url(route.base_url, "/messages") + headers = _anthropic_headers(route) + async with ClientSession(timeout=self.timeout) as session: + upstream = await session.post(url, json=body, headers=headers) + if upstream.status >= 400: + return await _error_response(upstream, slug=route.slug) + if body.get("stream"): + return await self._stream_raw_sse(request, upstream, route.slug) + payload = await upstream.json(content_type=None) + if isinstance(payload, dict): + payload["model"] = route.slug + return web.json_response(payload) + async def _stream_openai_chat( - self, request: web.Request, upstream, route: ShimModel, as_responses: bool + self, request: web.Request, upstream, route: ShimModel, as_responses: bool, body: dict[str, Any] | None = None ) -> web.StreamResponse: response = _sse_response() await response.prepare(request) if as_responses: - state = ResponsesStreamState(route.slug) + tool_types = _build_tool_types(body) if body else {} + state = ResponsesStreamState(route.slug, tool_types) try: if as_responses: await state.start(response) @@ -390,13 +853,41 @@ async def _stream_openai_chat( pass return response + async def _stream_openai_chat_as_anthropic( + self, request: web.Request, upstream, route: ShimModel + ) -> web.StreamResponse: + response = _sse_response() + await response.prepare(request) + state = AnthropicMessagesStreamState(route.slug) + try: + await state.start(response) + async for line in _sse_lines(upstream): + if line == "[DONE]": + break + try: + event = json.loads(line) + except json.JSONDecodeError: + continue + await state.write_chat_delta(response, event) + await state.finish(response) + except ClientDisconnected: + pass + finally: + upstream.release() + try: + await response.write_eof() + except Exception: + pass + return response + async def _stream_anthropic( - self, request: web.Request, upstream, route: ShimModel, as_responses: bool + self, request: web.Request, upstream, route: ShimModel, as_responses: bool, body: dict[str, Any] | None = None ) -> web.StreamResponse: response = _sse_response() await response.prepare(request) if as_responses: - state = ResponsesStreamState(route.slug) + tool_types = _build_tool_types(body) if body else {} + state = ResponsesStreamState(route.slug, tool_types) try: if as_responses: await state.start(response) @@ -425,6 +916,33 @@ async def _stream_anthropic( pass return response + async def _stream_raw_sse(self, request: web.Request, upstream, model_slug: str | None = None) -> web.StreamResponse: + response = _sse_response() + await response.prepare(request) + try: + async for line in _sse_lines(upstream): + if model_slug and line.startswith("{"): + try: + event = json.loads(line) + if isinstance(event, dict) and event.get("type") == "message_start": + msg = event.get("message") + if isinstance(msg, dict): + msg["model"] = model_slug + await _write_anthropic_sse(response, event.get("type", "message"), event) + continue + except json.JSONDecodeError: + pass + await _safe_write(response, f"data: {line}\n\n".encode()) + except ClientDisconnected: + pass + finally: + upstream.release() + try: + await response.write_eof() + except Exception: + pass + return response + _DROP_ITEM = object() @@ -474,6 +992,223 @@ def _rewrite_response_model(payload: Any, model: str | None) -> None: _rewrite_response_model(item, model) +class AnthropicMessagesStreamState: + """Translates OpenAI chat-completions chunks into Anthropic Messages SSE.""" + + def __init__(self, model: str): + self.message_id = f"msg_{uuid.uuid4().hex[:24]}" + self.model = model + self.next_index = 0 + self.text_index: int | None = None + self.reasoning_index: int | None = None + self.text_open = False + self.reasoning_open = False + self.tool_calls: dict[int, dict[str, Any]] = {} + self.usage: dict[str, Any] | None = None + self.stop_reason = "end_turn" + + async def start(self, response: web.StreamResponse) -> None: + await _write_anthropic_sse( + response, + "message_start", + { + "type": "message_start", + "message": { + "id": self.message_id, + "type": "message", + "role": "assistant", + "model": self.model, + "content": [], + "stop_reason": None, + "stop_sequence": None, + "usage": {"input_tokens": 0, "output_tokens": 0}, + }, + }, + ) + + async def write_chat_delta(self, response: web.StreamResponse, chunk: dict[str, Any]) -> None: + usage = chunk.get("usage") + if isinstance(usage, dict): + self.usage = normalize_responses_usage(usage) + choice = (chunk.get("choices") or [{}])[0] + finish_reason = choice.get("finish_reason") + if finish_reason: + self.stop_reason = _chat_finish_to_anthropic_stop(finish_reason) + delta = choice.get("delta") or {} + reasoning = delta.get("reasoning_content") or delta.get("reasoning") + if reasoning: + await self._reasoning_delta(response, str(reasoning)) + content = delta.get("content") + if content: + if self.reasoning_open: + await self._close_reasoning(response) + await self._text_delta(response, str(content)) + for call in delta.get("tool_calls") or []: + await self._tool_delta(response, call) + + async def finish(self, response: web.StreamResponse) -> None: + if self.reasoning_open: + await self._close_reasoning(response) + if self.text_open: + await self._close_text(response) + for index in sorted(self.tool_calls): + state = self.tool_calls[index] + if not state.get("closed"): + await self._close_tool(response, index, state) + await _write_anthropic_sse( + response, + "message_delta", + { + "type": "message_delta", + "delta": {"stop_reason": self.stop_reason, "stop_sequence": None}, + "usage": _responses_usage_to_anthropic_usage(self.usage) or {"output_tokens": 0}, + }, + ) + await _write_anthropic_sse(response, "message_stop", {"type": "message_stop"}) + + async def _text_delta(self, response: web.StreamResponse, text: str) -> None: + if self.text_index is None: + self.text_index = self.next_index + self.next_index += 1 + self.text_open = True + await _write_anthropic_sse( + response, + "content_block_start", + {"type": "content_block_start", "index": self.text_index, "content_block": {"type": "text", "text": ""}}, + ) + await _write_anthropic_sse( + response, + "content_block_delta", + {"type": "content_block_delta", "index": self.text_index, "delta": {"type": "text_delta", "text": text}}, + ) + + async def _close_text(self, response: web.StreamResponse) -> None: + if self.text_index is None: + return + await _write_anthropic_sse(response, "content_block_stop", {"type": "content_block_stop", "index": self.text_index}) + self.text_index = None + self.text_open = False + + async def _reasoning_delta(self, response: web.StreamResponse, text: str) -> None: + if self.reasoning_index is None: + self.reasoning_index = self.next_index + self.next_index += 1 + self.reasoning_open = True + await _write_anthropic_sse( + response, + "content_block_start", + { + "type": "content_block_start", + "index": self.reasoning_index, + "content_block": {"type": "thinking", "thinking": ""}, + }, + ) + await _write_anthropic_sse( + response, + "content_block_delta", + { + "type": "content_block_delta", + "index": self.reasoning_index, + "delta": {"type": "thinking_delta", "thinking": text}, + }, + ) + + async def _close_reasoning(self, response: web.StreamResponse) -> None: + if self.reasoning_index is None: + return + await _write_anthropic_sse( + response, + "content_block_stop", + {"type": "content_block_stop", "index": self.reasoning_index}, + ) + self.reasoning_index = None + self.reasoning_open = False + + async def _tool_delta(self, response: web.StreamResponse, call: dict[str, Any]) -> None: + index = int(call.get("index", 0)) + fn = call.get("function") or {} + state = self.tool_calls.setdefault( + index, + { + "id": "", + "name": "", + "arguments": "", + "emitted": 0, + "block_index": None, + "open": False, + "closed": False, + }, + ) + if call.get("id"): + state["id"] = call["id"] + if fn.get("name"): + state["name"] += fn["name"] + if fn.get("arguments"): + state["arguments"] += fn["arguments"] + if not state["open"] and state["name"]: + if self.reasoning_open: + await self._close_reasoning(response) + if self.text_open: + await self._close_text(response) + await self._open_tool(response, index, state) + if state["open"] and len(state["arguments"]) > state["emitted"]: + delta = state["arguments"][state["emitted"] :] + state["emitted"] = len(state["arguments"]) + await _write_anthropic_sse( + response, + "content_block_delta", + { + "type": "content_block_delta", + "index": state["block_index"], + "delta": {"type": "input_json_delta", "partial_json": delta}, + }, + ) + + async def _open_tool(self, response: web.StreamResponse, index: int, state: dict[str, Any]) -> None: + state["block_index"] = self.next_index + self.next_index += 1 + state["open"] = True + if not state["id"]: + state["id"] = f"call_{index}" + await _write_anthropic_sse( + response, + "content_block_start", + { + "type": "content_block_start", + "index": state["block_index"], + "content_block": { + "type": "tool_use", + "id": state["id"], + "name": state["name"] or "tool", + "input": {}, + }, + }, + ) + + async def _close_tool(self, response: web.StreamResponse, index: int, state: dict[str, Any]) -> None: + if not state["open"]: + await self._open_tool(response, index, state) + if len(state["arguments"]) > state["emitted"]: + delta = state["arguments"][state["emitted"] :] + state["emitted"] = len(state["arguments"]) + await _write_anthropic_sse( + response, + "content_block_delta", + { + "type": "content_block_delta", + "index": state["block_index"], + "delta": {"type": "input_json_delta", "partial_json": delta}, + }, + ) + await _write_anthropic_sse( + response, + "content_block_stop", + {"type": "content_block_stop", "index": state["block_index"]}, + ) + state["open"] = False + state["closed"] = True + + class ResponsesStreamState: """Translates upstream chat-completions / anthropic stream events into the Codex Desktop Responses-API event sequence. Keeps the message item and @@ -481,21 +1216,22 @@ class ResponsesStreamState: proper .added / .delta / .done / .completed events plus a final `response.completed` with the full reconciled `output` array.""" - def __init__(self, model: str): + def __init__(self, model: str, tool_types: dict[str, str] | None = None): self.response_id = f"resp_{int(time.time() * 1000)}" self.message_item_id = f"msg_{int(time.time() * 1000)}" self.model = model - self.message_index: int | None = None # output_index for the assistant message + self.message_index: int | None = None self.message_text = "" self.message_opened = False self.message_closed = False - # Tool call state, keyed by upstream "index" (chat-completions) or - # anthropic content_block_index. Each entry tracks its assigned - # output_index, accumulated arguments, name, etc. + self.usage: dict[str, Any] | None = None self.tool_calls: dict[int, dict[str, Any]] = {} - # Reasoning (extended thinking) blocks, keyed by upstream index. self.reasoning_blocks: dict[Any, dict[str, Any]] = {} self.next_output_index = 0 + # Map sanitized tool name -> original Responses tool type so we can + # emit the correct output item type (e.g. custom_tool_call for freeform + # apply_patch instead of generic function_call). + self.tool_types = tool_types or {} # ------------------------------------------------------------------ # Lifecycle @@ -519,6 +1255,9 @@ async def finish(self, response: web.StreamResponse) -> None: # Chat-completions (OpenAI-style) deltas # ------------------------------------------------------------------ async def write_chat_delta(self, response: web.StreamResponse, chunk: dict[str, Any]) -> None: + usage = chunk.get("usage") + if isinstance(usage, dict): + self.usage = normalize_responses_usage(usage) choice = (chunk.get("choices") or [{}])[0] delta = choice.get("delta") or {} reasoning = delta.get("reasoning_content") or delta.get("reasoning") @@ -577,6 +1316,11 @@ async def _chat_tool_delta(self, response: web.StreamResponse, call: dict[str, A # ------------------------------------------------------------------ async def write_anthropic_delta(self, response: web.StreamResponse, event: dict[str, Any]) -> None: event_type = event.get("type") + if event_type == "message_start": + message = event.get("message") or {} + usage = message.get("usage") + if isinstance(usage, dict): + self.usage = normalize_responses_usage(usage) if event_type == "content_block_start": block = event.get("content_block") or {} idx = int(event.get("index", 0)) @@ -644,6 +1388,22 @@ async def write_anthropic_delta(self, response: web.StreamResponse, event: dict[ if state is None: state = await self._open_reasoning(response, key=("anthropic_thinking", idx)) state["signature"] += delta.get("signature") or "" + elif event_type == "message_delta": + usage = event.get("usage") + if isinstance(usage, dict): + if self.usage is None or any( + key in usage for key in ("input_tokens", "prompt_tokens", "cache_read_input_tokens", "cache_creation_input_tokens") + ): + normalized = normalize_responses_usage(usage) + if normalized is not None: + self.usage = normalized if self.usage is None else {**self.usage, **normalized} + output_tokens = usage.get("output_tokens") + if isinstance(output_tokens, int) and not isinstance(output_tokens, bool): + if self.usage is None: + self.usage = normalize_responses_usage(usage) + else: + self.usage["output_tokens"] = output_tokens + self.usage["total_tokens"] = int(self.usage.get("input_tokens") or 0) + output_tokens elif event_type == "content_block_stop": idx = int(event.get("index", 0)) tool_state = self.tool_calls.get(("anthropic", idx)) @@ -742,6 +1502,15 @@ async def _open_tool(self, response: web.StreamResponse, *, key: Any, call_id: s await self._close_message(response) output_index = self.next_output_index self.next_output_index += 1 + # Determine output item type based on original tool type. + # Freeform tools (apply_patch with no schema) emit custom_tool_call + # so Codex Desktop knows not to validate against a fixed enum. + original_type = self.tool_types.get(name, "") + output_type = "function_call" + if original_type == "apply_patch": + output_type = "custom_tool_call" + elif original_type.startswith("web_search"): + output_type = "web_search_call" state: dict[str, Any] = { "id": call_id, "call_id": call_id, @@ -749,6 +1518,7 @@ async def _open_tool(self, response: web.StreamResponse, *, key: Any, call_id: s "arguments": "", "output_index": output_index, "closed": False, + "output_type": output_type, } self.tool_calls[key] = state await _write_sse( @@ -758,7 +1528,7 @@ async def _open_tool(self, response: web.StreamResponse, *, key: Any, call_id: s "output_index": output_index, "item": { "id": call_id, - "type": "function_call", + "type": output_type, "status": "in_progress", "call_id": call_id, "name": name, @@ -903,7 +1673,7 @@ def _message_item(self, status: str) -> dict[str, Any]: def _tool_item(self, state: dict[str, Any], status: str) -> dict[str, Any]: return { "id": state["id"], - "type": "function_call", + "type": state.get("output_type", "function_call"), "status": status, "call_id": state["call_id"], "name": state["name"], @@ -922,7 +1692,7 @@ def _response(self, status: str, *, final: bool = False) -> dict[str, Any]: collected.append((state["output_index"], self._tool_item(state, "completed"))) collected.sort(key=lambda pair: pair[0]) output = [item for _, item in collected] - return { + payload = { "id": self.response_id, "object": "response", "created_at": int(time.time()), @@ -930,6 +1700,11 @@ def _response(self, status: str, *, final: bool = False) -> dict[str, Any]: "model": self.model, "output": output, } + if self.usage is not None: + payload["usage"] = self.usage + elif final: + payload["usage"] = {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0} + return payload _THINKING_MAGIC = "anthropic-thinking-v1:" @@ -956,9 +1731,208 @@ def _decode_thinking_payload(encoded: str) -> dict[str, Any] | None: return data if isinstance(data, dict) else None +def _build_tool_types(body: dict[str, Any]) -> dict[str, str]: + """Build a map sanitized tool name -> original tool type from the request tools array. + + Codex Desktop emits native tools like `{"type": "apply_patch"}` and MCP tools + like `{"type": "mcp__node_repl", "function": {"name": "js"}}`. When we translate + those into chat-completions `function` tools, the original type is lost. We + preserve it here so the Responses streaming translator can emit the correct + output item type (e.g. `custom_tool_call` for freeform apply_patch instead of + generic `function_call`). + """ + tool_types: dict[str, str] = {} + for tool in body.get("tools") or []: + if not isinstance(tool, dict): + continue + tool_type = str(tool.get("type") or "").strip().lower() + fn = tool.get("function") + if isinstance(fn, dict) and fn.get("name"): + name = str(fn["name"]).strip() + elif tool.get("name"): + name = str(tool["name"]).strip() + else: + name = tool_type + clean = re.sub(r"[^a-zA-Z0-9_-]+", "_", name.strip())[:64].strip("_") + if clean: + tool_types[clean] = tool_type + return tool_types + +async def _perform_web_search(query: str) -> str: + """Execute a web search via DuckDuckGo and return text results. + + This is a server-side fallback for custom models whose provider does not + have a native web-search capability. Codex Desktop expects the shim to + return results as a `function_call_output` (or `web_search_call`) item; + when the model is BYOK, the Desktop app does not execute the search itself, + so the shim must do it and feed the results back into the conversation. + """ + import urllib.parse + import urllib.request + + if not query or not query.strip(): + return "No search query provided." + + # DuckDuckGo lite HTML endpoint (no API key required) + url = ( + "https://html.duckduckgo.com/html/" + + "?q=" + + urllib.parse.quote_plus(query.strip()) + ) + req = urllib.request.Request( + url, + headers={ + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36" + }, + ) + try: + with urllib.request.urlopen(req, timeout=10) as resp: + html = resp.read().decode("utf-8", errors="replace") + except Exception as exc: + return f"Web search failed: {exc}" + + # Extract title + snippet from result links + results: list[str] = [] + # Each result is in a `.result` div with `.result__a` (title/link) and `.result__snippet` + from html.parser import HTMLParser + + class _ResultParser(HTMLParser): + def __init__(self) -> None: + super().__init__() + self.in_result = False + self.in_a = False + self.in_snippet = False + self.current_title = "" + self.current_snippet = "" + self.results: list[dict[str, str]] = [] + self._tag_stack: list[str] = [] + self._class_stack: list[str] = [] + + def _current_class(self) -> str: + return self._class_stack[-1] if self._class_stack else "" + + def handle_starttag(self, tag: str, attrs_list: list[tuple[str, str | None]]) -> None: + attrs = dict(attrs_list) + cls = (attrs.get("class") or "").lower() + self._tag_stack.append(tag) + self._class_stack.append(cls) + if "result" in cls and tag == "div": + self.in_result = True + self.current_title = "" + self.current_snippet = "" + if self.in_result and tag == "a" and "result__a" in cls: + self.in_a = True + if self.in_result and ("result__snippet" in cls or "result__body" in cls): + self.in_snippet = True + + def handle_endtag(self, tag: str) -> None: + if self._tag_stack and self._tag_stack[-1] == tag: + self._tag_stack.pop() + self._class_stack.pop() + if tag == "div" and self.in_result: + if self.current_title or self.current_snippet: + self.results.append( + { + "title": self.current_title.strip(), + "snippet": self.current_snippet.strip(), + } + ) + self.in_result = False + if tag == "a": + self.in_a = False + if tag in {"div", "span", "p"}: + self.in_snippet = False + + def handle_data(self, data: str) -> None: + if self.in_a: + self.current_title += data + if self.in_snippet: + self.current_snippet += data + + parser = _ResultParser() + parser.feed(html) + for r in parser.results[:5]: + title = r["title"].replace("\n", " ") + snippet = r["snippet"].replace("\n", " ") + if title and snippet: + results.append(f"{title}\n{snippet}") + elif title: + results.append(title) + elif snippet: + results.append(snippet) + + if not results: + return "No web search results found." + return "\n\n".join(results) + +def _maybe_intercept_web_search(payload: dict[str, Any]) -> dict[str, Any] | None: + """If the response payload contains a web_search_call, execute it server-side + and return a new payload with the results embedded as a function_call_output. + + Returns None if no web_search_call is present (pass through unchanged). + """ + output = payload.get("output") or [] + if not isinstance(output, list): + return None + search_calls: list[tuple[int, dict[str, Any]]] = [] + for i, item in enumerate(output): + if isinstance(item, dict) and item.get("type") == "web_search_call": + search_calls.append((i, item)) + if not search_calls: + return None + + # Build synthetic search results + results: list[dict[str, Any]] = [] + for idx, call in search_calls: + try: + args = json.loads(call.get("arguments") or "{}") + except json.JSONDecodeError: + args = {} + query = args.get("query") or "" + # Run the search synchronously (non-streaming path only) + import asyncio + try: + loop = asyncio.get_running_loop() + result_text = loop.run_until_complete(_perform_web_search(query)) + except RuntimeError: + result_text = "Web search unavailable in this context." + results.append({ + "id": f"wso_{call.get('call_id', '0')}", + "type": "function_call_output", + "status": "completed", + "call_id": call.get("call_id"), + "output": result_text, + }) + + # Replace web_search_call items with their results + new_output: list[dict[str, Any]] = [] + for i, item in enumerate(output): + if isinstance(item, dict) and item.get("type") == "web_search_call": + # Find matching result + for r in results: + if r.get("call_id") == item.get("call_id"): + new_output.append(r) + break + else: + new_output.append(item) + else: + new_output.append(item) + + new_payload = dict(payload) + new_payload["output"] = new_output + return new_payload + + +_VERSIONED_BASE_RE = re.compile(r"(?:^|/)v\d+$") + + def _join_url(base_url: str, endpoint: str) -> str: base = base_url.rstrip("/") - if base.endswith("/v1"): + if _VERSIONED_BASE_RE.search(base): + # Already ends with /v (e.g. /v1, /api/coding/v3) — append + # the endpoint as-is rather than injecting another /v1/. return base + endpoint if endpoint == "/messages": return base + "/v1/messages" @@ -980,10 +1954,20 @@ def _anthropic_headers(route: ShimModel) -> dict[str, str]: } if route.api_key: headers.setdefault("x-api-key", route.api_key) - headers.setdefault("Authorization", f"Bearer {route.api_key}") return headers +def _anthropic_text(payload: Any) -> str: + if not isinstance(payload, dict): + return "" + parts = [ + str(block.get("text") or "") + for block in (payload.get("content") or []) + if isinstance(block, dict) and block.get("type") == "text" + ] + return "".join(parts) + + def _sse_response() -> web.StreamResponse: response = web.StreamResponse( status=200, @@ -1028,6 +2012,22 @@ async def _write_sse(response: web.StreamResponse, payload: dict[str, Any]) -> N raise +async def _write_anthropic_sse(response: web.StreamResponse, event: str, payload: dict[str, Any]) -> None: + data = json.dumps(payload, separators=(",", ":")) + try: + await response.write(f"event: {event}\ndata: {data}\n\n".encode()) + except (ConnectionResetError, ConnectionError) as exc: + raise ClientDisconnected() from exc + except Exception as exc: + if exc.__class__.__name__ in { + "ClientConnectionResetError", + "ClientConnectionError", + "ClientPayloadError", + }: + raise ClientDisconnected() from exc + raise + + class ClientDisconnected(Exception): """Raised when the downstream Codex client closes the SSE connection.""" @@ -1086,11 +2086,132 @@ def _anthropic_stream_to_chat_chunk(event: dict[str, Any], model: str) -> dict[s return {"object": "chat.completion.chunk", "model": model, "choices": [{"index": 0, "delta": {"content": content}, "finish_reason": None}]} -async def _error_response(upstream) -> web.Response: +def _compact_request_body(body: dict[str, Any], upstream_model: str) -> dict[str, Any]: + instructions = body.get("instructions") or _default_compact_instructions() + return { + "model": upstream_model, + "instructions": instructions, + "input": body.get("input") or [], + "max_output_tokens": body.get("max_output_tokens") or body.get("max_tokens") or 4096, + "stream": False, + } + + +def _default_compact_instructions() -> str: + return ( + "Compact the conversation into a concise state handoff for the next Codex turn. " + "Preserve the active task, user requirements, important file paths, commands already run, " + "tool results, decisions, blockers, and the latest state. Omit filler and repeated text." + ) + + +async def _as_compact_response(response: web.StreamResponse, model: str) -> web.Response: + if not isinstance(response, web.Response) or response.status >= 400: + return response + try: + payload = json.loads(response.text or "{}") + except json.JSONDecodeError: + return response + output = payload.get("output") if isinstance(payload, dict) else None + summary = _compact_summary_from_output(output) + compacted = _compact_response_payload(model, summary, payload.get("usage") if isinstance(payload, dict) else None) + return web.json_response(compacted) + + +def _compact_summary_from_output(output: Any) -> str: + parts: list[str] = [] + if isinstance(output, list): + for item in output: + if not isinstance(item, dict): + continue + if item.get("type") == "message": + content = item.get("content") or [] + if isinstance(content, list): + for part in content: + if isinstance(part, dict) and part.get("text"): + parts.append(str(part["text"])) + elif item.get("type") == "output_text" and item.get("text"): + parts.append(str(item["text"])) + return "\n".join(part for part in parts if part).strip() + + +def _compact_response_payload(model: str, summary: str, usage: Any = None) -> dict[str, Any]: + now = int(time.time()) + response_id = f"resp_compact_{now}" + text = summary or "No prior conversation state was available to compact." + payload = { + "id": response_id, + "object": "response", + "created_at": now, + "status": "completed", + "model": model, + "output": [ + { + "id": f"msg_compact_{now}", + "type": "message", + "status": "completed", + "role": "assistant", + "content": [{"type": "output_text", "text": text, "annotations": []}], + } + ], + } + if usage is not None: + payload["usage"] = usage + return payload + + +async def _error_response(upstream, *, slug: str | None = None) -> web.Response: text = await upstream.text() + if slug: + print( + f"[err] upstream {slug} returned {upstream.status}: {text[:500]}", + flush=True, + ) return web.Response(status=upstream.status, text=text, content_type=upstream.content_type or "text/plain") +async def _anthropic_error_response(upstream) -> web.Response: + text = await upstream.text() + message = text + error_type = "api_error" + try: + payload = json.loads(text) + except json.JSONDecodeError: + payload = None + if isinstance(payload, dict): + err = payload.get("error") + if isinstance(err, dict): + message = str(err.get("message") or message) + error_type = str(err.get("type") or error_type) + elif payload.get("message"): + message = str(payload["message"]) + status_type = { + 400: "invalid_request_error", + 401: "authentication_error", + 403: "permission_error", + 404: "not_found_error", + 413: "request_too_large", + 429: "rate_limit_error", + }.get(upstream.status) + if status_type: + error_type = status_type + body = { + "type": "error", + "error": {"type": error_type, "message": message}, + } + request_id = upstream.headers.get("request-id") or upstream.headers.get("x-request-id") + if request_id: + body["request_id"] = request_id + return web.json_response(body, status=upstream.status) + + +def _missing_api_key_message(route: ShimModel) -> str: + env_name = route.raw.get("api_key_env") or route.raw.get("apiKeyEnv") + if env_name: + return f"Model {route.slug} has no API key. Set {env_name} or add api_key/apiKey for this model." + return f"Model {route.slug} has no API key. Add api_key/apiKey or api_key_env/apiKeyEnv for this model." + + def _normalize_roles(messages: list[dict]) -> list[dict]: result = [] for message in messages: @@ -1102,6 +2223,244 @@ def _normalize_roles(messages: list[dict]) -> list[dict]: return result +def _dump_debug_request(slug: str, url: str, body: dict[str, Any]) -> None: + """Best-effort dump of the last forwarded request body for debugging. + + Writes ``.codex-shim/last_request.json`` next to the rest of the runtime + state (catalog, pid, log). Failures are silently swallowed — this is a + debug aid, not a code path the request should depend on. + """ + try: + dump_path = DEBUG_DIR / "last_request.json" + dump_path.parent.mkdir(parents=True, exist_ok=True) + payload = {"slug": slug, "url": url, "body": body} + full = json.dumps(payload, indent=2, default=str) + if len(full) > 2_000_000: + messages = body.get("messages") or [] + summary = { + "slug": slug, + "url": url, + "_truncated": True, + "_full_size": len(full), + "message_count": len(messages), + "tool_count": len(body.get("tools") or []), + "last_3_messages": messages[-3:], + } + dump_path.write_text(json.dumps(summary, indent=2, default=str)) + else: + dump_path.write_text(full) + except OSError as exc: + print(f"[debug] dump failed: {exc}", flush=True) + + +def _current_managed_model() -> str | None: + """Return the first ``model = "..."`` value from ~/.codex/config.toml.""" + if not CODEX_CONFIG_PATH.exists(): + return None + try: + text = CODEX_CONFIG_PATH.read_text() + except OSError: + return None + for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith("model = "): + return stripped.split("=", 1)[1].strip().strip('"') + return None + + +_MODEL_LINE_RE = re.compile(r'(?m)^(\s*model\s*=\s*")[^"]*(")') +_PROVIDER_NAME_RE = re.compile( + r'(\[model_providers\.' + re.escape(PROVIDER_NAME) + r'\][^\[]*?\n\s*name\s*=\s*")[^"]*(")', + re.DOTALL, +) + + +def _set_active_model(slug: str, display_name: str | None = None) -> None: + """Rewrite the active model + provider label in ~/.codex/config.toml.""" + if not CODEX_CONFIG_PATH.exists(): + return + try: + text = CODEX_CONFIG_PATH.read_text() + except OSError: + return + text = _MODEL_LINE_RE.sub(rf'\g<1>{slug}\g<2>', text, count=1) + if display_name: + text = _PROVIDER_NAME_RE.sub(rf'\g<1>{display_name}\g<2>', text, count=1) + try: + CODEX_CONFIG_PATH.write_text(text) + except OSError as exc: + print(f"[switch] failed to write {CODEX_CONFIG_PATH}: {exc}", flush=True) + return + print(f"[switch] set active model to {slug} ({display_name})", flush=True) + + +def _restart_codex_app() -> None: + """Quit and relaunch Codex Desktop in a background thread (non-blocking). + + Cross-platform: ``taskkill`` + ``Codex.exe`` on Windows, ``osascript`` + + ``open -a Codex`` on macOS. Linux has no Codex Desktop build today, so + the branch is a no-op there. + """ + import os as _os + import subprocess as _subprocess + import threading as _threading + import time as _time + + def _do_restart() -> None: + try: + if _os.name == "nt": + _subprocess.run( + ["taskkill", "/IM", "Codex.exe", "/F"], + check=False, + stdout=_subprocess.DEVNULL, + stderr=_subprocess.DEVNULL, + ) + _time.sleep(1.5) + local_appdata = _os.environ.get("LOCALAPPDATA", "") + codex_exe = Path(local_appdata) / "Programs" / "Codex" / "Codex.exe" + if codex_exe.exists(): + _subprocess.Popen([str(codex_exe)]) + else: + _subprocess.Popen(["Codex.exe"], shell=True) + elif sys.platform == "darwin": + quit_script = 'tell application "Codex" to if it is running then quit' + _subprocess.run( + ["osascript", "-e", quit_script], + check=False, + stdout=_subprocess.DEVNULL, + stderr=_subprocess.DEVNULL, + ) + _time.sleep(1.5) + _subprocess.Popen(["open", "-a", "Codex"]) + except OSError: + pass + + _threading.Thread(target=_do_restart, daemon=True).start() + + +def _picker_html(picker_token: str) -> str: + token_json = json.dumps(picker_token).replace("<", "\\u003c") + html = ''' + + + + +Codex Shim - Model Picker + + + +
+

Model Picker

+

Choose the active model for Codex Desktop

+
Loading models...
+
+ + +
+
+

Codex needs to restart to use the new model

+
+ + +''' + return ( + html.replace("@@TOKEN_JSON@@", token_json, 1).replace("@@PICKER_HEADER@@", PICKER_TOKEN_HEADER, 1) + ) + + def main(argv: list[str] | None = None) -> None: parser = argparse.ArgumentParser() parser.add_argument("--settings", type=Path, default=DEFAULT_SETTINGS) @@ -1109,7 +2468,7 @@ def main(argv: list[str] | None = None) -> None: parser.add_argument("--port", type=int, default=DEFAULT_PORT) args = parser.parse_args(argv) - shim = ShimServer(args.settings) + shim = ShimServer(args.settings, host=args.host) web.run_app(shim.app(), host=args.host, port=args.port, handle_signals=True) diff --git a/codex_shim/settings.py b/codex_shim/settings.py index a1e42cf4..11e1f38b 100644 --- a/codex_shim/settings.py +++ b/codex_shim/settings.py @@ -9,11 +9,31 @@ DEFAULT_SETTINGS = Path.home() / ".codex-shim" / "models.json" +DEFAULT_CURSOR_API_KEY_FILE = Path.home() / ".codex-shim" / "cursor-api-key" DEFAULT_CODEX_AUTH = Path.home() / ".codex" / "auth.json" +DEFAULT_CODEX_MODELS_CACHE = Path.home() / ".codex" / "models_cache.json" DEFAULT_HOST = "127.0.0.1" DEFAULT_PORT = 8765 PROVIDER_NAME = "codex_shim" CHATGPT_MODEL_SLUG = "gpt-5.5" +FALLBACK_CHATGPT_PASSTHROUGH_SLUGS = ( + "gpt-5.5", + "gpt-5.4", + "gpt-5.4-mini", + "gpt-5.3-codex", + "gpt-5.3-codex-spark", + "gpt-5.2", + "codex-auto-review", +) +FALLBACK_CHATGPT_DISPLAY_NAMES = { + "gpt-5.5": "GPT-5.5", + "gpt-5.4": "gpt-5.4", + "gpt-5.4-mini": "GPT-5.4-Mini", + "gpt-5.3-codex": "gpt-5.3-codex", + "gpt-5.3-codex-spark": "GPT-5.3-Codex-Spark", + "gpt-5.2": "gpt-5.2", + "codex-auto-review": "Codex Auto Review", +} def chatgpt_passthrough_available(auth_path: Path | None = None) -> bool: @@ -37,6 +57,111 @@ def chatgpt_passthrough_available(auth_path: Path | None = None) -> bool: return bool(tokens.get("access_token")) +def _is_listed_gpt_model(entry: dict[str, Any]) -> bool: + slug = str(entry.get("slug") or "").strip() + if not slug: + return False + if entry.get("visibility") == "hidden": + return False + lower = slug.lower() + return lower.startswith("gpt-") or lower.startswith("codex-") + + +def _minimal_chatgpt_passthrough_entry(slug: str, display_name: str) -> dict[str, Any]: + return { + "slug": slug, + "display_name": display_name, + "description": f"OpenAI {display_name} routed through ChatGPT passthrough.", + "context_window": 400000, + "max_context_window": 400000, + "auto_compact_token_limit": 320000, + "truncation_policy": {"mode": "tokens", "limit": 64000}, + "default_reasoning_level": "medium", + "supported_reasoning_levels": [ + {"effort": "minimal", "description": "Minimal reasoning"}, + {"effort": "low", "description": "Faster, lighter reasoning"}, + {"effort": "medium", "description": "Balanced"}, + {"effort": "high", "description": "Deeper reasoning"}, + {"effort": "xhigh", "description": "Maximum reasoning"}, + ], + "default_reasoning_summary": "auto", + "reasoning_summary_format": "experimental", + "supports_reasoning_summaries": True, + "default_verbosity": "medium", + "support_verbosity": True, + "apply_patch_tool_type": "freeform", + "web_search_tool_type": "text_and_image", + "supports_search_tool": True, + "supports_parallel_tool_calls": True, + "experimental_supported_tools": [], + "input_modalities": ["text", "image"], + "supports_image_detail_original": True, + "shell_type": "shell_command", + "visibility": "list", + "minimal_client_version": "0.0.1", + "supported_in_api": True, + "availability_nux": None, + "upgrade": None, + "priority": 10000 if slug == CHATGPT_MODEL_SLUG else 9000, + "prefer_websockets": False, + "available_in_plans": ["free", "plus", "pro", "team", "business", "enterprise"], + "base_instructions": f"You are Codex, a coding agent powered by {display_name}.", + "model_messages": { + "instructions_template": f"You are Codex, a coding agent powered by {display_name}.", + "instructions_variables": {"model_name": display_name}, + }, + **({"isDefault": True} if slug == CHATGPT_MODEL_SLUG else {}), + } + + +def load_chatgpt_passthrough_catalog_models(cache_path: Path | None = None) -> list[dict[str, Any]]: + path = Path(cache_path or DEFAULT_CODEX_MODELS_CACHE).expanduser() + if path.exists(): + try: + data = json.loads(path.read_text()) + except (OSError, json.JSONDecodeError): + data = None + if isinstance(data, dict): + models = data.get("models") + if isinstance(models, list): + entries = [dict(model) for model in models if isinstance(model, dict) and _is_listed_gpt_model(model)] + if entries: + return entries + return [ + _minimal_chatgpt_passthrough_entry( + slug, + FALLBACK_CHATGPT_DISPLAY_NAMES.get(slug, slug), + ) + for slug in FALLBACK_CHATGPT_PASSTHROUGH_SLUGS + ] + + +def chatgpt_passthrough_slugs(cache_path: Path | None = None) -> set[str]: + return {str(model["slug"]) for model in load_chatgpt_passthrough_catalog_models(cache_path) if model.get("slug")} + + +def chatgpt_passthrough_display_names(cache_path: Path | None = None) -> dict[str, str]: + return { + str(model["slug"]): str(model.get("display_name") or model["slug"]) + for model in load_chatgpt_passthrough_catalog_models(cache_path) + if model.get("slug") + } + + +def is_chatgpt_passthrough_slug(slug: str, cache_path: Path | None = None) -> bool: + if slug.startswith("openai-gpt-"): + return True + return slug in chatgpt_passthrough_slugs(cache_path) + + +def chatgpt_upstream_model(slug: str, cache_path: Path | None = None) -> str: + if slug.startswith("openai-gpt-"): + return CHATGPT_MODEL_SLUG + if slug in chatgpt_passthrough_slugs(cache_path): + return slug + return CHATGPT_MODEL_SLUG + + def slugify(value: str) -> str: slug = re.sub(r"[^a-zA-Z0-9]+", "-", value.strip().lower()).strip("-") return slug or "model" @@ -107,6 +232,12 @@ def load(self) -> list[ShimModel]: for k, v in (_field(row, "extra_headers", "extraHeaders", default={}) or {}).items() if v is not None } + api_key_env = str(_field(row, "api_key_env", "apiKeyEnv", default="")).strip() + api_key = str(_field(row, "api_key", "apiKey", default="")) + if api_key_env: + api_key = os.environ.get(api_key_env, api_key).strip() + else: + api_key = _resolve_api_key(api_key) models.append( ShimModel( slug=slug, @@ -114,7 +245,7 @@ def load(self) -> list[ShimModel]: display_name=display_name, provider=provider, base_url=base_url, - api_key=str(_field(row, "api_key", "apiKey", default="")), + api_key=api_key, index=index, max_context_limit=_int_or_none(_field(row, "max_context_limit", "maxContextLimit")), max_output_tokens=_int_or_none(_field(row, "max_output_tokens", "maxOutputTokens")), @@ -135,6 +266,12 @@ def by_slug_or_model(self, requested: str) -> ShimModel | None: return matches[0] return None + def load_router(self): + """Parse the optional top-level ``router`` block from the settings file.""" + from .router import load_router_config + + return load_router_config(self.path) + def _model_rows(data: Any) -> list[dict[str, Any]]: if isinstance(data, list): @@ -193,6 +330,20 @@ def _field(row: dict[str, Any], *keys: str, default: Any = None) -> Any: return default +def _resolve_api_key(value: str) -> str: + raw = value.strip() + if raw.startswith("${") and raw.endswith("}"): + raw = os.environ.get(raw[2:-1].strip(), "") + if not raw and DEFAULT_CURSOR_API_KEY_FILE.exists(): + try: + raw = DEFAULT_CURSOR_API_KEY_FILE.read_text().strip() + except OSError: + raw = "" + if not raw: + raw = os.environ.get("CURSOR_API_KEY", "").strip() + return raw + + def _int_or_none(value: Any) -> int | None: if value in (None, ""): return None @@ -203,13 +354,40 @@ def _int_or_none(value: Any) -> int | None: def default_model_slug(models: list[ShimModel], include_chatgpt: bool | None = None) -> str: + from .cursor_passthrough import CURSOR_MODEL_SLUG, cursor_passthrough_available + if include_chatgpt is None: include_chatgpt = chatgpt_passthrough_available() if include_chatgpt: return CHATGPT_MODEL_SLUG - if models: - return models[0].slug + usable = usable_byok_models(models) + if usable: + return usable[0].slug + if cursor_passthrough_available(): + return CURSOR_MODEL_SLUG raise ValueError( "No usable codex-shim models: add models to ~/.codex-shim/models.json, run `codex login`, " - "or unset CODEX_SHIM_DISABLE_CHATGPT if ChatGPT passthrough should be used." + "run `cursor-agent login`, or unset CODEX_SHIM_DISABLE_CHATGPT / CODEX_SHIM_DISABLE_CURSOR." ) + + +def usable_byok_models(models: list[ShimModel]) -> list[ShimModel]: + return [model for model in models if byok_model_has_credentials(model)] + + +def available_model_slugs(models: list[ShimModel]) -> set[str]: + """Every model slug the shim can route to right now: usable BYOK models plus + any available ChatGPT/Cursor passthrough slugs. Used by the Auto Router to + keep routing to candidates that actually exist.""" + from .cursor_passthrough import cursor_passthrough_available, cursor_passthrough_display_names + + slugs = {model.slug for model in usable_byok_models(models)} + if chatgpt_passthrough_available(): + slugs |= chatgpt_passthrough_slugs() + if cursor_passthrough_available(): + slugs |= set(cursor_passthrough_display_names()) + return slugs + + +def byok_model_has_credentials(model: ShimModel) -> bool: + return bool(model.api_key.strip()) diff --git a/codex_shim/translate.py b/codex_shim/translate.py index 1c25643e..a5765b26 100644 --- a/codex_shim/translate.py +++ b/codex_shim/translate.py @@ -61,7 +61,9 @@ def responses_to_chat(body: dict[str, Any], upstream_model: str) -> dict[str, An tools = _responses_tools_to_chat_tools(body.get("tools")) if tools: chat["tools"] = tools - _copy_if_present(body, chat, "tool_choice") + tool_choice = _responses_tool_choice_to_chat(body.get("tool_choice"), body.get("tools")) + if tool_choice is not None: + chat["tool_choice"] = tool_choice return chat @@ -103,9 +105,9 @@ def append(role: str, content: Any) -> None: blocks: list[dict[str, Any]] = [] blocks.extend(pending_thinking) pending_thinking = [] - text = chat_msg.get("content") - if text: - blocks.append({"type": "text", "text": _content_to_text(text)}) + content = chat_msg.get("content") + if content: + blocks.extend(_chat_content_to_anthropic_blocks(content)) for call in chat_msg.get("tool_calls") or []: fn = call.get("function") or {} args_raw = fn.get("arguments") or "" @@ -142,7 +144,7 @@ def append(role: str, content: Any) -> None: continue # user / anything else pending_thinking = [] - append(role, _content_to_text(chat_msg.get("content", ""))) + append(role, _chat_content_to_anthropic_content(chat_msg.get("content", ""))) # If reasoning items appeared without a following assistant turn (e.g. the # final pending think after a tool_use round-trip), emit an assistant @@ -188,6 +190,102 @@ def chat_to_anthropic(body: dict[str, Any], upstream_model: str, max_tokens: int return responses_to_anthropic(pseudo_responses, upstream_model, max_tokens) +def anthropic_messages_to_chat(body: dict[str, Any], upstream_model: str, max_tokens: int | None = None) -> dict[str, Any]: + messages: list[dict[str, Any]] = [] + system = body.get("system") + if system: + messages.append({"role": "system", "content": _anthropic_content_to_text(system)}) + + for raw_msg in body.get("messages") or []: + if not isinstance(raw_msg, dict): + continue + role = str(raw_msg.get("role") or "user") + content = raw_msg.get("content", "") + if role == "assistant": + messages.append(_anthropic_assistant_message_to_chat(content)) + elif role == "user": + messages.extend(_anthropic_user_message_to_chat(content)) + else: + messages.append({"role": role, "content": _anthropic_content_to_chat_content(content)}) + + messages = _sanitize_chat_messages(_merge_consecutive_messages(_normalize_chat_roles(messages))) + chat: dict[str, Any] = { + "model": upstream_model, + "messages": messages or [{"role": "user", "content": ""}], + "stream": bool(body.get("stream", False)), + } + _copy_if_present(body, chat, "temperature") + _copy_if_present(body, chat, "top_p") + _copy_if_present(body, chat, "max_tokens") + if max_tokens and "max_tokens" not in chat: + chat["max_tokens"] = max_tokens + _copy_if_present(body, chat, "stop_sequences", "stop") + + thinking = body.get("thinking") + if isinstance(thinking, dict) and thinking.get("effort"): + chat["reasoning_effort"] = thinking["effort"] + output_config = body.get("output_config") + if isinstance(output_config, dict) and output_config.get("effort"): + chat["reasoning_effort"] = output_config["effort"] + + tools = _anthropic_tools_to_chat_tools(body.get("tools")) + if tools: + chat["tools"] = tools + tool_choice = _anthropic_tool_choice_to_chat(body.get("tool_choice")) + if tool_choice is not None: + chat["tool_choice"] = tool_choice + if chat["stream"]: + stream_options = dict(body.get("stream_options") or {}) + stream_options["include_usage"] = True + chat["stream_options"] = stream_options + return chat + + +def chat_completion_to_anthropic_message(payload: dict[str, Any], requested_model: str) -> dict[str, Any]: + choice = (payload.get("choices") or [{}])[0] + message = choice.get("message") or {} + content: list[dict[str, Any]] = [] + reasoning = message.get("reasoning_content") or message.get("reasoning") + if reasoning: + content.append({"type": "thinking", "thinking": str(reasoning)}) + text = strip_think(message.get("content") or "") + if text: + content.append({"type": "text", "text": text}) + for call in message.get("tool_calls") or []: + if not isinstance(call, dict): + continue + fn = call.get("function") or {} + args_raw = fn.get("arguments") or "" + try: + args_obj = json.loads(args_raw) if args_raw else {} + except json.JSONDecodeError: + args_obj = {"_raw": args_raw} + content.append( + { + "type": "tool_use", + "id": call.get("id") or "call_0", + "name": fn.get("name") or "", + "input": args_obj, + } + ) + if not content: + content.append({"type": "text", "text": ""}) + + response: dict[str, Any] = { + "id": payload.get("id") or "msg_chat", + "type": "message", + "role": "assistant", + "model": requested_model, + "content": content, + "stop_reason": _chat_finish_to_anthropic_stop(choice.get("finish_reason")), + "stop_sequence": None, + } + usage = _responses_usage_to_anthropic_usage(normalize_responses_usage(payload.get("usage"))) + if usage is not None: + response["usage"] = usage + return response + + def anthropic_to_chat_response(payload: dict[str, Any], requested_model: str) -> dict[str, Any]: content = "" tool_calls = [] @@ -217,7 +315,7 @@ def anthropic_to_chat_response(payload: dict[str, Any], requested_model: str) -> } -def chat_completion_to_response(payload: dict[str, Any], requested_model: str) -> dict[str, Any]: +def chat_completion_to_response(payload: dict[str, Any], requested_model: str, tool_types: dict[str, str] | None = None) -> dict[str, Any]: choice = (payload.get("choices") or [{}])[0] message = choice.get("message") or {} output: list[dict[str, Any]] = [] @@ -242,15 +340,23 @@ def chat_completion_to_response(payload: dict[str, Any], requested_model: str) - "content": [{"type": "output_text", "text": text, "annotations": []}], } ) + tool_types = tool_types or {} for call in message.get("tool_calls") or []: fn = call.get("function") or {} + name = fn.get("name", "") + original_type = tool_types.get(name, "") + item_type = "function_call" + if original_type == "apply_patch": + item_type = "custom_tool_call" + elif original_type.startswith("web_search"): + item_type = "web_search_call" output.append( { "id": call.get("id", "call_0"), - "type": "function_call", + "type": item_type, "status": "completed", "call_id": call.get("id", "call_0"), - "name": fn.get("name", ""), + "name": name, "arguments": fn.get("arguments", ""), } ) @@ -261,12 +367,81 @@ def chat_completion_to_response(payload: dict[str, Any], requested_model: str) - "status": "completed", "model": requested_model, "output": output, - "usage": payload.get("usage"), + "usage": normalize_responses_usage(payload.get("usage")), + } + + +def anthropic_to_response(payload: dict[str, Any], requested_model: str, tool_types: dict[str, str] | None = None) -> dict[str, Any]: + response = chat_completion_to_response(anthropic_to_chat_response(payload, requested_model), requested_model, tool_types) + response["usage"] = normalize_responses_usage(payload.get("usage")) + return response + + +def normalize_responses_usage(usage: Any) -> dict[str, Any] | None: + if not isinstance(usage, dict): + return None + + input_tokens = _int_token(usage.get("input_tokens")) + if input_tokens is None: + input_tokens = _int_token(usage.get("prompt_tokens")) + + output_tokens = _int_token(usage.get("output_tokens")) + if output_tokens is None: + output_tokens = _int_token(usage.get("completion_tokens")) + + total_tokens = _int_token(usage.get("total_tokens")) + if total_tokens is None and input_tokens is not None and output_tokens is not None: + total_tokens = input_tokens + output_tokens + + if input_tokens is None: + input_tokens = max(total_tokens - output_tokens, 0) if total_tokens is not None and output_tokens is not None else 0 + if output_tokens is None: + output_tokens = max(total_tokens - input_tokens, 0) if total_tokens is not None else 0 + if total_tokens is None: + total_tokens = input_tokens + output_tokens + + normalized: dict[str, Any] = { + "input_tokens": input_tokens, + "output_tokens": output_tokens, + "total_tokens": total_tokens, } + input_details: dict[str, Any] = {} + if isinstance(usage.get("input_tokens_details"), dict): + input_details.update(usage["input_tokens_details"]) + if isinstance(usage.get("prompt_tokens_details"), dict): + input_details.update(usage["prompt_tokens_details"]) + + cache_read = _int_token(usage.get("cache_read_input_tokens")) + if cache_read is not None: + input_details.setdefault("cached_tokens", cache_read) + input_details.setdefault("cache_read_input_tokens", cache_read) + cache_created = _int_token(usage.get("cache_creation_input_tokens")) + if cache_created is not None: + input_details.setdefault("cache_creation_input_tokens", cache_created) + + if input_details: + normalized["input_tokens_details"] = input_details + + output_details: dict[str, Any] = {} + if isinstance(usage.get("output_tokens_details"), dict): + output_details.update(usage["output_tokens_details"]) + if isinstance(usage.get("completion_tokens_details"), dict): + output_details.update(usage["completion_tokens_details"]) + if output_details: + normalized["output_tokens_details"] = output_details + + return normalized + -def anthropic_to_response(payload: dict[str, Any], requested_model: str) -> dict[str, Any]: - return chat_completion_to_response(anthropic_to_chat_response(payload, requested_model), requested_model) +def _int_token(value: Any) -> int | None: + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, float) and value.is_integer(): + return int(value) + return None def strip_think(text: str) -> str: @@ -279,7 +454,7 @@ def _responses_input_to_messages(value: Any) -> list[dict[str, Any]]: if isinstance(value, str): return [{"role": "user", "content": value}] if not isinstance(value, list): - return [{"role": "user", "content": _content_to_text(value)}] + return [{"role": "user", "content": _responses_content_to_chat_content(value)}] messages: list[dict[str, Any]] = [] pending_tool_calls: list[dict[str, Any]] = [] @@ -301,10 +476,13 @@ def flush_pending_assistant_tool_calls(): role = item.get("role", "user") if role == "developer": role = "system" - messages.append({"role": role, "content": _content_to_text(item.get("content", ""))}) - elif item_type in {"input_text", "text"}: + messages.append({"role": role, "content": _responses_content_to_chat_content(item.get("content", ""))}) + elif item_type in {"input_text", "text", "input_image"}: + flush_pending_assistant_tool_calls() + messages.append({"role": "user", "content": _responses_content_to_chat_content(item)}) + elif item_type == "computer_call_output": flush_pending_assistant_tool_calls() - messages.append({"role": "user", "content": _content_to_text(item)}) + messages.append({"role": "user", "content": _computer_output_to_chat_content(item)}) elif item_type == "function_call": # Coalesce consecutive function_call items into a single assistant # message with multiple tool_calls so chat-completions upstreams @@ -322,7 +500,10 @@ def flush_pending_assistant_tool_calls(): ) elif item_type == "function_call_output": flush_pending_assistant_tool_calls() - messages.append({"role": "tool", "tool_call_id": item.get("call_id"), "content": _content_to_text(item.get("output", ""))}) + output = item.get("output", "") + messages.append({"role": "tool", "tool_call_id": item.get("call_id"), "content": _content_to_text(output)}) + if _has_visual_content(output): + messages.append({"role": "user", "content": _visual_feedback_chat_content(output, item.get("call_id"))}) elif item_type == "reasoning": # For Chat-Completions upstreams reasoning is informational only. # We keep it as a marker so the Anthropic translator can reattach @@ -341,6 +522,267 @@ def flush_pending_assistant_tool_calls(): return messages +def _responses_content_to_chat_content(content: Any) -> str | list[dict[str, Any]]: + parts = _chat_parts_from_content(content) + if not parts: + return "" + if any(part.get("type") == "image_url" for part in parts): + return parts + return "\n".join(str(part.get("text", "")) for part in parts if part.get("type") == "text") + + +def _computer_output_to_chat_content(item: dict[str, Any]) -> str | list[dict[str, Any]]: + call_id = item.get("call_id") or item.get("id") + prefix = f"Computer output for {call_id}." if call_id else "Computer output." + parts = _chat_parts_from_content(item.get("output", "")) + if any(part.get("type") == "image_url" for part in parts): + return [{"type": "text", "text": prefix}, *parts] + text = "\n".join(str(part.get("text", "")) for part in parts if part.get("type") == "text") + return f"{prefix}\n{text}" if text else prefix + + +def _visual_feedback_chat_content(output: Any, call_id: Any) -> list[dict[str, Any]]: + prefix = f"Visual tool output for {call_id}." if call_id else "Visual tool output." + parts = [part for part in _chat_parts_from_content(output) if part.get("type") == "image_url"] + return [{"type": "text", "text": prefix}, *parts] + + +def _chat_parts_from_content(content: Any) -> list[dict[str, Any]]: + if content is None: + return [] + if isinstance(content, str): + return [{"type": "text", "text": content}] if content else [] + if isinstance(content, list): + parts: list[dict[str, Any]] = [] + for part in content: + parts.extend(_chat_parts_from_content(part)) + return parts + if isinstance(content, dict): + content_type = str(content.get("type") or "") + if content_type in {"input_text", "output_text", "text"}: + text = str(content.get("text", "")) + return [{"type": "text", "text": text}] if text else [] + if content_type in {"input_image", "image_url"} or "image_url" in content: + image = _chat_image_part(content) + return [image] if image else [] + if content_type == "computer_call_output": + return _chat_parts_from_content(content.get("output")) + if "output" in content and _has_visual_content(content.get("output")): + return _chat_parts_from_content(content.get("output")) + if "content" in content: + return _chat_parts_from_content(content["content"]) + if "text" in content: + text = str(content.get("text", "")) + return [{"type": "text", "text": text}] if text else [] + return [] + + +def _chat_image_part(part: dict[str, Any]) -> dict[str, Any] | None: + url = _image_url_from_part(part) + if not url: + return None + image_url: dict[str, Any] = {"url": url} + detail = part.get("detail") or part.get("image_detail") + if detail and detail not in ("low", "auto", "high", "xhigh"): + # Codex Desktop sends "original" which is not a standard OpenAI Chat + # Completions value — providers like Kimi K2.6 reject it (400). + # Map it to "high" (the closest standard equivalent). Any unknown + # detail value falls back to "auto". + detail = "high" if detail == "original" else "auto" + if detail: + image_url["detail"] = detail + return {"type": "image_url", "image_url": image_url} + + +def _image_url_from_part(part: dict[str, Any]) -> str: + image_url = part.get("image_url") + if isinstance(image_url, str): + return image_url + if isinstance(image_url, dict) and isinstance(image_url.get("url"), str): + return image_url["url"] + for key in ("url", "file_url"): + value = part.get(key) + if isinstance(value, str): + return value + return "" + + +def _has_visual_content(content: Any) -> bool: + return any(part.get("type") == "image_url" for part in _chat_parts_from_content(content)) + + +def _chat_content_to_anthropic_content(content: Any) -> str | list[dict[str, Any]]: + blocks = _chat_content_to_anthropic_blocks(content) + if not any(block.get("type") == "image" for block in blocks): + return "\n".join(block.get("text", "") for block in blocks if block.get("type") == "text") + return blocks + + +def _chat_content_to_anthropic_blocks(content: Any) -> list[dict[str, Any]]: + blocks: list[dict[str, Any]] = [] + for part in _chat_parts_from_content(content): + if part.get("type") == "text": + text = str(part.get("text", "")) + if text: + blocks.append({"type": "text", "text": text}) + elif part.get("type") == "image_url": + block = _chat_image_part_to_anthropic(part) + if block: + blocks.append(block) + return blocks or [{"type": "text", "text": ""}] + + +def _chat_image_part_to_anthropic(part: dict[str, Any]) -> dict[str, Any] | None: + image_url = part.get("image_url") + url = "" + if isinstance(image_url, dict): + url = str(image_url.get("url") or "") + elif isinstance(image_url, str): + url = image_url + if not url: + return None + if url.startswith("data:"): + match = re.match(r"data:([^;,]+);base64,(.*)", url, re.DOTALL) + if not match: + return None + return {"type": "image", "source": {"type": "base64", "media_type": match.group(1), "data": match.group(2)}} + return {"type": "image", "source": {"type": "url", "url": url}} + + +def _anthropic_content_to_chat_content(content: Any) -> str | list[dict[str, Any]]: + parts = _anthropic_content_to_chat_parts(content) + if not parts: + return "" + if any(part.get("type") == "image_url" for part in parts): + return parts + return "\n".join(str(part.get("text", "")) for part in parts if part.get("type") == "text") + + +def _anthropic_content_to_text(content: Any) -> str: + chat_content = _anthropic_content_to_chat_content(content) + return _content_to_text(chat_content) + + +def _anthropic_content_to_chat_parts(content: Any) -> list[dict[str, Any]]: + if content is None: + return [] + if isinstance(content, str): + return [{"type": "text", "text": content}] if content else [] + if isinstance(content, list): + parts: list[dict[str, Any]] = [] + for block in content: + if not isinstance(block, dict): + parts.extend(_anthropic_content_to_chat_parts(block)) + continue + block_type = block.get("type") + if block_type == "text": + text = str(block.get("text", "")) + if text: + parts.append({"type": "text", "text": text}) + elif block_type == "image": + image = _anthropic_image_block_to_chat_part(block) + if image: + parts.append(image) + elif block_type == "image_url" or "image_url" in block: + image = _chat_image_part(block) + if image: + parts.append(image) + elif "content" in block: + parts.extend(_anthropic_content_to_chat_parts(block.get("content"))) + return parts + if isinstance(content, dict): + return _anthropic_content_to_chat_parts([content]) + return [{"type": "text", "text": str(content)}] + + +def _anthropic_image_block_to_chat_part(block: dict[str, Any]) -> dict[str, Any] | None: + source = block.get("source") + if not isinstance(source, dict): + return None + url = "" + if source.get("type") == "base64": + media_type = str(source.get("media_type") or "image/png") + data = str(source.get("data") or "") + if data: + url = f"data:{media_type};base64,{data}" + elif source.get("type") == "url": + url = str(source.get("url") or "") + if not url: + return None + return {"type": "image_url", "image_url": {"url": url}} + + +def _anthropic_assistant_message_to_chat(content: Any) -> dict[str, Any]: + text_parts: list[Any] = [] + tool_calls: list[dict[str, Any]] = [] + reasoning_parts: list[str] = [] + blocks = content if isinstance(content, list) else [{"type": "text", "text": content}] + for block in blocks: + if not isinstance(block, dict): + text_parts.append(block) + continue + block_type = block.get("type") + if block_type == "tool_use": + tool_calls.append( + { + "id": block.get("id") or f"call_{len(tool_calls)}", + "type": "function", + "function": { + "name": block.get("name") or "", + "arguments": _jsonish(block.get("input", {})), + }, + } + ) + elif block_type in {"thinking", "redacted_thinking"}: + thinking = block.get("thinking") or block.get("data") or "" + if thinking: + reasoning_parts.append(str(thinking)) + else: + text_parts.append(block) + message: dict[str, Any] = {"role": "assistant", "content": _anthropic_content_to_chat_content(text_parts) if text_parts else ""} + if tool_calls: + message["tool_calls"] = tool_calls + if reasoning_parts: + message["reasoning_content"] = "\n".join(reasoning_parts) + return message + + +def _anthropic_user_message_to_chat(content: Any) -> list[dict[str, Any]]: + if not isinstance(content, list): + return [{"role": "user", "content": _anthropic_content_to_chat_content(content)}] + tool_messages: list[dict[str, Any]] = [] + user_parts: list[Any] = [] + for block in content: + if isinstance(block, dict) and block.get("type") == "tool_result": + tool_content = block.get("content", "") + tool_use_id = block.get("tool_use_id") or "call_0" + tool_messages.append( + { + "role": "tool", + "tool_call_id": tool_use_id, + "content": _anthropic_content_to_text(tool_content), + } + ) + if _anthropic_has_visual_content(tool_content): + user_parts.extend(_anthropic_visual_tool_result_chat_parts(tool_content, tool_use_id)) + else: + user_parts.append(block) + messages = list(tool_messages) + if user_parts or not messages: + messages.append({"role": "user", "content": _anthropic_content_to_chat_content(user_parts)}) + return messages + + +def _anthropic_has_visual_content(content: Any) -> bool: + return any(part.get("type") == "image_url" for part in _anthropic_content_to_chat_parts(content)) + + +def _anthropic_visual_tool_result_chat_parts(content: Any, tool_use_id: Any) -> list[dict[str, Any]]: + prefix = f"Visual tool result for {tool_use_id}." if tool_use_id else "Visual tool result." + images = [part for part in _anthropic_content_to_chat_parts(content) if part.get("type") == "image_url"] + return [{"type": "text", "text": prefix}, *images] + + def _content_to_text(content: Any) -> str: if isinstance(content, str): return content @@ -352,10 +794,16 @@ def _content_to_text(content: Any) -> str: elif isinstance(part, dict): if part.get("type") in {"input_text", "output_text", "text"}: parts.append(str(part.get("text", ""))) + elif part.get("type") in {"input_image", "image_url"} or "image_url" in part: + parts.append("[image]") elif "content" in part: parts.append(_content_to_text(part["content"])) return "\n".join(p for p in parts if p) if isinstance(content, dict): + if content.get("type") in {"input_image", "image_url"} or "image_url" in content: + return "[image]" + if "output" in content: + return _content_to_text(content.get("output")) if "text" in content: return str(content.get("text", "")) return str(content) @@ -367,38 +815,147 @@ def _responses_tools_to_chat_tools(tools: Any) -> list[dict[str, Any]]: return [] converted = [] for tool in tools: - if not isinstance(tool, dict): - continue - if tool.get("type") == "function": - if "function" in tool: - converted.append(tool) - elif "name" in tool: - converted.append( - { - "type": "function", - "function": { - "name": tool.get("name"), - "description": tool.get("description", ""), - "parameters": tool.get("parameters") - or {"type": "object", "properties": {}, "additionalProperties": True}, - }, - } - ) - elif "name" in tool: - converted.append( - { - "type": "function", - "function": { - "name": tool.get("name"), - "description": tool.get("description", ""), - "parameters": tool.get("parameters") - or {"type": "object", "properties": {"input": {"type": "string"}}, "required": ["input"]}, - }, - } - ) + function_tool = _responses_tool_to_chat_function(tool) + if function_tool: + converted.append(function_tool) return converted +def _responses_tool_to_chat_function(tool: Any) -> dict[str, Any] | None: + if not isinstance(tool, dict): + return None + if tool.get("type") == "function" and "function" in tool: + return tool + name = _responses_tool_function_name(tool) + if not name: + return None + return { + "type": "function", + "function": { + "name": name, + "description": tool.get("description") or _native_tool_description(tool), + "parameters": tool.get("parameters") or _native_tool_parameters(tool), + }, + } + + +def _responses_tool_function_name(tool: dict[str, Any]) -> str: + fn = tool.get("function") + if isinstance(fn, dict) and fn.get("name"): + return _sanitize_tool_name(str(fn["name"])) + if tool.get("name"): + return _sanitize_tool_name(str(tool["name"])) + tool_type = str(tool.get("type") or "").strip().lower() + aliases = { + "web_search": "web_search", + "web_search_preview": "web_search", + "computer_use": "computer_use", + "computer_use_preview": "computer_use", + "apply_patch": "apply_patch", + "local_shell": "local_shell", + "shell": "local_shell", + } + if tool_type in aliases: + return aliases[tool_type] + if tool_type.startswith("mcp"): + return _sanitize_tool_name(tool_type) + return "" + + +def _sanitize_tool_name(name: str) -> str: + clean = re.sub(r"[^a-zA-Z0-9_-]+", "_", name.strip())[:64] + return clean.strip("_") or "tool" + + +def _native_tool_description(tool: dict[str, Any]) -> str: + tool_type = str(tool.get("type") or "tool") + if tool_type.startswith("web_search"): + return "Search the web using Codex's web-search tool fallback." + if tool_type.startswith("computer_use"): + return "Request a Codex computer-use action." + if tool_type == "apply_patch": + return "Apply a unified diff patch to the working tree." + if tool_type in {"local_shell", "shell"}: + return "Run a local shell command through Codex." + if tool_type.startswith("mcp"): + return "Interact with Codex MCP resources." + return f"Codex tool fallback for Responses tool type {tool_type}." + + +def _native_tool_parameters(tool: dict[str, Any]) -> dict[str, Any]: + tool_type = str(tool.get("type") or "").strip().lower() + if tool_type.startswith("web_search"): + return { + "type": "object", + "properties": { + "query": {"type": "string", "description": "Search query"}, + }, + "required": ["query"], + "additionalProperties": True, + } + if tool_type.startswith("computer_use"): + return { + "type": "object", + "properties": { + "action": {"type": "string", "description": "Computer action to perform"}, + "x": {"type": "number", "description": "Screen x coordinate, when relevant"}, + "y": {"type": "number", "description": "Screen y coordinate, when relevant"}, + "text": {"type": "string", "description": "Text to type, when relevant"}, + }, + "required": ["action"], + "additionalProperties": True, + } + if tool_type == "apply_patch": + return { + "type": "object", + "properties": {"patch": {"type": "string", "description": "Unified diff patch"}}, + "required": ["patch"], + "additionalProperties": True, + } + if tool_type in {"local_shell", "shell"}: + return { + "type": "object", + "properties": {"command": {"type": "string", "description": "Shell command to run"}}, + "required": ["command"], + "additionalProperties": True, + } + return {"type": "object", "properties": {"input": {"type": "string"}}, "additionalProperties": True} + + +def _responses_tool_choice_to_chat(tool_choice: Any, tools: Any) -> Any: + if tool_choice is None: + return None + if isinstance(tool_choice, str): + if tool_choice in {"auto", "none", "required"}: + return tool_choice + name = _tool_choice_name(tool_choice, tools) + return {"type": "function", "function": {"name": name}} if name else tool_choice + if isinstance(tool_choice, dict): + if tool_choice.get("type") == "function" and "function" in tool_choice: + return tool_choice + name = _tool_choice_name(str(tool_choice.get("name") or tool_choice.get("type") or ""), tools) + return {"type": "function", "function": {"name": name}} if name else tool_choice + return tool_choice + + +def _tool_choice_name(choice: str, tools: Any) -> str: + choice = choice.lower().strip() + if isinstance(tools, list): + for tool in tools: + if not isinstance(tool, dict): + continue + names = { + str(tool.get("type") or "").lower(), + str(tool.get("name") or "").lower(), + } + fn = tool.get("function") + if isinstance(fn, dict): + names.add(str(fn.get("name") or "").lower()) + if choice in names: + return _responses_tool_function_name(tool) + return _sanitize_tool_name(choice) + + def _responses_tools_to_anthropic_tools(tools: Any) -> list[dict[str, Any]]: chat_tools = _responses_tools_to_chat_tools(tools) converted = [] @@ -414,6 +971,77 @@ def _responses_tools_to_anthropic_tools(tools: Any) -> list[dict[str, Any]]: return [tool for tool in converted if tool.get("name")] +def _anthropic_tools_to_chat_tools(tools: Any) -> list[dict[str, Any]]: + if not isinstance(tools, list): + return [] + converted: list[dict[str, Any]] = [] + for tool in tools: + if not isinstance(tool, dict): + continue + name = tool.get("name") + if not name: + continue + converted.append( + { + "type": "function", + "function": { + "name": str(name), + "description": tool.get("description") or "", + "parameters": tool.get("input_schema") or {"type": "object", "properties": {}}, + }, + } + ) + return converted + + +def _anthropic_tool_choice_to_chat(tool_choice: Any) -> Any: + if tool_choice is None: + return None + if isinstance(tool_choice, str): + if tool_choice in {"auto", "none"}: + return tool_choice + if tool_choice == "any": + return "required" + return {"type": "function", "function": {"name": tool_choice}} + if isinstance(tool_choice, dict): + choice_type = tool_choice.get("type") + if choice_type in {"auto", "none"}: + return choice_type + if choice_type == "any": + return "required" + if choice_type == "tool" and tool_choice.get("name"): + return {"type": "function", "function": {"name": str(tool_choice["name"])}} + return tool_choice + + +def _chat_finish_to_anthropic_stop(reason: Any) -> str: + if reason in {"tool_calls", "function_call"}: + return "tool_use" + if reason == "length": + return "max_tokens" + if reason == "content_filter": + return "refusal" + return "end_turn" + + +def _responses_usage_to_anthropic_usage(usage: dict[str, Any] | None) -> dict[str, Any] | None: + if usage is None: + return None + result = { + "input_tokens": int(usage.get("input_tokens") or 0), + "output_tokens": int(usage.get("output_tokens") or 0), + } + input_details = usage.get("input_tokens_details") + if isinstance(input_details, dict): + cache_read = input_details.get("cache_read_input_tokens", input_details.get("cached_tokens")) + if isinstance(cache_read, int) and not isinstance(cache_read, bool): + result["cache_read_input_tokens"] = cache_read + cache_created = input_details.get("cache_creation_input_tokens") + if isinstance(cache_created, int) and not isinstance(cache_created, bool): + result["cache_creation_input_tokens"] = cache_created + return result + + def _copy_if_present(src: dict[str, Any], dst: dict[str, Any], src_key: str, dst_key: str | None = None) -> None: if src_key in src and src[src_key] is not None: dst[dst_key or src_key] = src[src_key] @@ -436,6 +1064,27 @@ def _sanitize_string(value: str) -> str: return "".join(char for char in value if char in "\n\r\t" or ord(char) >= 0x20) +def _sanitize_chat_content_parts(parts: list[Any]) -> list[dict[str, Any]]: + cleaned = [] + for part in parts: + if isinstance(part, str): + cleaned.append({"type": "text", "text": _sanitize_string(part)}) + continue + if not isinstance(part, dict): + continue + current = dict(part) + if current.get("type") == "text": + current["text"] = _sanitize_string(str(current.get("text", ""))) + elif current.get("type") == "image_url": + image_url = current.get("image_url") + if isinstance(image_url, dict): + current["image_url"] = {k: _sanitize_string(str(v)) for k, v in image_url.items() if v is not None} + elif isinstance(image_url, str): + current["image_url"] = {"url": _sanitize_string(image_url)} + cleaned.append(current) + return cleaned + + def _sanitize_chat_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]: cleaned = [] for message in messages: @@ -445,16 +1094,14 @@ def _sanitize_chat_messages(messages: list[dict[str, Any]]) -> list[dict[str, An current.pop("summary", None) role = current.get("role", "user") content = current.get("content") - if role != "assistant": - if content is None: - current["content"] = "" - elif not isinstance(content, str): - current["content"] = _content_to_text(content) - current["content"] = _sanitize_string(current["content"]) - elif content is not None: - if not isinstance(content, str): - content = _content_to_text(content) + if content is None: + current["content"] = None if role == "assistant" else "" + elif isinstance(content, list): + current["content"] = _sanitize_chat_content_parts(content) + elif isinstance(content, str): current["content"] = _sanitize_string(content) + else: + current["content"] = _sanitize_string(_content_to_text(content)) if isinstance(current.get("reasoning_content"), str): current["reasoning_content"] = _sanitize_string(current["reasoning_content"]) @@ -500,12 +1147,7 @@ def _merge_consecutive_messages(messages: list[dict[str, Any]]) -> list[dict[str role = current.get("role") if merged and role == merged[-1].get("role") and role in {"system", "user", "assistant"}: previous = merged[-1] - previous_content = previous.get("content") or "" - current_content = current.get("content") or "" - if previous_content and current_content: - previous["content"] = f"{previous_content}\n\n{current_content}" - elif current_content: - previous["content"] = current_content + previous["content"] = _merge_chat_content(previous.get("content"), current.get("content")) if role == "assistant": if current.get("reasoning_content") and not previous.get("reasoning_content"): previous["reasoning_content"] = current["reasoning_content"] @@ -515,3 +1157,18 @@ def _merge_consecutive_messages(messages: list[dict[str, Any]]) -> list[dict[str continue merged.append(current) return merged + + +def _merge_chat_content(left: Any, right: Any) -> Any: + if not left: + return right or "" + if not right: + return left + if isinstance(left, list) or isinstance(right, list): + merged: list[Any] = [] + merged.extend(left if isinstance(left, list) else [{"type": "text", "text": str(left)}]) + if merged and right: + merged.append({"type": "text", "text": ""}) + merged.extend(right if isinstance(right, list) else [{"type": "text", "text": str(right)}]) + return merged + return f"{left}\n\n{right}" diff --git a/docs/AUTO_ROUTER.md b/docs/AUTO_ROUTER.md new file mode 100644 index 00000000..9c3fa7cc --- /dev/null +++ b/docs/AUTO_ROUTER.md @@ -0,0 +1,211 @@ +# Auto Router — pick the right model for every task, automatically + +The Auto Router adds one extra entry to the Codex picker: **`Auto (smart +routing)`** (slug `codex-auto`). Choose it and the shim decides, *per task*, +which of your configured models to use — sending trivial turns to a cheap model +and hard turns to your strongest one. The goal is to keep frontier-level quality +while cutting cost, running entirely on the models *you* already configure. + +It is **off unless you configure it**, fully local (standard library only on top +of the shim's existing `aiohttp` dependency), and degrades safely: if anything +goes wrong it falls back to a sensible default and never breaks a request. + +--- + +## How it works (30 seconds) + +1. You pick **`Auto (smart routing)`** in the Codex picker (or + `codex-shim model use codex-auto`). +2. On each new task, the shim sends a *tiny* scoring request to a cheap + **classifier** model you nominate. The classifier reads the task plus a short + **capability card** for each candidate and returns a score `0.0–1.0` per + candidate — the probability that model nails the task on the first try. +3. The shim picks the **cheapest candidate whose score clears `threshold`** + (default `0.7`). If none clear it, it takes the highest-scoring one. +4. The real request is routed to that backend. The decision is **cached for the + rest of that task** (its tool-call round-trips reuse it), so you pay the + classifier cost once per task, not once per request. + +``` + ┌──────────────────────────────┐ + you pick "Auto" ──▶ │ classifier (cheap model) │ + │ scores each candidate 0–1 │ + └───────────────┬──────────────┘ + │ scores + ▼ + cheapest candidate with score ≥ threshold (else best) + │ + ▼ + real /v1/responses ──▶ that backend (OpenAI chat, Anthropic, …) +``` + +The classifier **never sees price**. It scores on capability only; cost is +applied afterward by the "cheapest among those good enough" rule. That keeps the +classifier from being biased toward expensive models. + +--- + +## Prove it works (offline, no keys) + +A self-contained demo spins up a mock multi-backend server, starts the **real** +codex-shim server with the router enabled, and runs real `/v1/responses` tasks +of increasing difficulty through it — showing which backend each one hit: + +```bash +python3 examples/auto_router_demo.py +``` + +```text +# Task Classifier scores Routed to Cost +1 add a docstring to the foo() helper cheap=0.90 mid=0.92 strong=0.95 cheap $0.3 +2 write a CRUD REST endpoint with tests cheap=0.50 mid=0.85 strong=0.95 mid $1.0 +3 refactor the auth module across 8 files ... cheap=0.40 mid=0.55 strong=0.95 strong $5.0 +4 what does this screenshot show? cheap=0.90 mid=0.92 strong=0.95 strong $5.0 <- only image-capable +5 (repeat task #1) served from cache cheap $0.3 <- classifier not re-called +RESULT: PASS +``` + +Row 4: even though the cheap model *scored* 0.90, the task has an image, so the +shim hard-zeroes the models that can't see and the only vision-capable candidate +wins. Row 5 shows the per-task cache: the repeat is served without re-calling the +classifier. + +The behavior is locked down by 48 offline tests: + +```bash +python3 -m pytest tests/test_router.py tests/test_router_integration.py -q +``` + +They cover config loading, task-signal extraction, score parsing (prose, code +fences, partial scores), cheapest-among-viable selection, image hard-rejects, +the per-task cache across an agent tool-loop, OpenAI **and** Anthropic +classifiers, streaming, compaction, the chat-completions endpoint, the exact +HTTP payload/headers the shim sends to the classifier, threshold tuning, +graceful fallback on garbage classifier output, candidate availability gating, +concurrency, and the picker switch. + +--- + +## Quick start + +Add a `router` block to `~/.codex-shim/models.json` (the same file that holds +your models): + +```jsonc +{ + "models": [ + { "slug": "minimax-m3", "model": "MiniMax-M3", "provider": "openai", + "base_url": "https://api.example.com/v1", "api_key": "${MINIMAX_KEY}", + "display_name": "MiniMax M3" }, + { "slug": "opus", "model": "claude-opus-4-7", "provider": "anthropic", + "base_url": "https://api.anthropic.com/v1", "api_key": "${ANTHROPIC_KEY}", + "display_name": "Claude Opus 4.7" } + ], + "router": { + "enabled": true, + "slug": "codex-auto", + "display_name": "Auto (smart routing)", + "classifier": "minimax-m3", + "threshold": 0.7, + "default": "minimax-m3", + "cache": true, + "candidates": [ + { "slug": "minimax-m3", "cost": 0.3, "supports_images": false, + "card": "Very cheap, fast. Strong on single-file edits, codegen from a clear spec, simple refactors, data wrangling. Weak on big multi-file refactors, subtle debugging, niche domains." }, + { "slug": "opus", "cost": 5.0, "supports_images": true, + "card": "Frontier reasoning + agentic coding. Best for the hardest work: large multi-file refactors, subtle debugging, architecture, long autonomous workflows, and image tasks." } + ] + } +} +``` + +Then: + +1. `codex-shim generate` — the catalog now includes `codex-auto`. +2. `codex-shim start` (or `enable` / `app`), pick **`Auto (smart routing)`**. + +That's it. Any candidate whose backend isn't usable right now (no API key, or a +passthrough that isn't logged in) is **skipped automatically**, so the router +keeps working with whatever subset remains. + +--- + +## Configuration reference + +The `router` block in your settings file: + +| Field | Meaning | +|-------|---------| +| `enabled` | Turn the router on/off. Also overridable at runtime with `CODEX_SHIM_DISABLE_ROUTER=1`. | +| `slug` | The picker slug for the Auto entry. Defaults to `codex-auto`. | +| `display_name` | Picker label. Defaults to `Auto (smart routing)`. | +| `classifier` | A model **slug** (one of your configured BYOK models) used as the scorer. Use your cheapest, fastest model. Must be an `openai`/`generic-chat-completion-api` or `anthropic` model with a key. If it's missing/unavailable, the router falls back to the cheapest candidate **without scoring**. | +| `threshold` | `0–1`. The bar a candidate's score must clear. **Lower = more aggressive savings** (cheap models win more often); **higher = escalate to strong models sooner**. `0.7` is a good start. | +| `default` | Candidate slug used when classification can't run at all. Defaults to the cheapest candidate. | +| `cache` | Reuse one classification across a task's follow-up tool calls. Recommended. | +| `timeout` | Seconds to wait for the classifier before falling back. Default `12`. Overridable with `CODEX_SHIM_ROUTER_TIMEOUT`. | +| `max_tokens` | Max tokens for the classifier reply (it only emits small JSON). Default `600`. Overridable with `CODEX_SHIM_ROUTER_MAX_TOKENS`. | +| `candidates[].slug` | Must match a model slug, ChatGPT passthrough slug, or Cursor passthrough slug. Unusable ones are silently skipped. | +| `candidates[].cost` | A **relative** weight; units don't matter, only the ordering. Lowest cost among the "good enough" candidates wins. | +| `candidates[].supports_images` | If `false`, the candidate is hard-scored `0` whenever the task includes images. | +| `candidates[].card` | The capability description the classifier reads. **This is the single most important field** — be honest about strengths and weaknesses; that's what makes routing smart. | + +### The capability card is where the intelligence lives + +The classifier only knows about a model what the card tells it. A good card lists +concrete strengths *and* weaknesses: + +> *"Cheap, fast generalist. Good at standard servers/CRUD, data processing, +> conventional multi-file edits, and tool/test loops. Less reliable on hard +> algorithms, exotic build systems, or long autonomous debugging."* + +Vague cards ("a good model") produce vague routing. Specific cards produce sharp +routing. + +--- + +## Tuning & knobs + +| Env var | Default | What it does | +|---------|---------|--------------| +| `CODEX_SHIM_DISABLE_ROUTER` | unset | Set `1` to disable the router even if `enabled` in config. | +| `CODEX_SHIM_ROUTER_TIMEOUT` | `12` | Seconds to wait for the classifier before falling back. | +| `CODEX_SHIM_ROUTER_MAX_TOKENS` | `600` | Max tokens for the classifier's reply. | +| `CODEX_SHIM_ROUTER_LOG` | unset | Set `1` to log every routing decision + the raw scores. | + +**Watch it decide:** run with `CODEX_SHIM_ROUTER_LOG=1` and tail the shim log +(`.codex-shim/shim.log`). Each task prints a line like: + +``` +[router] -> opus (score=0.93; score>=0.70, cheapest) scores={"minimax-m3": 0.55, "opus": 0.93} +[router] codex-auto -> opus +``` + +That log is the honest way to tell whether routing is helping you — watch which +tasks escalate and tune `threshold` / cards accordingly. + +--- + +## Failure behavior (it never breaks your request) + +| Situation | What happens | +|-----------|--------------| +| Classifier times out / errors | Falls back to `default` (or cheapest candidate). | +| Classifier slug not configured/usable | Deterministic: routes to the cheapest candidate, no scoring. | +| A candidate's backend isn't usable | That candidate is skipped; routing continues with the rest. | +| Only one candidate available | Routes straight to it (no classifier call). | +| Task has images, candidate can't | That candidate is scored `0` (can't win). | +| Router disabled but `codex-auto` somehow requested | The slug isn't advertised; an explicit request falls through to the cheapest candidate. | + +--- + +## Notes + +- The Auto Router targets Codex's primary path, `POST /v1/responses` (and + `/v1/responses/compact`). It also applies to `/v1/chat/completions`. +- Candidates can be BYOK models **or** passthrough slugs (`gpt-5.5`, + `composer-2-5`) when those are available — the router routes to whatever is + usable. +- Per-task caching means you pay the classifier tax once per task, not once per + tool-call round-trip. Every decision is logged with `CODEX_SHIM_ROUTER_LOG=1`, + so you can measure whether routing is actually helping before trusting it. diff --git a/docs/subscription-integration.md b/docs/subscription-integration.md new file mode 100644 index 00000000..347f39f2 --- /dev/null +++ b/docs/subscription-integration.md @@ -0,0 +1,221 @@ +# Subscription passthrough integrations + +`codex-shim` can expose subscription-backed models without storing Dashboard +API keys in `~/.codex-shim/models.json`: + +- **ChatGPT/Codex passthrough** uses the Codex access token created by + `codex login` and forwards native `/v1/responses` requests to ChatGPT's Codex + backend. +- **Cursor/Composer passthrough** uses the local `cursor-agent` OAuth session + created by `cursor-agent login` and exposes Composer 2.5 as `composer-2-5`. + +Both integrations are optional, auth-gated, and advertised only when the local +login state is usable. They are different from BYOK routes: you do not add a +Dashboard API key for these subscription flows. + +--- + +## Quick check + +```bash +codex-shim doctor +codex-shim list +codex-shim status +``` + +Useful health fields: + +```json +{ + "chatgpt_passthrough": true, + "cursor_passthrough": true +} +``` + +`codex-shim doctor` is the safest first diagnostic because it does not start or +stop the daemon, write config, call model providers, or print token contents. + +--- + +## ChatGPT/Codex passthrough + +### What it does + +When `~/.codex/auth.json` exists and contains `tokens.access_token`, the shim +adds ChatGPT/Codex model slugs to discovery surfaces such as: + +- `codex-shim list` +- `/health` +- `/v1/models` +- the generated `.codex-shim/custom_model_catalog.json` + +Current fallback slugs include `gpt-5.5` and related GPT/Codex slugs. The shim +keeps Codex's native Responses payload shape and forwards it to: + +```text +https://chatgpt.com/backend-api/codex/responses +``` + +It sends the Codex access token as `Authorization: Bearer ...` and, when +present, the account id from `auth.json`. The token is not written into the +custom model catalog. + +### Setup + +```bash +codex login +codex-shim generate +codex-shim list +``` + +If `gpt-5.5` appears, you can select it from the Codex picker or run: + +```bash +codex-shim model use gpt-5.5 +``` + +For passthrough-only use, `~/.codex-shim/models.json` may be missing. The shim +can still generate a catalog containing subscription-backed entries when the +Codex auth file is valid. + +### Disable + +```bash +export CODEX_SHIM_DISABLE_CHATGPT=1 +``` + +After disabling, regenerate or restart the shim if you need discovery surfaces +to stop listing ChatGPT passthrough entries immediately. + +### Troubleshooting + +- Run `codex login` again if `codex-shim doctor` reports ChatGPT passthrough as + unavailable. +- Confirm the auth file exists at `~/.codex/auth.json`. Do not paste or upload + the file; it contains tokens. +- If the model picker still does not show GPT/Codex slugs, run + `codex-shim generate` and check `codex-shim list` before debugging Desktop + picker behavior. +- If `/health` reports `chatgpt_passthrough: false`, the daemon process may have + been started before login or with `CODEX_SHIM_DISABLE_CHATGPT` set. + +--- + +## Cursor/Composer passthrough + +### What it does + +When `cursor-agent status` reports an active login, the shim exposes Composer +2.5 as: + +```text +composer-2-5 +``` + +Requests to that slug are converted into a prompt for `cursor-agent --print` +using your local CLI OAuth session. This is subscription passthrough, not +Dashboard API-key billing. + +### Setup + +```bash +cursor-agent login +cursor-agent status +codex-shim generate +codex-shim list +``` + +Then select `Composer 2.5` in the picker or run: + +```bash +codex-shim model use composer-2-5 +``` + +The helper script is optional, but convenient: + +```bash +scripts/codex-shim-install-cursor-composer +``` + +It regenerates the local catalog/config and sets `composer-2-5` as the active +model when `cursor-agent status` reports an active login. + +### Binary and workspace overrides + +If `cursor-agent` is not on `PATH`, point the shim at it explicitly: + +```bash +export CURSOR_AGENT_BIN=/path/to/cursor-agent +``` + +By default, the cursor-agent child process runs in the current working +directory. Override that with: + +```bash +export CODEX_SHIM_CURSOR_WORKSPACE=/path/to/workspace +``` + +### Disable + +```bash +export CODEX_SHIM_DISABLE_CURSOR=1 +``` + +### Important: do not use Dashboard API keys for this flow + +Do **not** configure Composer through `cursor-api.standardagents.ai` unless you +intentionally want Dashboard API-key billing (`crsr_...`). For subscription +passthrough, the shim relies on `cursor-agent login` instead. + +The shim also removes `CURSOR_API_KEY` from the child `cursor-agent` environment +so a stale shell variable cannot override your CLI OAuth login. + +### Current limitations + +- The bridge is prompt-based because `cursor-agent --print` is a CLI interface, + not a native OpenAI/Anthropic provider endpoint. +- Image inputs are described/omitted in the prompt bridge rather than forwarded + as a native multimodal API payload. +- Tool-call fidelity is lower than native ChatGPT/Codex passthrough or BYOK + providers that support structured tool calls directly. + +### Troubleshooting + +- Run `cursor-agent status` first. If it says you are not logged in, run + `cursor-agent login`. +- Run `codex-shim doctor` and check the `Cursor passthrough` section. +- Check `/health`; `cursor_passthrough: true` means the daemon can expose + Composer. +- If the daemon was already running when you logged in, restart it so discovery + endpoints and generated catalog/config are refreshed. +- If `CURSOR_AGENT_BIN` is set, verify it points to an executable + `cursor-agent` binary. + +--- + +## Security and privacy notes + +- The generated catalog does not contain ChatGPT tokens or Cursor OAuth tokens. +- `codex-shim doctor` reports auth availability and paths, but does not print + token contents. +- ChatGPT passthrough reads `~/.codex/auth.json` at request time and forwards + the access token only to ChatGPT's Codex backend. +- Cursor passthrough spawns `cursor-agent` locally and sends the constructed + prompt to that CLI process through stdin. +- Do not share `~/.codex/auth.json`, shell history containing tokens, or any + request dump/log that may contain private prompts. + +--- + +## Subscription passthrough vs BYOK models + +| Flow | Credential source | Slug examples | Upstream shape | +|---|---|---|---| +| ChatGPT/Codex passthrough | `codex login` / `~/.codex/auth.json` | `gpt-5.5` | Native Codex Responses backend | +| Cursor/Composer passthrough | `cursor-agent login` | `composer-2-5` | `cursor-agent --print` bridge | +| BYOK OpenAI-compatible | `api_key` or `api_key_env` in settings | your configured slug | `/chat/completions` | +| BYOK Anthropic-compatible | `api_key` or `api_key_env` in settings | your configured slug | `/messages` | + +Use subscription passthrough when you want to spend subscription quota through +the local authenticated CLI. Use BYOK models when you want to route to a provider +endpoint and API key you control directly. diff --git a/examples/auto_router_demo.py b/examples/auto_router_demo.py new file mode 100644 index 00000000..a0284cc6 --- /dev/null +++ b/examples/auto_router_demo.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""auto_router_demo.py -- a live, offline proof that the Auto Router routes each +task to the right backend. + +It spins up a mock multi-backend server (a cheap classifier + three candidate +models with different price/capability), starts the REAL codex-shim server with +the router enabled, then sends a series of real Codex ``/v1/responses`` requests +of varying difficulty and prints which backend each one actually hit. + +No network, no API keys, standard library only (the shim itself needs aiohttp): + + python3 examples/auto_router_demo.py + +Expected: trivial -> cheapest model, medium -> mid model, hard -> strong model, +image task -> the only image-capable model, and a repeat task served from cache +without re-calling the classifier. +""" +import json +import os +import signal +import subprocess +import sys +import threading +import time +import urllib.request +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +MOCK_PORT = int(os.environ.get("DEMO_MOCK_PORT", "8799")) +SHIM_PORT = int(os.environ.get("DEMO_SHIM_PORT", "8766")) + +# Relative costs only matter for the cheapest-among-viable tie-break. +COST = {"cheap-real": 0.3, "mid-real": 1.0, "strong-real": 5.0} +ROUTED = {"cheap-real": "cheap", "mid-real": "mid", "strong-real": "strong"} + + +def _score_task(task: str): + """Stand-in for a real classifier: score each candidate 0-1 from the task. + A real run uses your configured classifier model; the routing logic the + shim applies to these scores is identical.""" + t = task.lower() + hard = any(k in t for k in ("refactor", "debug", "concurren", "architecture", "race condition", "across")) + medium = any(k in t for k in ("endpoint", "crud", "parse", "implement", "api", "migrate", "tests")) + if hard: + return {"cheap": 0.40, "mid": 0.55, "strong": 0.95} + if medium: + return {"cheap": 0.50, "mid": 0.85, "strong": 0.95} + return {"cheap": 0.90, "mid": 0.92, "strong": 0.95} + + +class Mock(BaseHTTPRequestHandler): + classifier_calls = 0 + last_scores = {} + last_backend = None + + def log_message(self, *a): + pass + + def _json(self, status, obj): + b = json.dumps(obj).encode() + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(b))) + self.end_headers() + self.wfile.write(b) + + def do_POST(self): + n = int(self.headers.get("Content-Length") or 0) + body = json.loads(self.rfile.read(n) if n else b"{}") + if not self.path.split("?")[0].endswith("/v1/chat/completions"): + self._json(404, {"e": "nope"}) + return + model = body.get("model") + if model == "classifier-real": + user = "\n".join( + m.get("content", "") + for m in body.get("messages", []) + if m.get("role") == "user" and isinstance(m.get("content"), str) + ) + task = "" + for line in user.splitlines(): + if line.strip().startswith("current_task") or task: + task += line + " " + scores = _score_task(task or user) + Mock.classifier_calls += 1 + Mock.last_scores = scores + self._json(200, { + "choices": [{"message": {"role": "assistant", "content": json.dumps({"scores": scores, "reasoning": "demo"})}}], + "usage": {"prompt_tokens": 20, "completion_tokens": 8, "total_tokens": 28}, + }) + return + Mock.last_backend = model + self._json(200, { + "choices": [{"message": {"role": "assistant", "content": "Handled by %s." % model}}], + "usage": {"prompt_tokens": 50, "completion_tokens": 10, "total_tokens": 60}, + }) + + +def _post_response(task, with_image=False): + if with_image: + content = [ + {"type": "input_text", "text": task}, + {"type": "input_image", "image_url": "data:image/png;base64,xx"}, + ] + payload = {"model": "codex-auto", "input": [{"role": "user", "content": content}]} + else: + payload = {"model": "codex-auto", "input": task} + data = json.dumps(payload).encode() + req = urllib.request.Request( + "http://127.0.0.1:%d/v1/responses" % SHIM_PORT, + data=data, method="POST", headers={"Content-Type": "application/json"}, + ) + urllib.request.urlopen(req, timeout=30).read() + + +def main(): + mock = "http://127.0.0.1:%d" % MOCK_PORT + srv = ThreadingHTTPServer(("127.0.0.1", MOCK_PORT), Mock) + threading.Thread(target=srv.serve_forever, daemon=True).start() + + settings = { + "models": [ + {"slug": "cheap", "model": "cheap-real", "display_name": "Cheap", "provider": "openai", "base_url": mock + "/v1", "api_key": "k"}, + {"slug": "mid", "model": "mid-real", "display_name": "Mid", "provider": "openai", "base_url": mock + "/v1", "api_key": "k"}, + {"slug": "strong", "model": "strong-real", "display_name": "Strong", "provider": "openai", "base_url": mock + "/v1", "api_key": "k"}, + {"slug": "classifier", "model": "classifier-real", "display_name": "Classifier", "provider": "openai", "base_url": mock + "/v1", "api_key": "k"}, + ], + "router": { + "enabled": True, + "slug": "codex-auto", + "classifier": "classifier", + "threshold": 0.7, + "default": "cheap", + "cache": True, + "candidates": [ + {"slug": "cheap", "cost": 0.3, "supports_images": False, "card": "Very cheap, fast. Single-file edits, codegen, simple changes."}, + {"slug": "mid", "cost": 1.0, "supports_images": False, "card": "Cheap generalist. Standard servers/CRUD, data processing, moderate multi-file edits."}, + {"slug": "strong", "cost": 5.0, "supports_images": True, "card": "Frontier. Big multi-file refactors, hard debugging, architecture, and image tasks."}, + ], + }, + } + cfg_f = os.path.join(REPO, "_demo_models.json") + with open(cfg_f, "w") as fh: + fh.write(json.dumps(settings)) + + env = dict( + os.environ, + PYTHONPATH=REPO + os.pathsep + os.environ.get("PYTHONPATH", ""), + CODEX_SHIM_DISABLE_CHATGPT="1", + CODEX_SHIM_DISABLE_CURSOR="1", + ) + proc = subprocess.Popen( + [sys.executable, "-m", "codex_shim.server", "--settings", cfg_f, "--port", str(SHIM_PORT)], + env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + + def fmt_scores(s): + return "cheap=%.2f mid=%.2f strong=%.2f" % (s.get("cheap", 0), s.get("mid", 0), s.get("strong", 0)) + + try: + for _ in range(80): + try: + urllib.request.urlopen("http://127.0.0.1:%d/health" % SHIM_PORT, timeout=2).read() + break + except Exception: + time.sleep(0.1) + + print("=" * 92) + print("codex-shim Auto Router -- live routing proof (offline, real server)") + print("=" * 92) + print("Candidates: cheap ($0.3, no images) | mid ($1.0, no images) | strong ($5.0, images)") + print("Rule: cheapest candidate scoring >= 0.70 wins; image tasks skip models that can't see.\n") + print("%-2s %-44s %-32s %-14s %s" % ("#", "Task", "Classifier scores", "Routed to", "Cost")) + print("-" * 92) + + cases = [ + ("add a docstring to the foo() helper", False, "cheap-real"), + ("write a CRUD REST endpoint with tests", False, "mid-real"), + ("refactor the auth module across 8 files and fix the race condition", False, "strong-real"), + ("what does this screenshot show?", True, "strong-real"), + ] + ok = True + for i, (task, img, expected) in enumerate(cases, 1): + Mock.last_backend = None + _post_response(task, with_image=img) + backend = Mock.last_backend + ok = ok and backend == expected + note = " <- only image-capable model" if img else "" + shown = (task[:41] + "...") if len(task) > 44 else task + print("%-2d %-44s %-32s %-14s $%-4s%s" % ( + i, shown, fmt_scores(Mock.last_scores), ROUTED.get(backend, "?"), COST.get(backend, "?"), note)) + + # Caching: repeat case #1; the classifier must NOT be called again. + calls_before = Mock.classifier_calls + Mock.last_backend = None + _post_response(cases[0][0], with_image=False) + cached = Mock.classifier_calls == calls_before + ok = ok and cached and Mock.last_backend == "cheap-real" + print("%-2s %-44s %-32s %-14s $%-4s <- %s" % ( + "5", "(repeat task #1)", "served from cache" if cached else "RE-SCORED (bug!)", + ROUTED.get(Mock.last_backend, "?"), COST.get(Mock.last_backend, "?"), + "cache hit: classifier not re-called" if cached else "cache MISS")) + + print("-" * 92) + print("Classifier was called %d times for 5 requests (caching saved 1)." % Mock.classifier_calls) + print("RESULT: PASS" if ok else "RESULT: FAIL") + return 0 if ok else 1 + finally: + proc.send_signal(signal.SIGTERM) + try: + proc.wait(timeout=5) + except Exception: + proc.kill() + srv.shutdown() + try: + os.remove(cfg_f) + except OSError: + pass + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/launch.ps1 b/launch.ps1 new file mode 100644 index 00000000..9815516f --- /dev/null +++ b/launch.ps1 @@ -0,0 +1,504 @@ +[Console]::OutputEncoding = [System.Text.Encoding]::UTF8 +$ErrorActionPreference = "Stop" + +$repoRoot = $PSScriptRoot +if (-not $repoRoot) { $repoRoot = Split-Path -Parent $MyInvocation.MyCommand.Path } +$port = 8766 +if ($env:CODEX_SHIM_PORT) { $port = [int]$env:CODEX_SHIM_PORT } +$templatePath = Join-Path $HOME ".codex-shim\models.json" +$resolvedPath = Join-Path $HOME ".codex-shim\models.resolved.json" +$catalogPath = Join-Path $repoRoot ".codex-shim\custom_model_catalog.json" + +$costInfo = @{ + "glm-5.1" = @{ cost = 0.014; tier = "Premium"; limit5h = 880; limitMo = 4300 } + "glm-5" = @{ cost = 0.010; tier = "Premium"; limit5h = 1150; limitMo = 5750 } + "kimi-k2.5" = @{ cost = 0.006; tier = "Standard"; limit5h = 1850; limitMo = 9250 } + "kimi-k2.6" = @{ cost = 0.010; tier = "Premium"; limit5h = 1150; limitMo = 5750 } + "deepseek-v4-pro" = @{ cost = 0.003; tier = "Standard"; limit5h = 3450; limitMo = 17150 } + "deepseek-v4-flash" = @{ cost = 0.0004; tier = "Economy"; limit5h = 31650; limitMo = 158150 } + "mimo-v2.5" = @{ cost = 0.0004; tier = "Economy"; limit5h = 30100; limitMo = 150400 } + "mimo-v2.5-pro" = @{ cost = 0.004; tier = "Standard"; limit5h = 3250; limitMo = 16300 } + "minimax-m2.7" = @{ cost = 0.004; tier = "Standard"; limit5h = 3400; limitMo = 17000 } + "minimax-m2.5" = @{ cost = 0.002; tier = "Standard"; limit5h = 6300; limitMo = 31800 } + "qwen3.7-max" = @{ cost = 0.012; tier = "Premium"; limit5h = 950; limitMo = 4770 } + "qwen3.6-plus" = @{ cost = 0.004; tier = "Standard"; limit5h = 3300; limitMo = 16300 } +} +$tierColor = @{ "Economy" = "Green"; "Standard" = "Yellow"; "Premium" = "Red" } + +function Get-ShimCommand { + $py = Get-Command "py" -ErrorAction SilentlyContinue + if ($py) { return [PSCustomObject]@{ File = $py.Source; Args = @("-3", "-m", "codex_shim.cli") } } + $python3 = Get-Command "python3" -ErrorAction SilentlyContinue + if ($python3) { return [PSCustomObject]@{ File = $python3.Source; Args = @("-m", "codex_shim.cli") } } + $python = Get-Command "python" -ErrorAction SilentlyContinue + if ($python) { return [PSCustomObject]@{ File = $python.Source; Args = @("-m", "codex_shim.cli") } } + $installed = Get-Command "codex-shim" -ErrorAction SilentlyContinue + if ($installed) { return [PSCustomObject]@{ File = $installed.Source; Args = @() } } + throw "Could not find Python or codex-shim on PATH. Install with: py -3.11 -m pip install --user -e ." +} + +$shimCommand = Get-ShimCommand + +function Invoke-Shim { + param([string[]]$Arguments) + + $oldPythonPath = $env:PYTHONPATH + try { + if ($oldPythonPath) { $env:PYTHONPATH = "$repoRoot;$oldPythonPath" } else { $env:PYTHONPATH = $repoRoot } + $baseArgs = @($shimCommand.Args) + @("--settings", $resolvedPath, "--port", [string]$port) + @($Arguments) + & $shimCommand.File @baseArgs + } finally { + if ($null -eq $oldPythonPath) { Remove-Item Env:PYTHONPATH -ErrorAction SilentlyContinue } else { $env:PYTHONPATH = $oldPythonPath } + } +} + +function Get-EnvVarValue($name) { + foreach ($scope in @("Process", "User", "Machine")) { + $value = [Environment]::GetEnvironmentVariable($name, $scope) + if ($value) { return $value } + } + return $null +} + +function Set-EnvVarForChild($name, $value) { + Set-Item -Path "Env:$name" -Value $value +} + +function Get-PlaceholderNames($text) { + $names = @{} + foreach ($match in [regex]::Matches($text, '%%([A-Za-z_][A-Za-z0-9_]*)%%')) { $names[$match.Groups[1].Value] = $true } + foreach ($match in [regex]::Matches($text, '\$\{([A-Za-z_][A-Za-z0-9_]*)\}')) { $names[$match.Groups[1].Value] = $true } + return @($names.Keys) +} + +function Get-ZaiQuota { + $key = Get-EnvVarValue "ZAI_API_KEY" + if (-not $key) { return $null } + try { + return Invoke-RestMethod -Uri "https://api.z.ai/api/monitor/usage/quota/limit" -Headers @{ + Authorization = $key + "Accept-Language" = "en-US,en" + "Content-Type" = "application/json" + } -TimeoutSec 8 + } catch { return $null } +} + +function Get-CodexQuota { + $authPath = Join-Path $HOME ".codex\auth.json" + if (-not (Test-Path $authPath)) { return $null } + try { + $auth = Get-Content $authPath -Raw | ConvertFrom-Json + $token = $auth.tokens.access_token + $acctId = $auth.tokens.account_id + if (-not $token -or -not $acctId) { return $null } + return Invoke-RestMethod -Uri "https://chatgpt.com/backend-api/wham/usage" -Headers @{ + Authorization = "Bearer $token" + Accept = "application/json" + "ChatGPT-Account-Id" = $acctId + Origin = "https://chatgpt.com" + Referer = "https://chatgpt.com/" + "User-Agent" = "Mozilla/5.0" + } -TimeoutSec 8 + } catch { return $null } +} + +function Format-ResetTime($ts) { + if ($null -eq $ts) { return "unknown" } + $dt = [DateTimeOffset]::FromUnixTimeMilliseconds([int64]$ts).LocalDateTime + $diff = $dt - (Get-Date) + if ($diff.TotalSeconds -le 0) { return "now" } + if ($diff.TotalHours -gt 24) { return "{0}d {1}h" -f [math]::Floor($diff.TotalDays), $diff.Hours } + return "{0}h {1}m" -f [math]::Floor($diff.TotalHours), $diff.Minutes +} + +function Format-PercentBar($pct) { + if ($null -eq $pct) { $pct = 0 } + $pct = [math]::Min(100, [math]::Max(0, [double]$pct)) + $filled = [int][math]::Floor($pct / 10) + $empty = 10 - $filled + $bar = ("#" * $filled) + ("-" * $empty) + if ($pct -ge 90) { $color = "Red" } + elseif ($pct -ge 60) { $color = "Yellow" } + else { $color = "Green" } + return @{ bar = $bar; color = $color } +} + +function Format-TitleCase($value) { + if (-not $value) { return "" } + $text = [string]$value + return $text.Substring(0, 1).ToUpper() + $text.Substring(1).ToLower() +} + +function Get-CodexDisplayName { + $codexQ = Get-CodexQuota + if (-not $codexQ -or -not $codexQ.rate_limit) { return "GPT-5.5" } + $plan = ([string]$codexQ.plan_type).ToUpper() + $pri = $codexQ.rate_limit.primary_window + $sec = $codexQ.rate_limit.secondary_window + $priReset = "" + if ($pri.reset_after_seconds) { + $h = [math]::Floor($pri.reset_after_seconds / 3600) + $m = [math]::Floor(($pri.reset_after_seconds % 3600) / 60) + $priReset = " ${h}h${m}m" + } + return "GPT-5.5 ($plan | 5h:$($pri.used_percent)%$priReset wk:$($sec.used_percent)%)" +} + +function Get-ModelRows($json) { + if ($json -is [array]) { return @($json) } + foreach ($key in @("models", "customModels", "launchModels", "launch_models")) { + if ($json.PSObject.Properties.Name -contains $key) { return @($json.$key) } + } + return @() +} + +function Get-ModelId($model) { + if ($model.PSObject.Properties.Name -contains "model") { return [string]$model.model } + return "" +} + +function Get-ModelDisplayName($model) { + foreach ($key in @("display_name", "displayName", "name")) { + if ($model.PSObject.Properties.Name -contains $key) { return [string]$model.$key } + } + return (Get-ModelId $model) +} + +function Set-ModelDisplayName($model, $value) { + foreach ($key in @("display_name", "displayName", "name")) { + if ($model.PSObject.Properties.Name -contains $key) { $model.$key = $value; return } + } + $model | Add-Member -NotePropertyName "display_name" -NotePropertyValue $value -Force +} + +function Resolve-Keys { + if (Test-Path $templatePath) { + $raw = [System.IO.File]::ReadAllText($templatePath) + } else { + Write-Host " No $templatePath found; using ChatGPT/Codex passthrough only." -ForegroundColor Yellow + $raw = '{"models":[]}' + } + + $placeholderNames = Get-PlaceholderNames $raw + $missing = @() + foreach ($varName in $placeholderNames) { + $val = Get-EnvVarValue $varName + if ($val) { Set-EnvVarForChild $varName $val } else { $missing += $varName } + } + foreach ($varName in $missing) { + Write-Host " Missing env var: $varName" -ForegroundColor Red + Write-Host " setx $varName `"your-key`"" -ForegroundColor White + Write-Host " Enter key for $varName (or Enter to skip): " -NoNewline -ForegroundColor Green + $inputKey = Read-Host + if ($inputKey) { + Set-EnvVarForChild $varName $inputKey + setx $varName $inputKey | Out-Null + Write-Host " Saved $varName for future shells." -ForegroundColor Green + } else { + Write-Host " Continuing without $varName; models using it may be unavailable." -ForegroundColor Yellow + } + } + + $raw = [regex]::Replace($raw, '%%([A-Za-z_][A-Za-z0-9_]*)%%', { param($match) '${' + $match.Groups[1].Value + '}' }) + try { + $json = $raw | ConvertFrom-Json + } catch { + Write-Host " $templatePath is not valid JSON: $_" -ForegroundColor Red + return $false + } + + $zaiResp = Get-ZaiQuota + $zaiQ = $null + if ($zaiResp -and $zaiResp.success) { $zaiQ = $zaiResp.data } + $zaiLevel = $null + $zaiTag = "" + if ($zaiQ) { + $zaiLevel = Format-TitleCase $zaiQ.level + $tok5h = ($zaiQ.limits | Where-Object { $_.type -eq "TOKENS_LIMIT" -and $_.unit -eq 3 } | Select-Object -First 1).percentage + $week = ($zaiQ.limits | Where-Object { $_.type -eq "TOKENS_LIMIT" -and $_.unit -eq 6 } | Select-Object -First 1).percentage + $zaiTag = " ($zaiLevel | 5h:${tok5h}% wk:${week}%)" + } + + foreach ($m in Get-ModelRows $json) { + $modelId = Get-ModelId $m + $displayName = Get-ModelDisplayName $m + if ($zaiQ -and ($modelId -eq "glm-5.1" -or $displayName -eq "Z.AI GLM 5.1")) { + Set-ModelDisplayName $m "Z.AI $zaiLevel | GLM 5.1$zaiTag" + } elseif ($zaiQ -and ($modelId -eq "glm-5" -or $displayName -eq "Z.AI GLM 5")) { + Set-ModelDisplayName $m "Z.AI $zaiLevel | GLM 5$zaiTag" + } + if ($displayName -match "^Go ") { + $c = $costInfo[$modelId] + if ($c) { + $baseName = $displayName.Substring(3) -replace '\s+\|\s+(Economy|Standard|Premium)\s+~\$.*?/req$', '' + Set-ModelDisplayName $m "Go $baseName | $($c.tier) ~`$$($c.cost)/req" + } + } + } + + $resolvedDir = Split-Path -Parent $resolvedPath + if (-not (Test-Path $resolvedDir)) { New-Item -ItemType Directory -Path $resolvedDir | Out-Null } + $resolvedJson = ConvertTo-Json -InputObject $json -Depth 20 + [System.IO.File]::WriteAllText($resolvedPath, $resolvedJson + "`n", (New-Object System.Text.UTF8Encoding $false)) + return $true +} + +function Patch-Catalog { + if (-not (Test-Path $catalogPath)) { return } + $newName = Get-CodexDisplayName + if ($newName -eq "GPT-5.5") { return } + try { + $catalog = Get-Content $catalogPath -Raw | ConvertFrom-Json + } catch { return } + + $changed = $false + foreach ($m in @($catalog.models)) { + if ($m.slug -eq "gpt-5.5") { + $m.display_name = $newName + if ($m.model_messages -and $m.model_messages.instructions_variables) { + $m.model_messages.instructions_variables.model_name = $newName + } + $changed = $true + } + } + if ($changed) { + $raw = ConvertTo-Json -InputObject $catalog -Depth 20 + [System.IO.File]::WriteAllText($catalogPath, $raw + "`n", (New-Object System.Text.UTF8Encoding $false)) + } +} + +function Ensure-ShimRunning { + $status = Invoke-Shim @("status") 2>&1 + if ($LASTEXITCODE -eq 0 -and ($status -match "running")) { return } + Write-Host "`n Starting shim on port $port..." -ForegroundColor Yellow + Invoke-Shim @("start") 2>&1 | ForEach-Object { Write-Host " $_" } + if ($LASTEXITCODE -ne 0) { throw "codex-shim failed to start." } + Start-Sleep -Milliseconds 500 +} + +function Get-Models { + $output = Invoke-Shim @("model", "list") 2>&1 + $models = @() + foreach ($line in $output) { + if ($line -match '^(\S+)\s{2,}(.+)\s+->\s+(.+)\s+\(([^()]*)\)$') { + $models += [PSCustomObject]@{ + Slug = $Matches[1].Trim() + Name = $Matches[2].Trim() + UpModel = $Matches[3].Trim() + Provider = $Matches[4].Trim() + } + } + } + return $models +} + +function Set-LoopbackNoProxy { + foreach ($key in @("NO_PROXY", "no_proxy")) { + $values = @() + if ((Get-Item "Env:$key" -ErrorAction SilentlyContinue) -and (Get-Item "Env:$key").Value) { + $values = ((Get-Item "Env:$key").Value -split ',') | ForEach-Object { $_.Trim() } | Where-Object { $_ } + } + foreach ($hostName in @("127.0.0.1", "localhost", "::1")) { + if (($values | ForEach-Object { $_.ToLowerInvariant() }) -notcontains $hostName.ToLowerInvariant()) { $values += $hostName } + } + Set-Item -Path "Env:$key" -Value ($values -join ',') + } +} + +function Launch-CodexApp { + $codex = Get-Command "codex" -ErrorAction SilentlyContinue + if (-not $codex) { throw "codex command not found on PATH. Install/authenticate Codex CLI first." } + Set-LoopbackNoProxy + & codex app . +} + +function Show-Menu { + Clear-Host + Write-Host "" + Write-Host " ========================================" -ForegroundColor Cyan + Write-Host " CODEX-SHIM MODEL LAUNCHER" -ForegroundColor Cyan + Write-Host " ========================================" -ForegroundColor Cyan + Write-Host "" + + # --- Z.AI Live Quota --- + $zaiResp = Get-ZaiQuota + $zaiQ = $null + if ($zaiResp -and $zaiResp.success) { $zaiQ = $zaiResp.data } + if ($zaiQ) { + $level = ([string]$zaiQ.level).ToUpper() + Write-Host " Z.AI CODING PLAN ($level)" -ForegroundColor Magenta + foreach ($lim in $zaiQ.limits) { + if ($lim.type -eq "TIME_LIMIT") { + $pct = $lim.percentage + $pb = Format-PercentBar $pct + $reset = Format-ResetTime $lim.nextResetTime + Write-Host " MCP Monthly: " -NoNewline + Write-Host "[$($pb.bar)]" -NoNewline -ForegroundColor $pb.color + Write-Host " $($lim.currentValue)/$($lim.number) calls (${pct}%) resets in $reset" -ForegroundColor DarkGray + } + if ($lim.type -eq "TOKENS_LIMIT" -and $lim.unit -eq 3) { + $pct = $lim.percentage + $pb = Format-PercentBar $pct + $reset = Format-ResetTime $lim.nextResetTime + Write-Host " 5h Tokens: " -NoNewline + Write-Host "[$($pb.bar)]" -NoNewline -ForegroundColor $pb.color + Write-Host " ${pct}% used resets in $reset" -ForegroundColor DarkGray + } + if ($lim.type -eq "TOKENS_LIMIT" -and $lim.unit -eq 6) { + $pct = $lim.percentage + $pb = Format-PercentBar $pct + $reset = Format-ResetTime $lim.nextResetTime + Write-Host " Weekly: " -NoNewline + Write-Host "[$($pb.bar)]" -NoNewline -ForegroundColor $pb.color + Write-Host " ${pct}% used resets in $reset" -ForegroundColor DarkGray + } + } + Write-Host "" + } + + # --- Codex/ChatGPT Live Quota --- + $codexQ = Get-CodexQuota + if ($codexQ -and $codexQ.rate_limit) { + $plan = ([string]$codexQ.plan_type).ToUpper() + $pri = $codexQ.rate_limit.primary_window + $sec = $codexQ.rate_limit.secondary_window + $limitReached = $codexQ.rate_limit.limit_reached + Write-Host " CODEX / CHATGPT ($plan)" -ForegroundColor DarkYellow + if ($limitReached) { + Write-Host " STATUS: " -NoNewline; Write-Host "LIMIT REACHED" -ForegroundColor Red + } + $pb5h = Format-PercentBar $pri.used_percent + $reset5h = "" + if ($pri.reset_after_seconds) { + $h = [math]::Floor($pri.reset_after_seconds / 3600) + $m = [math]::Floor(($pri.reset_after_seconds % 3600) / 60) + $reset5h = " resets in ${h}h ${m}m" + } + Write-Host " 5h Window: " -NoNewline + Write-Host "[$($pb5h.bar)]" -NoNewline -ForegroundColor $pb5h.color + Write-Host " $($pri.used_percent)% used$reset5h" -ForegroundColor DarkGray + $pbWk = Format-PercentBar $sec.used_percent + $resetWk = "" + if ($sec.reset_after_seconds) { + $d = [math]::Floor($sec.reset_after_seconds / 86400) + $h = [math]::Floor(($sec.reset_after_seconds % 86400) / 3600) + $resetWk = " resets in ${d}d ${h}h" + } + Write-Host " Weekly: " -NoNewline + Write-Host "[$($pbWk.bar)]" -NoNewline -ForegroundColor $pbWk.color + Write-Host " $($sec.used_percent)% used$resetWk" -ForegroundColor DarkGray + Write-Host "" + } + + $models = Get-Models + if ($models.Count -eq 0) { + Write-Host " No models found." -ForegroundColor Red + Write-Host "" + pause + return $null + } + + # Build display-ordered list (matches what user sees) + $zai = @($models | Where-Object { $_.Name -match "^Z\.AI" }) + $go = @($models | Where-Object { $_.Name -match "^Go " }) + $other = @($models | Where-Object { $_.Name -notmatch "^(Z\.AI|Go )" }) + $ordered = @($zai) + @($go) + @($other) + + $i = 1 + if ($zai.Count -gt 0) { + Write-Host " -- Z.AI Models --------------------------" -ForegroundColor Magenta + foreach ($m in $zai) { + Write-Host " [$i] " -NoNewline -ForegroundColor Yellow + Write-Host "$($m.Name)" -ForegroundColor White + $i++ + } + Write-Host "" + } + + if ($go.Count -gt 0) { + Write-Host " -- OpenCode Go (5h=`$12 wk=`$30 mo=`$60) --" -ForegroundColor Cyan + Write-Host " Quota: https://opencode.ai/auth" -ForegroundColor Blue + Write-Host " Model Cost/req Tier ~5h ~Month" -ForegroundColor DarkGray + Write-Host " ------ --------- ------ ------ ------" -ForegroundColor DarkGray + foreach ($m in $go) { + $c = $costInfo[$m.UpModel] + if ($c) { + $tc = $tierColor[$c.tier] + Write-Host " [$i] " -NoNewline -ForegroundColor Yellow + Write-Host ("{0,-22}" -f $m.Name) -NoNewline -ForegroundColor White + Write-Host ("${0:N4}" -f $c.cost) -NoNewline -ForegroundColor DarkGray + Write-Host " " -NoNewline + Write-Host ("{0,-10}" -f $c.tier) -NoNewline -ForegroundColor $tc + Write-Host ("{0,6}req" -f $c.limit5h) -NoNewline -ForegroundColor DarkGray + Write-Host (" {0,7}req" -f $c.limitMo) -ForegroundColor DarkGray + } else { + Write-Host " [$i] " -NoNewline -ForegroundColor Yellow + Write-Host $m.Name -ForegroundColor White + } + $i++ + } + Write-Host "" + } + + if ($other.Count -gt 0) { + Write-Host " -- Other --------------------------------" -ForegroundColor DarkYellow + foreach ($m in $other) { + Write-Host " [$i] " -NoNewline -ForegroundColor Yellow + Write-Host "$($m.Name)" -ForegroundColor White + $i++ + } + Write-Host "" + } + + Write-Host " [S] Restart shim [Q] Quit" -ForegroundColor Yellow + Write-Host "" + return $ordered +} + +# --- Main --- +if (-not (Resolve-Keys)) { Write-Host ""; pause; exit 1 } +try { + Ensure-ShimRunning + Patch-Catalog +} catch { + Write-Host " $_" -ForegroundColor Red + pause + exit 1 +} + +while ($true) { + $models = Show-Menu + if ($null -eq $models) { break } + + Write-Host " Choose: " -NoNewline -ForegroundColor Green + $choice = (Read-Host).Trim() + + if ($choice -eq "q" -or $choice -eq "Q") { break } + + if ($choice -eq "s" -or $choice -eq "S") { + Invoke-Shim @("stop") 2>&1 | Out-Null + Start-Sleep 1 + Ensure-ShimRunning + Patch-Catalog + Write-Host " Shim restarted.`n" -ForegroundColor Green + pause + continue + } + + $idx = 0 + if ([int]::TryParse($choice, [ref]$idx) -and $idx -ge 1 -and $idx -le $models.Count) { + $picked = $models[$idx - 1] + Ensure-ShimRunning + Write-Host "`n Switching to $($picked.Name)..." -ForegroundColor Yellow + Invoke-Shim @("model", "use", $picked.Slug) 2>&1 | ForEach-Object { Write-Host " $_" } + if ($LASTEXITCODE -ne 0) { throw "codex-shim model use failed." } + Patch-Catalog + + Write-Host "`n Launching Codex Desktop..." -ForegroundColor Yellow + Launch-CodexApp + break + } else { + Write-Host " Invalid choice." -ForegroundColor Red + Start-Sleep 1 + } +} diff --git a/scripts/codex-shim-install-cursor-composer b/scripts/codex-shim-install-cursor-composer new file mode 100755 index 00000000..84232dee --- /dev/null +++ b/scripts/codex-shim-install-cursor-composer @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -euo pipefail + +export PATH="${HOME}/.local/bin:/opt/homebrew/bin:${PATH}" + +if ! command -v cursor-agent >/dev/null 2>&1; then + echo "cursor-agent is not installed." >&2 + echo "Install Cursor CLI, then run: cursor-agent login" >&2 + exit 1 +fi + +# Match codex-shim subscription-first auth: stale CURSOR_API_KEY must not block login. +if ! env -u CURSOR_API_KEY cursor-agent status >/dev/null 2>&1; then + echo "Cursor CLI is not logged in." >&2 + echo "Run: cursor-agent login" >&2 + exit 1 +fi + +# Composer uses built-in Cursor subscription passthrough in codex-shim. +# Remove any stale BYOK standardagents entry that would require CURSOR_API_KEY. +MODELS_JSON="${HOME}/.codex-shim/models.json" +mkdir -p "${HOME}/.codex-shim" +python3 - "${MODELS_JSON}" <<'PY' +import json +import sys +from pathlib import Path + +path = Path(sys.argv[1]) +payload = {"models": []} +if path.exists(): + try: + data = json.loads(path.read_text()) + except json.JSONDecodeError: + data = {} + rows = data.get("models") if isinstance(data, dict) else [] + if isinstance(rows, list): + payload["models"] = [ + row for row in rows + if not ( + isinstance(row, dict) + and str(row.get("slug") or row.get("model") or "").replace(".", "-") + in {"composer-2-5", "composer-2.5"} + ) + ] +path.write_text(json.dumps(payload, indent=2) + "\n") +PY + +codex-shim restart +codex-shim generate +codex-shim model use composer-2-5 + +if [[ "$(uname -s)" == "Darwin" ]]; then + codex-shim patch-app || echo "Warning: patch-app failed; Desktop picker may need manual patch." >&2 +fi + +echo "Composer 2.5 is wired through your Cursor subscription (cursor-agent login)." +echo "No Dashboard API key (crsr_...) is required." +echo +codex-shim list +echo +echo "Relaunch Desktop with: codex-app" diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 00000000..46105c1c --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,20 @@ +from __future__ import annotations + +import pytest + + +@pytest.fixture(autouse=True) +def _disable_cursor_passthrough_by_default(monkeypatch, request): + if "cursor_present" in request.fixturenames: + return + + def _off(**_kwargs): + return False + + for target in ( + "codex_shim.cursor_passthrough.cursor_passthrough_available", + "codex_shim.server.cursor_passthrough_available", + "codex_shim.catalog.cursor_passthrough_available", + "codex_shim.cli.cursor_passthrough_available", + ): + monkeypatch.setattr(target, _off, raising=False) diff --git a/tests/test_cli_doctor.py b/tests/test_cli_doctor.py new file mode 100644 index 00000000..52667a31 --- /dev/null +++ b/tests/test_cli_doctor.py @@ -0,0 +1,274 @@ +from __future__ import annotations + +import json +from types import SimpleNamespace + +import pytest + +from codex_shim import cli + + +@pytest.fixture(autouse=True) +def doctor_safe_environment(monkeypatch, tmp_path): + monkeypatch.setattr(cli, "CATALOG_PATH", tmp_path / "custom_model_catalog.json") + monkeypatch.setattr(cli, "CONFIG_PATH", tmp_path / "config.toml") + monkeypatch.setattr(cli, "PID_PATH", tmp_path / "shim.pid") + monkeypatch.setattr(cli, "LOG_PATH", tmp_path / "shim.log") + monkeypatch.setattr(cli, "CODEX_CONFIG_PATH", tmp_path / "codex-config.toml") + monkeypatch.setattr(cli, "DEFAULT_CODEX_AUTH", tmp_path / "auth.json") + monkeypatch.setattr("codex_shim.settings.DEFAULT_CURSOR_API_KEY_FILE", tmp_path / "missing-cursor-api-key") + monkeypatch.delenv("CURSOR_API_KEY", raising=False) + monkeypatch.delenv("CURSOR_AGENT_BIN", raising=False) + monkeypatch.delenv("CODEX_SHIM_DISABLE_CHATGPT", raising=False) + monkeypatch.delenv("CODEX_SHIM_DISABLE_CURSOR", raising=False) + monkeypatch.delenv("CODEX_SHIM_DISABLE_ROUTER", raising=False) + monkeypatch.setenv("NO_PROXY", "127.0.0.1,localhost,::1") + monkeypatch.delenv("no_proxy", raising=False) + monkeypatch.setattr(cli.importlib.util, "find_spec", lambda name: object()) + monkeypatch.setattr(cli.shutil, "which", lambda command: None) + monkeypatch.setattr(cli, "_health", lambda port: None) + monkeypatch.setattr(cli, "_read_pid", lambda: None) + monkeypatch.setattr(cli, "_pid_running", lambda pid: False) + monkeypatch.setattr( + cli, + "available_model_slugs", + lambda models: {model.slug for model in models if model.api_key.strip()}, + ) + monkeypatch.setattr(cli, "chatgpt_passthrough_available", lambda: False) + monkeypatch.setattr(cli, "cursor_passthrough_available", lambda: False) + + +def _settings(path, models, router=None): + data = {"models": models} + if router is not None: + data["router"] = router + path.write_text(json.dumps(data)) + return path + + +def test_doctor_command_returns_zero_with_healthy_mocked_environment(monkeypatch, tmp_path, capsys): + settings = _settings( + tmp_path / "models.json", + [ + { + "model": "claude-upstream", + "display_name": "Claude Upstream", + "provider": "anthropic", + "base_url": "https://example.invalid/v1", + "api_key": "secret-anthropic", + }, + { + "model": "gpt-upstream", + "display_name": "GPT Upstream", + "provider": "generic-chat-completion-api", + "base_url": "https://example.invalid/v1", + "api_key": "secret-openai", + }, + ], + router={ + "enabled": True, + "slug": "codex-auto", + "classifier": "gpt-upstream", + "candidates": [{"slug": "claude-upstream"}], + }, + ) + monkeypatch.setattr( + cli.shutil, + "which", + lambda command: {"codex": "/usr/local/bin/codex", "cursor-agent": "/usr/local/bin/cursor-agent"}.get(command), + ) + monkeypatch.setattr( + cli.subprocess, + "run", + lambda *args, **kwargs: SimpleNamespace(returncode=0, stdout="codex-cli 0.133.0-alpha.1\n", stderr=""), + ) + monkeypatch.setattr(cli, "_read_pid", lambda: 12345) + monkeypatch.setattr(cli, "_pid_running", lambda pid: pid == 12345) + monkeypatch.setattr( + cli, + "_health", + lambda port: { + "ok": True, + "models": 4, + "chatgpt_passthrough": True, + "cursor_passthrough": True, + "auto_router": True, + }, + ) + monkeypatch.setattr(cli, "chatgpt_passthrough_available", lambda: True) + monkeypatch.setattr(cli, "cursor_passthrough_available", lambda: True) + + code = cli.main(["--settings", str(settings), "--port", "8765", "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + for section in ( + "Python", + "Dependencies", + "Codex CLI", + "Settings", + "Runtime files", + "Shim daemon", + "ChatGPT passthrough", + "Cursor passthrough", + "Proxy", + "Codex config", + "Summary", + ): + assert section in out + assert "health ok: 4 models" in out + assert "auto router active: codex-auto" in out + assert "secret-anthropic" not in out + assert "secret-openai" not in out + + +def test_invalid_settings_json_returns_one(tmp_path, capsys): + settings = tmp_path / "broken.json" + settings.write_text('{"models": [') + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 1 + assert "invalid JSON" in out + assert "Summary" in out + + +def test_missing_settings_file_warns_without_failing(tmp_path, capsys): + settings = tmp_path / "missing.json" + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "settings file not found" in out + assert "Summary" in out + + +def test_missing_api_key_is_warn(tmp_path, capsys): + settings = _settings( + tmp_path / "models.json", + [ + { + "model": "missing-key-model", + "display_name": "Missing Key Model", + "provider": "generic-chat-completion-api", + "base_url": "https://example.invalid/v1", + } + ], + ) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "models missing API keys: 1" in out + + +def test_no_proxy_complete_is_ok(monkeypatch, tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + monkeypatch.setenv("NO_PROXY", "127.0.0.1,localhost,::1") + monkeypatch.delenv("no_proxy", raising=False) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "loopback hosts covered by NO_PROXY/no_proxy" in out + assert "NO_PROXY/no_proxy does not include all loopback hosts" not in out + + +def test_no_proxy_missing_is_warn(monkeypatch, tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + monkeypatch.delenv("NO_PROXY", raising=False) + monkeypatch.delenv("no_proxy", raising=False) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "NO_PROXY/no_proxy does not include all loopback hosts" in out + + +def test_daemon_health_output(monkeypatch, tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + monkeypatch.setattr( + cli, + "_health", + lambda port: { + "ok": True, + "models": 2, + "chatgpt_passthrough": True, + "cursor_passthrough": False, + "auto_router": True, + }, + ) + + code = cli.main(["--settings", str(settings), "--port", "8765", "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "health ok: 2 models" in out + assert "chatgpt_passthrough: true" in out + assert "cursor_passthrough: false" in out + assert "auto_router: true" in out + + +def test_codex_cli_not_found_is_warn_not_fail(monkeypatch, tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + monkeypatch.setattr(cli.shutil, "which", lambda command: None) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "codex not found on PATH" in out + + +def test_aiohttp_missing_is_fail(monkeypatch, tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + monkeypatch.setattr( + cli.importlib.util, + "find_spec", + lambda name: None if name == "aiohttp" else object(), + ) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 1 + assert "aiohttp is not importable" in out + assert "FAIL" in out + + +def test_codex_config_installed_reports_provider_and_active_model(tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + cli.CODEX_CONFIG_PATH.write_text( + "\n".join( + [ + cli.MANAGED_BEGIN, + 'model = "gpt-5.5"', + 'model_provider = "codex_shim"', + cli.MANAGED_END, + "[model_providers.codex_shim]", + 'name = "codex_shim"', + ] + ) + ) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "shim provider configured" in out + assert "active shim model: gpt-5.5" in out + + +def test_codex_config_uninstalled_is_info(tmp_path, capsys): + settings = _settings(tmp_path / "models.json", []) + + code = cli.main(["--settings", str(settings), "doctor"]) + + out = capsys.readouterr().out + assert code == 0 + assert "shim provider is not currently installed" in out diff --git a/tests/test_cursor_passthrough.py b/tests/test_cursor_passthrough.py new file mode 100644 index 00000000..03f2d7ef --- /dev/null +++ b/tests/test_cursor_passthrough.py @@ -0,0 +1,113 @@ +from __future__ import annotations + +import json + +from codex_shim.cursor_passthrough import ( + CursorStreamParser, + build_cursor_prompt, + is_cursor_passthrough_slug, + iter_cursor_agent_events, +) + + +def test_is_cursor_passthrough_slug(): + assert is_cursor_passthrough_slug("composer-2-5") + assert is_cursor_passthrough_slug("composer-2.5") + assert not is_cursor_passthrough_slug("gpt-5.5") + + +def test_build_cursor_prompt_from_responses_body(): + body = { + "model": "composer-2-5", + "instructions": "You are Codex.", + "input": [{"role": "user", "content": "Hello"}], + } + prompt = build_cursor_prompt(body) + assert "You are Codex." in prompt + assert "Hello" in prompt + + +def test_cursor_stream_parser_emits_deltas(): + parser = CursorStreamParser() + line1 = json.dumps( + { + "type": "assistant", + "message": {"role": "assistant", "content": [{"type": "text", "text": "Hel"}]}, + "timestamp_ms": 1, + } + ) + line2 = json.dumps( + { + "type": "assistant", + "message": {"role": "assistant", "content": [{"type": "text", "text": "Hello"}]}, + "timestamp_ms": 2, + } + ) + assert parser.feed_line(line1) == "Hel" + assert parser.feed_line(line2) == "lo" + + +async def test_iter_cursor_agent_events_does_not_kill_normal_completion(monkeypatch): + class FakeStdin: + def write(self, data): + self.data = data + + async def drain(self): + pass + + def close(self): + pass + + class FakeReader: + def __init__(self, chunks): + self.chunks = list(chunks) + + async def read(self, _size): + if self.chunks: + return self.chunks.pop(0) + return b"" + + class FakeProc: + def __init__(self): + self.stdin = FakeStdin() + self.stdout = FakeReader( + [ + json.dumps( + { + "type": "result", + "subtype": "success", + "result": "Hello", + } + ).encode() + + b"\n", + b"", + ] + ) + self.stderr = FakeReader([b""]) + self.returncode = None + self.killed = False + + def kill(self): + self.killed = True + self.returncode = -9 + + async def wait(self): + if self.returncode is None: + self.returncode = 0 + return self.returncode + + proc = FakeProc() + + async def fake_create_subprocess_exec(*_args, **_kwargs): + return proc + + monkeypatch.setattr( + "codex_shim.cursor_passthrough.asyncio.create_subprocess_exec", + fake_create_subprocess_exec, + ) + + events = [event async for event in iter_cursor_agent_events("prompt", "composer-2.5")] + + assert proc.killed is False + assert proc.returncode == 0 + assert events[-1] == {"type": "completed", "text": "Hello"} diff --git a/tests/test_hostguard.py b/tests/test_hostguard.py new file mode 100644 index 00000000..f32d14dd --- /dev/null +++ b/tests/test_hostguard.py @@ -0,0 +1,111 @@ +from __future__ import annotations + +import json + +import pytest +from aiohttp.test_utils import TestClient, TestServer + +from codex_shim.hostguard import build_allowed_hosts, host_only +from codex_shim.server import ShimServer + + +@pytest.mark.parametrize( + "header,expected", + [ + ("127.0.0.1:8765", "127.0.0.1"), + ("localhost:8765", "localhost"), + ("127.0.0.1", "127.0.0.1"), + ("[::1]:8765", "::1"), + ("[::1]", "::1"), + ("attacker.example:8765", "attacker.example"), + ("attacker.example", "attacker.example"), + ("", ""), + (" 127.0.0.1:8765 ", "127.0.0.1"), + ], +) +def test_host_only_strips_port(header, expected): + assert host_only(header) == expected + + +def test_build_allowed_hosts_defaults_to_loopback(): + assert build_allowed_hosts("127.0.0.1") == {"127.0.0.1", "localhost", "::1"} + + +def test_build_allowed_hosts_adds_bind_host_and_env(monkeypatch): + monkeypatch.setenv("CODEX_SHIM_ALLOWED_HOSTS", "shim.lan , Extra.Host") + allowed = build_allowed_hosts("192.168.1.5") + assert {"127.0.0.1", "localhost", "::1", "192.168.1.5", "shim.lan", "extra.host"} <= allowed + + +def test_build_allowed_hosts_ignores_wildcard_bind(): + assert build_allowed_hosts("0.0.0.0") == {"127.0.0.1", "localhost", "::1"} + + +@pytest.fixture +def settings_file(tmp_path): + path = tmp_path / "settings.json" + path.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "openai", + "baseUrl": "http://example.invalid/v1", + "apiKey": "secret", + } + ] + } + ) + ) + return path + + +async def test_loopback_host_is_allowed(settings_file): + client = TestClient(TestServer(ShimServer(settings_file).app())) + await client.start_server() + try: + resp = await client.get("/health") + assert resp.status == 200 + finally: + await client.close() + + +async def test_rebinding_host_is_rejected_on_health(settings_file): + client = TestClient(TestServer(ShimServer(settings_file).app())) + await client.start_server() + try: + resp = await client.get("/health", headers={"Host": "attacker.example"}) + assert resp.status == 403 + finally: + await client.close() + + +async def test_rebinding_host_is_rejected_before_upstream_forwarding(settings_file): + # A POST with an attacker Host must be refused before the shim forwards the + # request upstream with the user's credentials. example.invalid would fail + # to resolve if we ever reached the upstream call, so a 403 proves the guard + # short-circuits first. + client = TestClient(TestServer(ShimServer(settings_file).app())) + await client.start_server() + try: + resp = await client.post( + "/v1/responses", + json={"model": "real-openai", "input": "hi"}, + headers={"Host": "attacker.example"}, + ) + assert resp.status == 403 + finally: + await client.close() + + +async def test_env_allowlisted_host_is_allowed(monkeypatch, settings_file): + monkeypatch.setenv("CODEX_SHIM_ALLOWED_HOSTS", "shim.local") + client = TestClient(TestServer(ShimServer(settings_file).app())) + await client.start_server() + try: + resp = await client.get("/health", headers={"Host": "shim.local"}) + assert resp.status == 200 + finally: + await client.close() diff --git a/tests/test_native_tool_types.py b/tests/test_native_tool_types.py new file mode 100644 index 00000000..425faa35 --- /dev/null +++ b/tests/test_native_tool_types.py @@ -0,0 +1,160 @@ +"""Tests for native tool type mapping in the streaming translator and non-streaming response helpers.""" +from __future__ import annotations + +import json +from codex_shim.translate import chat_completion_to_response, anthropic_to_response +from codex_shim.server import _build_tool_types + + +def test_build_tool_types_native_tools(): + """Native tools like apply_patch and web_search are preserved.""" + body = { + "tools": [ + {"type": "apply_patch"}, + {"type": "web_search_preview"}, + {"type": "local_shell"}, + ] + } + tool_types = _build_tool_types(body) + assert tool_types["apply_patch"] == "apply_patch" + assert tool_types["web_search_preview"] == "web_search_preview" + assert tool_types["local_shell"] == "local_shell" + + +def test_build_tool_types_mcp_tools(): + """MCP tools with function names are preserved.""" + body = { + "tools": [ + {"type": "mcp__node_repl", "function": {"name": "js"}}, + {"type": "mcp__node_repl", "function": {"name": "eval"}}, + ] + } + tool_types = _build_tool_types(body) + assert tool_types["js"] == "mcp__node_repl" + assert tool_types["eval"] == "mcp__node_repl" + + +def test_build_tool_types_empty_and_missing(): + """Empty or missing tools arrays return empty dict.""" + assert _build_tool_types({}) == {} + assert _build_tool_types({"tools": []}) == {} + assert _build_tool_types({"tools": None}) == {} + + +def test_chat_completion_to_response_apply_patch_custom_tool_call(): + """apply_patch tool type maps to custom_tool_call output item.""" + payload = { + "choices": [ + { + "message": { + "content": "", + "tool_calls": [ + { + "id": "call_1", + "function": {"name": "apply_patch", "arguments": '{"patch": "diff"}'}, + } + ], + } + } + ] + } + tool_types = {"apply_patch": "apply_patch"} + response = chat_completion_to_response(payload, "model", tool_types) + output = response["output"] + call_items = [o for o in output if o["type"] in ("function_call", "custom_tool_call", "web_search_call")] + assert len(call_items) == 1 + assert call_items[0]["type"] == "custom_tool_call" + assert call_items[0]["name"] == "apply_patch" + + +def test_chat_completion_to_response_web_search_call(): + """web_search tool type maps to web_search_call output item.""" + payload = { + "choices": [ + { + "message": { + "content": "", + "tool_calls": [ + { + "id": "call_2", + "function": {"name": "web_search", "arguments": '{"query": "test"}'}, + } + ], + } + } + ] + } + tool_types = {"web_search": "web_search"} + response = chat_completion_to_response(payload, "model", tool_types) + output = response["output"] + call_items = [o for o in output if o["type"] in ("function_call", "custom_tool_call", "web_search_call")] + assert len(call_items) == 1 + assert call_items[0]["type"] == "web_search_call" + assert call_items[0]["name"] == "web_search" + + +def test_chat_completion_to_response_unknown_tool_function_call(): + """Unknown tool types fall back to generic function_call.""" + payload = { + "choices": [ + { + "message": { + "content": "", + "tool_calls": [ + { + "id": "call_3", + "function": {"name": "random_tool", "arguments": '{"x": 1}'}, + } + ], + } + } + ] + } + tool_types = {"random_tool": "mcp__random"} + response = chat_completion_to_response(payload, "model", tool_types) + output = response["output"] + call_items = [o for o in output if o["type"] in ("function_call", "custom_tool_call", "web_search_call")] + assert len(call_items) == 1 + assert call_items[0]["type"] == "function_call" + assert call_items[0]["name"] == "random_tool" + + +def test_anthropic_to_response_with_tool_types(): + """Anthropic path also maps apply_patch to custom_tool_call.""" + payload = { + "content": [ + {"type": "text", "text": "hello"}, + {"type": "tool_use", "id": "tu_1", "name": "apply_patch", "input": {"patch": "diff"}}, + ], + "id": "msg_1", + } + tool_types = {"apply_patch": "apply_patch"} + response = anthropic_to_response(payload, "model", tool_types) + output = response["output"] + call_items = [o for o in output if o["type"] in ("function_call", "custom_tool_call", "web_search_call")] + assert len(call_items) == 1 + assert call_items[0]["type"] == "custom_tool_call" + + +def test_chat_completion_to_response_no_tool_types_backward_compat(): + """Without tool_types, everything falls back to function_call (backward compat).""" + payload = { + "choices": [ + { + "message": { + "content": "", + "tool_calls": [ + { + "id": "call_1", + "function": {"name": "apply_patch", "arguments": '{"patch": "diff"}'}, + } + ], + } + } + ] + } + response = chat_completion_to_response(payload, "model") + output = response["output"] + call_items = [o for o in output if o["type"] in ("function_call", "custom_tool_call", "web_search_call")] + assert len(call_items) == 1 + assert call_items[0]["type"] == "function_call" diff --git a/tests/test_router.py b/tests/test_router.py new file mode 100644 index 00000000..0b1a6e6c --- /dev/null +++ b/tests/test_router.py @@ -0,0 +1,454 @@ +from __future__ import annotations + +import json + +import pytest +from aiohttp import web +from aiohttp.test_utils import TestClient, TestServer + +from codex_shim import router +from codex_shim.server import ShimServer +from codex_shim.catalog import write_catalog + + +@pytest.fixture(autouse=True) +def _reset_router_cache(): + router.reset_cache() + yield + router.reset_cache() + + +@pytest.fixture +def auth_missing(monkeypatch, tmp_path): + """Keep ChatGPT passthrough out of discovery so model lists are deterministic.""" + missing = tmp_path / "missing-auth.json" + monkeypatch.setattr("codex_shim.settings.DEFAULT_CODEX_AUTH", missing) + monkeypatch.setattr("codex_shim.server.DEFAULT_CODEX_AUTH", missing) + + +CANDIDATES = [ + router.RouterCandidate(slug="cheap", cost=1.0, card="cheap fast model", supports_images=False), + router.RouterCandidate(slug="strong", cost=5.0, card="frontier model", supports_images=True), +] + + +def _settings_with_router(tmp_path, upstream_v1, *, cache=False, enabled=True, classifier="classifier"): + settings = tmp_path / "models.json" + settings.write_text( + json.dumps( + { + "models": [ + {"slug": "cheap", "model": "cheap-real", "display_name": "Cheap", "provider": "openai", "base_url": upstream_v1, "api_key": "k"}, + {"slug": "strong", "model": "strong-real", "display_name": "Strong", "provider": "openai", "base_url": upstream_v1, "api_key": "k"}, + {"slug": "classifier", "model": "classifier-real", "display_name": "Classifier", "provider": "openai", "base_url": upstream_v1, "api_key": "k"}, + ], + "router": { + "enabled": enabled, + "slug": "codex-auto", + "display_name": "Auto (smart routing)", + "classifier": classifier, + "threshold": 0.7, + "default": "cheap", + "cache": cache, + "candidates": [ + {"slug": "cheap", "cost": 1, "supports_images": False, "card": "cheap fast single-file edits"}, + {"slug": "strong", "cost": 5, "supports_images": True, "card": "frontier multi-file refactors, debugging, images"}, + ], + }, + } + ) + ) + return settings + + +# --------------------------------------------------------------------------- +# Config loading +# --------------------------------------------------------------------------- +def test_load_router_config_parses_block(tmp_path): + config = router.load_router_config(_settings_with_router(tmp_path, "http://x/v1")) + assert config is not None + assert config.enabled is True + assert config.slug == "codex-auto" + assert config.classifier == "classifier" + assert config.threshold == 0.7 + assert [c.slug for c in config.candidates] == ["cheap", "strong"] + assert config.candidates[1].supports_images is True + + +def test_load_router_config_absent_returns_none(tmp_path): + settings = tmp_path / "models.json" + settings.write_text(json.dumps({"models": []})) + assert router.load_router_config(settings) is None + assert router.load_router_config(tmp_path / "nope.json") is None + + +def test_disable_via_env_overrides_enabled(tmp_path, monkeypatch): + config = router.load_router_config(_settings_with_router(tmp_path, "http://x/v1")) + assert config.effective_enabled is True + monkeypatch.setenv("CODEX_SHIM_DISABLE_ROUTER", "1") + assert config.effective_enabled is False + + +def test_env_overrides_timeout_and_max_tokens(tmp_path, monkeypatch): + monkeypatch.setenv("CODEX_SHIM_ROUTER_TIMEOUT", "3") + monkeypatch.setenv("CODEX_SHIM_ROUTER_MAX_TOKENS", "42") + config = router.load_router_config(_settings_with_router(tmp_path, "http://x/v1")) + assert config.timeout == 3.0 + assert config.max_tokens == 42 + + +# --------------------------------------------------------------------------- +# Task signal extraction +# --------------------------------------------------------------------------- +def test_latest_user_text_from_responses_input(): + body = { + "input": [ + {"role": "user", "content": "first ask"}, + {"type": "function_call", "name": "shell", "arguments": "{}"}, + {"type": "function_call_output", "call_id": "c1", "output": "done"}, + ] + } + assert router.latest_user_text(body) == "first ask" + + +def test_latest_user_text_from_chat_messages(): + body = {"messages": [{"role": "system", "content": "x"}, {"role": "user", "content": "hello there"}]} + assert router.latest_user_text(body) == "hello there" + + +def test_has_images_detects_input_image(): + body = {"input": [{"role": "user", "content": [{"type": "input_text", "text": "look"}, {"type": "input_image", "image_url": "data:..."}]}]} + assert router.has_images(body) is True + assert router.has_images({"input": "just text"}) is False + + +def test_task_signal_counts_tools_and_items(): + body = {"input": [{"role": "user", "content": "do it"}], "tools": [{"type": "function"}, {"type": "function"}]} + signal = router.task_signal(body) + assert signal["task"] == "do it" + assert signal["tool_count"] == 2 + assert signal["input_items"] == 1 + assert signal["has_images"] is False + + +# --------------------------------------------------------------------------- +# Score parsing + selection +# --------------------------------------------------------------------------- +def test_parse_scores_pulls_json_object(): + text = 'Sure! {"scores": {"cheap": 0.4, "strong": 1.5}, "reasoning": "x"}' + scores = router.parse_scores(text, ["cheap", "strong"]) + assert scores == {"cheap": 0.4, "strong": 1.0} # clamped + + +def test_parse_scores_handles_garbage(): + assert router.parse_scores("no json here", ["cheap"]) == {} + assert router.parse_scores("", ["cheap"]) == {} + + +def test_pick_candidate_prefers_cheapest_viable(): + scores = {"cheap": 0.9, "strong": 0.95} + slug, score, why = router.pick_candidate(scores, CANDIDATES, 0.7, has_image_task=False) + assert slug == "cheap" + assert score == 0.9 + + +def test_pick_candidate_escalates_when_cheap_below_bar(): + scores = {"cheap": 0.4, "strong": 0.95} + slug, _score, _why = router.pick_candidate(scores, CANDIDATES, 0.7, has_image_task=False) + assert slug == "strong" + + +def test_pick_candidate_hard_zeros_image_incapable(): + scores = {"cheap": 0.99, "strong": 0.8} + slug, _score, _why = router.pick_candidate(scores, CANDIDATES, 0.7, has_image_task=True) + assert slug == "strong" + + +def test_pick_candidate_below_bar_takes_best(): + scores = {"cheap": 0.5, "strong": 0.6} + slug, _score, why = router.pick_candidate(scores, CANDIDATES, 0.7, has_image_task=False) + assert slug == "strong" + assert "below bar" in why + + +def test_fallback_slug_uses_default_then_cheapest(): + config = router.RouterConfig( + enabled=True, slug="codex-auto", display_name="Auto", classifier=None, + threshold=0.7, default="strong", cache=True, candidates=tuple(CANDIDATES), timeout=12.0, max_tokens=600, + ) + assert router.fallback_slug(config, list(CANDIDATES)) == "strong" + config_no_default = router.RouterConfig( + enabled=True, slug="codex-auto", display_name="Auto", classifier=None, + threshold=0.7, default=None, cache=True, candidates=tuple(CANDIDATES), timeout=12.0, max_tokens=600, + ) + assert router.fallback_slug(config_no_default, list(CANDIDATES)) == "cheap" + + +def test_fallback_slug_skips_image_incapable_when_task_has_image(): + """When the task has images and the classifier-free fallback path runs + (e.g. scoring returned all zeros, or the classifier was unreachable), the + chosen fallback MUST be a candidate that can actually see images. + + Regression: a multi-modal task fell through to ``default = cheap`` (text-only), + the request was forwarded to a vision-incapable upstream, and the upstream + rejected the body with `unknown variant ``image_url`` `. The user-visible + contract is "Auto Router never breaks a request"; sending an image task to + a text-only model breaks that contract. + """ + config = router.RouterConfig( + enabled=True, slug="codex-auto", display_name="Auto", classifier=None, + threshold=0.7, + # default points at the text-only cheap model on purpose: that is the + # configuration we ship with, so the fix must work for it. + default="cheap", + cache=True, candidates=tuple(CANDIDATES), timeout=12.0, max_tokens=600, + ) + # Has-image=False: keep current behaviour (default wins). + assert router.fallback_slug(config, list(CANDIDATES), has_image_task=False) == "cheap" + # Has-image=True: must skip "cheap" (supports_images=False) and pick the + # vision-capable candidate, even though it is not the configured default. + assert router.fallback_slug(config, list(CANDIDATES), has_image_task=True) == "strong" + + +def test_fallback_slug_image_task_with_no_vision_candidate_returns_none(): + """If a task has images but no candidate supports them, fallback returns + None rather than knowingly routing to a model that will reject the body. + Returning None bubbles up to server.py which can surface the failure + instead of producing a confusing 400 from the upstream. + """ + text_only = [router.RouterCandidate(slug="only-text", cost=1.0, card="x", supports_images=False)] + config = router.RouterConfig( + enabled=True, slug="codex-auto", display_name="Auto", classifier=None, + threshold=0.7, default="only-text", cache=True, candidates=tuple(text_only), + timeout=12.0, max_tokens=600, + ) + assert router.fallback_slug(config, text_only, has_image_task=True) is None + + +def test_fallback_slug_default_kwarg_preserves_old_call_sites(): + """``has_image_task`` must default to False so existing call sites and + tests that don't pass the kwarg keep their previous behaviour.""" + config = router.RouterConfig( + enabled=True, slug="codex-auto", display_name="Auto", classifier=None, + threshold=0.7, default="strong", cache=True, candidates=tuple(CANDIDATES), + timeout=12.0, max_tokens=600, + ) + # No kwarg = old behaviour = honor default ("strong"), regardless of vision. + assert router.fallback_slug(config, list(CANDIDATES)) == "strong" + + +# --------------------------------------------------------------------------- +# resolve_auto orchestration (with an injected fake classifier) +# --------------------------------------------------------------------------- +def _config(**overrides): + base = dict( + enabled=True, slug="codex-auto", display_name="Auto", classifier="classifier", + threshold=0.7, default="cheap", cache=True, candidates=tuple(CANDIDATES), timeout=12.0, max_tokens=600, + ) + base.update(overrides) + return router.RouterConfig(**base) + + +async def test_resolve_single_candidate_skips_classifier(): + calls = [] + + async def classify(_s, _u): + calls.append(1) + return "{}" + + slug, _info = await router.resolve_auto(_config(), [CANDIDATES[0]], {"input": "hi"}, classify) + assert slug == "cheap" + assert calls == [] + + +async def test_resolve_no_classifier_is_deterministic(): + slug, info = await router.resolve_auto(_config(), list(CANDIDATES), {"input": "hi"}, None) + assert slug == "cheap" # cheapest fallback + assert info["reason"] == "no classifier" + + +async def test_resolve_uses_classifier_scores(): + async def classify(_s, _u): + return json.dumps({"scores": {"cheap": 0.4, "strong": 0.95}}) + + slug, _info = await router.resolve_auto(_config(cache=False), list(CANDIDATES), {"input": "hard refactor"}, classify) + assert slug == "strong" + + +async def test_resolve_caches_decision(): + calls = [] + + async def classify(_s, _u): + calls.append(1) + return json.dumps({"scores": {"cheap": 0.9, "strong": 0.95}}) + + body = {"input": "add a docstring"} + first, _ = await router.resolve_auto(_config(cache=True), list(CANDIDATES), body, classify) + second, info = await router.resolve_auto(_config(cache=True), list(CANDIDATES), body, classify) + assert first == second == "cheap" + assert len(calls) == 1 # second served from cache + assert info["reason"] == "cache" + + +async def test_resolve_classifier_error_falls_back(): + async def classify(_s, _u): + raise RuntimeError("boom") + + slug, info = await router.resolve_auto(_config(default="strong", cache=False), list(CANDIDATES), {"input": "x"}, classify) + assert slug == "strong" + assert info["reason"] == "classifier error" + + +# --------------------------------------------------------------------------- +# End-to-end through the real ShimServer +# --------------------------------------------------------------------------- +async def _make_upstream(captured): + async def chat(request): + body = await request.json() + model = body.get("model") + if model == "classifier-real": + user = " ".join( + m.get("content", "") for m in body.get("messages", []) if m.get("role") == "user" and isinstance(m.get("content"), str) + ) + hard = any(k in user.lower() for k in ("refactor", "across", "debug")) + scores = {"cheap": 0.4 if hard else 0.9, "strong": 0.95} + return web.json_response( + {"choices": [{"message": {"role": "assistant", "content": json.dumps({"scores": scores})}}], "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8}} + ) + captured["backend"] = model + return web.json_response( + {"choices": [{"message": {"role": "assistant", "content": f"handled by {model}"}}], "usage": {"prompt_tokens": 2, "completion_tokens": 1, "total_tokens": 3}} + ) + + app = web.Application() + app.router.add_post("/v1/chat/completions", chat) + client = TestClient(TestServer(app)) + await client.start_server() + return client + + +async def test_auto_router_routes_trivial_to_cheap(tmp_path, auth_missing): + captured = {} + upstream = await _make_upstream(captured) + settings = _settings_with_router(tmp_path, str(upstream.make_url("/v1"))) + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "add a docstring to foo()"}) + assert resp.status == 200 + payload = await resp.json() + assert payload["output"][0]["content"][0]["text"] == "handled by cheap-real" + assert captured["backend"] == "cheap-real" + + await shim.close() + await upstream.close() + + +async def test_auto_router_escalates_hard_task_to_strong(tmp_path, auth_missing): + captured = {} + upstream = await _make_upstream(captured) + settings = _settings_with_router(tmp_path, str(upstream.make_url("/v1"))) + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + resp = await shim.post( + "/v1/responses", + json={"model": "codex-auto", "input": "refactor the auth module across 8 files"}, + ) + assert resp.status == 200 + payload = await resp.json() + assert captured["backend"] == "strong-real" + assert payload["output"][0]["content"][0]["text"] == "handled by strong-real" + + await shim.close() + await upstream.close() + + +async def test_auto_router_image_task_picks_image_capable(tmp_path, auth_missing): + captured = {} + upstream = await _make_upstream(captured) + settings = _settings_with_router(tmp_path, str(upstream.make_url("/v1"))) + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + resp = await shim.post( + "/v1/responses", + json={ + "model": "codex-auto", + "input": [ + {"role": "user", "content": [ + {"type": "input_text", "text": "what is in this screenshot?"}, + {"type": "input_image", "image_url": "data:image/png;base64,xx"}, + ]} + ], + }, + ) + assert resp.status == 200 + # The cheap model scores high but can't see; only the image-capable model is viable. + assert captured["backend"] == "strong-real" + + await shim.close() + await upstream.close() + + +async def test_auto_router_falls_back_when_classifier_missing(tmp_path, auth_missing): + captured = {} + upstream = await _make_upstream(captured) + # Point classifier at a slug that does not exist -> deterministic cheapest. + settings = _settings_with_router(tmp_path, str(upstream.make_url("/v1")), classifier="nonexistent") + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "refactor across files"}) + assert resp.status == 200 + assert captured["backend"] == "cheap-real" # cheapest, no scoring + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Discovery: the virtual model shows up only when active +# --------------------------------------------------------------------------- +async def test_discovery_includes_auto_model(tmp_path, auth_missing): + settings = _settings_with_router(tmp_path, "http://upstream.invalid/v1") + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + models = await (await shim.get("/v1/models")).json() + assert "codex-auto" in [m["id"] for m in models["data"]] + + api = await (await shim.get("/api/models")).json() + auto = [m for m in api if m["slug"] == "codex-auto"] + assert auto and auto[0]["provider"] == "auto" + + health = await (await shim.get("/health")).json() + assert health["auto_router"] is True + + await shim.close() + + +async def test_disabled_router_not_in_discovery(tmp_path, auth_missing): + settings = _settings_with_router(tmp_path, "http://upstream.invalid/v1", enabled=False) + shim = TestClient(TestServer(ShimServer(settings).app())) + await shim.start_server() + + models = await (await shim.get("/v1/models")).json() + assert "codex-auto" not in [m["id"] for m in models["data"]] + health = await (await shim.get("/health")).json() + assert health["auto_router"] is False + + await shim.close() + + +def test_write_catalog_includes_auto_entry(tmp_path, auth_missing): + from codex_shim.settings import ModelSettings + + settings = _settings_with_router(tmp_path, "http://upstream.invalid/v1") + models = ModelSettings(settings).load() + config = router.load_router_config(settings) + catalog_path = tmp_path / "catalog.json" + write_catalog(models, catalog_path, router_config=config) + data = json.loads(catalog_path.read_text()) + slugs = [m["slug"] for m in data["models"]] + assert slugs[0] == "codex-auto" diff --git a/tests/test_router_integration.py b/tests/test_router_integration.py new file mode 100644 index 00000000..1c593843 --- /dev/null +++ b/tests/test_router_integration.py @@ -0,0 +1,624 @@ +"""End-to-end integration tests for the Auto Router against the real ShimServer. + +These prove the router behaves correctly on the paths Codex actually uses: +streaming, compaction, the chat endpoint, the agent tool-call loop (per-task +cache), an Anthropic-shaped classifier, the exact HTTP the shim sends to the +classifier, and the full set of failure/edge cases. Everything runs offline +against mock upstreams — no keys, no network. +""" +from __future__ import annotations + +import asyncio +import json + +import pytest +from aiohttp import web +from aiohttp.test_utils import TestClient, TestServer + +from codex_shim import router +from codex_shim import server as server_module +from codex_shim.server import PICKER_TOKEN_HEADER, ShimServer + + +@pytest.fixture(autouse=True) +def _reset_router_cache(): + router.reset_cache() + yield + router.reset_cache() + + +@pytest.fixture(autouse=True) +def auth_missing(monkeypatch, tmp_path): + """Keep ChatGPT passthrough out of discovery so model lists are deterministic.""" + missing = tmp_path / "missing-auth.json" + monkeypatch.setattr("codex_shim.settings.DEFAULT_CODEX_AUTH", missing) + monkeypatch.setattr("codex_shim.server.DEFAULT_CODEX_AUTH", missing) + + +# --------------------------------------------------------------------------- +# A versatile mock upstream: OpenAI chat completions + Anthropic messages, with +# request capture, streaming, and a classifier that scores from the task text. +# --------------------------------------------------------------------------- +def _ci_get(headers: dict, name: str): + for key, value in headers.items(): + if key.lower() == name.lower(): + return value + return None + + +def _user_text_from_chat(messages): + for message in messages: + if message.get("role") == "user" and isinstance(message.get("content"), str): + return message["content"] + return "" + + +def _default_score(user_text: str) -> dict: + t = user_text.lower() + if any(k in t for k in ("refactor", "across", "debug", "architecture", "race condition")): + return {"cheap": 0.40, "strong": 0.95} + if any(k in t for k in ("crud", "endpoint", "api", "tests", "implement", "parser")): + return {"cheap": 0.55, "strong": 0.95} + return {"cheap": 0.92, "strong": 0.96} + + +async def make_upstream(state): + state.setdefault("chat_requests", []) + state.setdefault("anthropic_requests", []) + state.setdefault("classifier_calls", 0) + state.setdefault("last_backend", None) + state.setdefault("last_scores", None) + state.setdefault("classifier_models", {"classifier-real", "claude-classifier-real"}) + state.setdefault("score_fn", _default_score) + state.setdefault("fixed_scores", None) + + def _scores(user_text): + if state["fixed_scores"] is not None: + return state["fixed_scores"] + return state["score_fn"](user_text) + + async def chat(request): + body = await request.json() + state["chat_requests"].append({"body": body, "headers": dict(request.headers)}) + model = body.get("model") + if model in state["classifier_models"]: + state["classifier_calls"] += 1 + scores = _scores(_user_text_from_chat(body.get("messages", []))) + state["last_scores"] = scores + return web.json_response( + { + "choices": [{"message": {"role": "assistant", "content": json.dumps({"scores": scores, "reasoning": "x"})}}], + "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8}, + } + ) + state["last_backend"] = model + if body.get("stream"): + resp = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await resp.prepare(request) + await resp.write(("data: %s\n\n" % json.dumps({"choices": [{"delta": {"content": f"hi from {model}"}}]})).encode()) + await resp.write( + ("data: %s\n\n" % json.dumps({"choices": [{"delta": {}}], "usage": {"prompt_tokens": 2, "completion_tokens": 1, "total_tokens": 3}})).encode() + ) + await resp.write(b"data: [DONE]\n\n") + await resp.write_eof() + return resp + return web.json_response( + { + "choices": [{"message": {"role": "assistant", "content": f"handled by {model}"}}], + "usage": {"prompt_tokens": 2, "completion_tokens": 1, "total_tokens": 3}, + } + ) + + async def messages(request): + body = await request.json() + state["anthropic_requests"].append({"body": body, "headers": dict(request.headers)}) + model = body.get("model") + if model in state["classifier_models"]: + state["classifier_calls"] += 1 + user_text = body.get("messages", [{}])[-1].get("content", "") + if isinstance(user_text, list): + user_text = " ".join(str(p.get("text", "")) for p in user_text if isinstance(p, dict)) + scores = _scores(str(user_text)) + state["last_scores"] = scores + return web.json_response( + {"content": [{"type": "text", "text": json.dumps({"scores": scores})}], "stop_reason": "end_turn", "usage": {"input_tokens": 5, "output_tokens": 3}} + ) + state["last_backend"] = model + return web.json_response( + {"content": [{"type": "text", "text": f"handled by {model}"}], "stop_reason": "end_turn", "usage": {"input_tokens": 2, "output_tokens": 1}} + ) + + app = web.Application() + app.router.add_post("/v1/chat/completions", chat) + app.router.add_post("/v1/messages", messages) + client = TestClient(TestServer(app)) + await client.start_server() + return client + + +def _settings( + tmp_path, + upstream_v1, + *, + classifier="classifier", + classifier_provider="openai", + classifier_model="classifier-real", + threshold=0.7, + cache=True, + enabled=True, + strong_provider="openai", + extra_models=None, + candidates=None, +): + models = [ + {"slug": "cheap", "model": "cheap-real", "display_name": "Cheap", "provider": "openai", "base_url": upstream_v1, "api_key": "ck"}, + {"slug": "strong", "model": "strong-real", "display_name": "Strong", "provider": strong_provider, "base_url": upstream_v1, "api_key": "sk"}, + {"slug": "classifier", "model": classifier_model, "display_name": "Classifier", "provider": classifier_provider, "base_url": upstream_v1, "api_key": "xk"}, + ] + if extra_models: + models.extend(extra_models) + if candidates is None: + candidates = [ + {"slug": "cheap", "cost": 1, "supports_images": False, "card": "cheap fast single-file edits"}, + {"slug": "strong", "cost": 5, "supports_images": True, "card": "frontier multi-file refactors, debugging, images"}, + ] + settings = tmp_path / "models.json" + settings.write_text( + json.dumps( + { + "models": models, + "router": { + "enabled": enabled, + "slug": "codex-auto", + "display_name": "Auto (smart routing)", + "classifier": classifier, + "threshold": threshold, + "default": "cheap", + "cache": cache, + "candidates": candidates, + }, + } + ) + ) + return settings + + +async def _shim(settings): + client = TestClient(TestServer(ShimServer(settings).app())) + await client.start_server() + return client + + +def _sse_events(text: str) -> list[dict]: + events = [] + for block in text.split("\n\n"): + if not block.startswith("data:"): + continue + data = block.removeprefix("data:").strip() + if data and data != "[DONE]": + events.append(json.loads(data)) + return events + + +# --------------------------------------------------------------------------- +# Streaming +# --------------------------------------------------------------------------- +async def test_streaming_through_auto_router_completes_and_routes(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "refactor across modules", "stream": True}) + assert resp.status == 200 + events = _sse_events(await resp.text()) + completed = [e for e in events if e.get("type") == "response.completed"] + assert completed, "missing response.completed event" + assert state["last_backend"] == "strong-real" + assert completed[-1]["response"]["usage"]["total_tokens"] == 3 + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Compaction +# --------------------------------------------------------------------------- +async def test_compact_through_auto_router(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + resp = await shim.post( + "/v1/responses/compact", + json={ + "model": "codex-auto", + "input": [ + {"role": "user", "content": "implement the parser"}, + {"type": "function_call_output", "call_id": "c1", "output": "done"}, + ], + "stream": True, + }, + ) + assert resp.status == 200 + payload = await resp.json() + assert payload["status"] == "completed" + # medium task -> strong; the compact request reaches the routed backend. + assert state["last_backend"] == "strong-real" + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Chat-completions endpoint +# --------------------------------------------------------------------------- +async def test_chat_completions_through_auto_router(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + resp = await shim.post( + "/v1/chat/completions", + json={"model": "codex-auto", "messages": [{"role": "user", "content": "add a docstring"}]}, + ) + assert resp.status == 200 + payload = await resp.json() + assert payload["choices"][0]["message"]["content"] == "handled by cheap-real" + assert state["last_backend"] == "cheap-real" + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# The agent tool-loop: per-task cache means one classification across many turns +# --------------------------------------------------------------------------- +async def test_tool_loop_reuses_one_classification_for_a_task(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), cache=True)) + + # Turn 1: the original ask. + await shim.post("/v1/responses", json={"model": "codex-auto", "input": [{"role": "user", "content": "implement the parser"}]}) + backend_1 = state["last_backend"] + + # Turns 2 and 3: the same task continuing through tool-call round-trips. + for _ in range(2): + state["last_backend"] = None + await shim.post( + "/v1/responses", + json={ + "model": "codex-auto", + "input": [ + {"role": "user", "content": "implement the parser"}, + {"type": "function_call", "name": "shell", "arguments": "{}", "call_id": "c1"}, + {"type": "function_call_output", "call_id": "c1", "output": "ran tests"}, + ], + }, + ) + assert state["last_backend"] == backend_1 # same backend each turn + + assert state["classifier_calls"] == 1 # classifier paid once for the whole task + + await shim.close() + await upstream.close() + + +async def test_distinct_tasks_each_get_classified(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), cache=True)) + + await shim.post("/v1/responses", json={"model": "codex-auto", "input": "add a docstring"}) + await shim.post("/v1/responses", json={"model": "codex-auto", "input": "refactor across the codebase"}) + assert state["classifier_calls"] == 2 + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Anthropic-shaped classifier +# --------------------------------------------------------------------------- +async def test_anthropic_classifier_scores_and_routes(tmp_path): + state = {} + upstream = await make_upstream(state) + settings = _settings( + tmp_path, + str(upstream.make_url("/v1")), + classifier_provider="anthropic", + classifier_model="claude-classifier-real", + ) + shim = await _shim(settings) + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "refactor across files"}) + assert resp.status == 200 + assert state["last_backend"] == "strong-real" + # The classifier call went out as an Anthropic /v1/messages request. + classifier_reqs = [r for r in state["anthropic_requests"] if r["body"].get("model") == "claude-classifier-real"] + assert len(classifier_reqs) == 1 + assert _ci_get(classifier_reqs[0]["headers"], "x-api-key") == "xk" + assert _ci_get(classifier_reqs[0]["headers"], "anthropic-version") + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Anthropic-shaped candidate (the routed model itself speaks Anthropic) +# --------------------------------------------------------------------------- +async def test_anthropic_candidate_is_routable(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), strong_provider="anthropic")) + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "debug the race condition"}) + assert resp.status == 200 + payload = await resp.json() + assert state["last_backend"] == "strong-real" + assert payload["output"][0]["content"][0]["text"] == "handled by strong-real" + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# The exact request the shim sends to the classifier +# --------------------------------------------------------------------------- +async def test_classifier_request_payload_and_headers(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + await shim.post("/v1/responses", json={"model": "codex-auto", "input": "write CRUD endpoints"}) + + classifier_reqs = [r for r in state["chat_requests"] if r["body"].get("model") == "classifier-real"] + assert len(classifier_reqs) == 1 + req = classifier_reqs[0] + body = req["body"] + assert body["stream"] is False + assert body["temperature"] == 0 + assert body["max_tokens"] == 600 # default + assert _ci_get(req["headers"], "Authorization") == "Bearer xk" + + system = body["messages"][0] + user = body["messages"][1] + assert system["role"] == "system" + assert "task-routing classifier" in system["content"] + # Capability cards + candidate slugs are handed to the classifier. + assert "cheap fast single-file edits" in system["content"] + assert "frontier multi-file refactors" in system["content"] + assert "slug: cheap" in system["content"] and "slug: strong" in system["content"] + assert user["role"] == "user" + assert "write CRUD endpoints" in user["content"] + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# No overhead for normal (non-auto) requests +# --------------------------------------------------------------------------- +async def test_non_auto_request_never_calls_classifier(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + resp = await shim.post("/v1/responses", json={"model": "cheap", "input": "anything"}) + assert resp.status == 200 + assert state["classifier_calls"] == 0 + assert state["last_backend"] == "cheap-real" + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Threshold flows through from config +# --------------------------------------------------------------------------- +async def test_low_threshold_keeps_cheap(tmp_path): + state = {"fixed_scores": {"cheap": 0.65, "strong": 0.95}} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), threshold=0.6, cache=False)) + + await shim.post("/v1/responses", json={"model": "codex-auto", "input": "task"}) + assert state["last_backend"] == "cheap-real" # 0.65 >= 0.6, cheapest wins + + await shim.close() + await upstream.close() + + +async def test_high_threshold_escalates_to_strong(tmp_path): + state = {"fixed_scores": {"cheap": 0.65, "strong": 0.95}} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), threshold=0.9, cache=False)) + + await shim.post("/v1/responses", json={"model": "codex-auto", "input": "task"}) + assert state["last_backend"] == "strong-real" # only 0.95 clears 0.9 + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Robust to bad classifier output +# --------------------------------------------------------------------------- +async def test_garbage_classifier_output_falls_back_to_default(tmp_path): + # Classifier returns prose with no JSON -> empty scores -> fallback to default ("cheap"). + state = {"backend": None, "calls": 0} + + async def chat(request): + body = await request.json() + if body.get("model") == "classifier-real": + state["calls"] += 1 + return web.json_response({"choices": [{"message": {"content": "I cannot help with that."}}], "usage": {}}) + state["backend"] = body.get("model") + return web.json_response({"choices": [{"message": {"content": f"handled by {body.get('model')}"}}], "usage": {}}) + + app = web.Application() + app.router.add_post("/v1/chat/completions", chat) + upstream = TestClient(TestServer(app)) + await upstream.start_server() + + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), cache=False)) + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "refactor across files"}) + assert resp.status == 200 + assert state["calls"] == 1 # classifier was consulted + assert state["backend"] == "cheap-real" # but its garbage reply fell back to default + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Discovery gating +# --------------------------------------------------------------------------- +async def test_health_count_excludes_virtual_auto_model(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + + body = await (await shim.get("/health")).json() + assert body["auto_router"] is True + assert body["models"] == 3 # cheap + strong + classifier; NOT the virtual auto entry + + await shim.close() + await upstream.close() + + +async def test_disable_env_removes_auto_from_discovery(tmp_path, monkeypatch): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")))) + monkeypatch.setenv("CODEX_SHIM_DISABLE_ROUTER", "1") + + models = await (await shim.get("/v1/models")).json() + assert "codex-auto" not in [m["id"] for m in models["data"]] + health = await (await shim.get("/health")).json() + assert health["auto_router"] is False + + await shim.close() + await upstream.close() + + +async def test_unavailable_candidates_hide_auto_entry(tmp_path): + state = {} + upstream = await make_upstream(state) + # Candidates reference slugs that have no usable backend. + settings = _settings( + tmp_path, + str(upstream.make_url("/v1")), + candidates=[ + {"slug": "ghost-a", "cost": 1, "card": "x"}, + {"slug": "ghost-b", "cost": 5, "card": "y"}, + ], + ) + shim = await _shim(settings) + + models = await (await shim.get("/v1/models")).json() + assert "codex-auto" not in [m["id"] for m in models["data"]] + health = await (await shim.get("/health")).json() + assert health["auto_router"] is False + + await shim.close() + await upstream.close() + + +async def test_candidate_without_credentials_is_skipped(tmp_path): + state = {} + upstream = await make_upstream(state) + # "expensive" has no api_key -> unusable -> must never be selected even though + # the classifier would score it highest. + settings = _settings( + tmp_path, + str(upstream.make_url("/v1")), + extra_models=[{"slug": "expensive", "model": "expensive-real", "display_name": "Expensive", "provider": "openai", "base_url": str(upstream.make_url("/v1"))}], + candidates=[ + {"slug": "cheap", "cost": 1, "card": "cheap"}, + {"slug": "expensive", "cost": 99, "card": "best of all"}, + ], + ) + state["score_fn"] = lambda _t: {"cheap": 0.8, "expensive": 0.99} + shim = await _shim(settings) + + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": "task"}) + assert resp.status == 200 + assert state["last_backend"] == "cheap-real" # expensive dropped (no key) + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Picker switch accepts the virtual model +# --------------------------------------------------------------------------- +async def test_switch_model_accepts_auto_slug(tmp_path, monkeypatch): + captured = {} + monkeypatch.setattr( + server_module, + "_set_active_model", + lambda slug, display=None: captured.update({"slug": slug, "display": display}), + ) + state = {} + upstream = await make_upstream(state) + server = ShimServer(_settings(tmp_path, str(upstream.make_url("/v1")))) + shim = TestClient(TestServer(server.app())) + await shim.start_server() + + resp = await shim.post( + "/api/switch", + json={"slug": "codex-auto", "restart_codex": False}, + headers={PICKER_TOKEN_HEADER: server.picker_token}, + ) + assert resp.status == 200 + data = await resp.json() + assert data["ok"] is True and data["model"] == "codex-auto" + assert captured["slug"] == "codex-auto" + assert captured["display"] == "Auto (smart routing)" + + await shim.close() + await upstream.close() + + +# --------------------------------------------------------------------------- +# Score parsing robustness (unit) — prose, code fences, partial scores +# --------------------------------------------------------------------------- +def test_parse_scores_handles_code_fence(): + text = "```json\n{\"scores\": {\"cheap\": 0.8, \"strong\": 0.9}}\n```" + assert router.parse_scores(text, ["cheap", "strong"]) == {"cheap": 0.8, "strong": 0.9} + + +def test_parse_scores_handles_prose_around_json(): + text = 'Here are my scores: {"scores": {"cheap": 0.3, "strong": 0.95}} — hope that helps!' + assert router.parse_scores(text, ["cheap", "strong"]) == {"cheap": 0.3, "strong": 0.95} + + +def test_parse_scores_defaults_missing_candidate_to_zero(): + text = '{"scores": {"cheap": 0.9}}' + assert router.parse_scores(text, ["cheap", "strong"]) == {"cheap": 0.9, "strong": 0.0} + + +# --------------------------------------------------------------------------- +# Concurrency: many simultaneous requests route correctly (shared cache safety) +# --------------------------------------------------------------------------- +async def test_concurrent_requests_route_correctly(tmp_path): + state = {} + upstream = await make_upstream(state) + shim = await _shim(_settings(tmp_path, str(upstream.make_url("/v1")), cache=True)) + + tasks = [("add a docstring", "cheap-real"), ("refactor across the whole codebase", "strong-real")] * 8 + + async def fire(prompt): + resp = await shim.post("/v1/responses", json={"model": "codex-auto", "input": prompt}) + payload = await resp.json() + return resp.status, payload["output"][0]["content"][0]["text"] + + results = await asyncio.gather(*(fire(p) for p, _ in tasks)) + + # Each response carries its own routed backend, independent of shared state. + for (_prompt, expected), (status, text) in zip(tasks, results): + assert status == 200 + assert text == f"handled by {expected}" + + await shim.close() + await upstream.close() diff --git a/tests/test_server.py b/tests/test_server.py index 01b4c5a7..695155a8 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -6,7 +6,18 @@ from aiohttp import web from aiohttp.test_utils import TestClient, TestServer -from codex_shim.server import ShimServer, _rewrite_response_model, _sanitize_chatgpt_passthrough_body +from codex_shim import server as server_module +from codex_shim.server import ( + PICKER_TOKEN_HEADER, + ResponsesStreamState, + ShimServer, + _current_managed_model, + _picker_html, + _rewrite_response_model, + _sanitize_chatgpt_passthrough_body, + _set_active_model, +) +from codex_shim.settings import FALLBACK_CHATGPT_PASSTHROUGH_SLUGS from codex_shim.translate import SHIM_ENCRYPTED_CONTENT_PREFIX @@ -179,7 +190,7 @@ async def chat(request): { "id": "chatcmpl_fake", "choices": [{"message": {"role": "assistant", "content": "hello"}}], - "usage": {"total_tokens": 3}, + "usage": {"prompt_tokens": 2, "completion_tokens": 1, "total_tokens": 3}, } ) @@ -211,6 +222,7 @@ async def chat(request): assert resp.status == 200 payload = await resp.json() assert payload["output"][0]["content"][0]["text"] == "hello" + assert payload["usage"] == {"input_tokens": 2, "output_tokens": 1, "total_tokens": 3} assert captured["body"]["model"] == "real-openai" assert captured["headers"]["Authorization"] == "Bearer secret" @@ -218,7 +230,266 @@ async def chat(request): await upstream_client.close() -async def test_health_and_models_include_chatgpt_passthrough_when_auth_present(tmp_path, auth_present): +async def test_missing_api_key_env_has_model_specific_error(monkeypatch, tmp_path): + monkeypatch.delenv("OPENCODE_GO_API_KEY", raising=False) + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "glm-5.1", + "displayName": "OpenCode Go GLM-5.1", + "provider": "generic-chat-completion-api", + "baseUrl": "https://opencode.ai/zen/go/v1", + "apiKeyEnv": "OPENCODE_GO_API_KEY", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post("/v1/responses", json={"model": "glm-5-1", "input": "hi"}) + + assert resp.status == 401 + text = await resp.text() + assert "OPENCODE_GO_API_KEY" in text + assert "CURSOR_API_KEY" not in text + + await shim_client.close() + + +def _sse_events(text: str) -> list[dict]: + events = [] + for block in text.split("\n\n"): + if not block.startswith("data:"): + continue + data = block.removeprefix("data:").strip() + if data and data != "[DONE]": + events.append(json.loads(data)) + return events + + +def _named_sse_events(text: str) -> list[tuple[str | None, dict]]: + events = [] + for block in text.split("\n\n"): + event_name = None + data = None + for line in block.splitlines(): + if line.startswith("event:"): + event_name = line.removeprefix("event:").strip() + elif line.startswith("data:"): + data = line.removeprefix("data:").strip() + if data and data != "[DONE]": + events.append((event_name, json.loads(data))) + return events + + +async def test_streaming_openai_chat_response_completed_includes_usage(tmp_path): + async def chat(request): + await request.json() + response = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await response.prepare(request) + await response.write(b'data: {"choices":[{"delta":{"content":"hello"}}]}\n\n') + await response.write( + b'data: {"choices":[{"delta":{}}],"usage":{"prompt_tokens":4,"completion_tokens":2,"total_tokens":6,"prompt_tokens_details":{"cached_tokens":3}}}\n\n' + ) + await response.write(b"data: [DONE]\n\n") + await response.write_eof() + return response + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "openai", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post("/v1/responses", json={"model": "real-openai", "input": "hi", "stream": True}) + assert resp.status == 200 + events = _sse_events(await resp.text()) + completed = [event for event in events if event.get("type") == "response.completed"][-1] + assert completed["response"]["usage"] == { + "input_tokens": 4, + "output_tokens": 2, + "total_tokens": 6, + "input_tokens_details": {"cached_tokens": 3}, + } + + await shim_client.close() + await upstream_client.close() + + +async def test_streaming_anthropic_response_completed_includes_usage(): + class FakeResponse: + def __init__(self): + self.chunks: list[bytes] = [] + + async def write(self, data: bytes): + self.chunks.append(data) + + downstream = FakeResponse() + state = ResponsesStreamState("claude-real") + await state.write_anthropic_delta( + downstream, + { + "type": "message_start", + "message": { + "usage": { + "input_tokens": 5, + "cache_read_input_tokens": 4, + "output_tokens": 1, + } + }, + }, + ) + await state.write_anthropic_delta( + downstream, + { + "type": "message_delta", + "delta": {"stop_reason": "end_turn"}, + "usage": {"output_tokens": 3}, + }, + ) + await state.finish(downstream) + + events = _sse_events(b"".join(downstream.chunks).decode()) + completed = [event for event in events if event.get("type") == "response.completed"][-1] + assert completed["response"]["usage"] == { + "input_tokens": 5, + "output_tokens": 3, + "total_tokens": 8, + "input_tokens_details": { + "cached_tokens": 4, + "cache_read_input_tokens": 4, + }, + } + + +async def test_responses_compact_routes_to_openai_chat_and_returns_compacted_window(tmp_path): + captured = {} + + async def chat(request): + captured["body"] = await request.json() + return web.json_response( + { + "id": "chatcmpl_compact", + "choices": [{"message": {"role": "assistant", "content": "Task: keep implementing compact support."}}], + "usage": {"prompt_tokens": 9, "completion_tokens": 2, "total_tokens": 11}, + } + ) + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "openai", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/responses/compact", + json={ + "model": "real-openai", + "input": [ + {"role": "user", "content": "implement compact"}, + {"type": "function_call_output", "call_id": "call_1", "output": "tests pass"}, + ], + "service_tier": "priority", + "stream": True, + }, + ) + assert resp.status == 200 + payload = await resp.json() + assert payload["status"] == "completed" + assert payload["model"] == "real-openai" + assert payload["output"][0]["content"][0]["text"] == "Task: keep implementing compact support." + assert payload["usage"] == {"input_tokens": 9, "output_tokens": 2, "total_tokens": 11} + assert captured["body"]["model"] == "real-openai" + assert captured["body"]["stream"] is False + assert "service_tier" not in captured["body"] + assert "Compact the conversation" in captured["body"]["messages"][0]["content"] + + await shim_client.close() + await upstream_client.close() + + +async def test_responses_compact_chatgpt_passthrough_uses_compact_endpoint(monkeypatch, tmp_path, auth_present): + captured = {} + + class FakeUpstream: + status = 200 + content_type = "application/json" + + async def json(self, content_type=None): + return {"id": "resp_compact", "model": "gpt-5.5", "output": [{"type": "message", "model": "gpt-5.5"}]} + + def release(self): + pass + + async def fake_post(self, url, json=None, headers=None): + captured["url"] = url + captured["body"] = json + captured["headers"] = headers + return FakeUpstream() + + monkeypatch.setattr("codex_shim.server.ClientSession.post", fake_post) + settings = tmp_path / "settings.json" + settings.write_text(json.dumps({"customModels": []})) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post("/v1/responses/compact", json={"model": "openai-gpt-5-5-codex-max", "input": "hi", "stream": True}) + assert resp.status == 200 + payload = await resp.json() + assert payload["model"] == "openai-gpt-5-5-codex-max" + assert payload["output"][0]["model"] == "openai-gpt-5-5-codex-max" + assert captured["url"] == "https://chatgpt.com/backend-api/codex/responses/compact" + assert captured["body"]["model"] == "gpt-5.5" + assert "stream" not in captured["body"] + assert captured["headers"]["Accept"] == "application/json" + + await shim_client.close() + + +async def test_health_and_models_include_chatgpt_passthrough_when_auth_present(tmp_path, auth_present, monkeypatch): + missing_cache = tmp_path / "missing-models-cache.json" + monkeypatch.setattr("codex_shim.settings.DEFAULT_CODEX_MODELS_CACHE", missing_cache) settings = tmp_path / "settings.json" settings.write_text(json.dumps({"customModels": []})) shim_client = TestClient(TestServer(ShimServer(settings).app())) @@ -227,13 +498,13 @@ async def test_health_and_models_include_chatgpt_passthrough_when_auth_present(t health = await shim_client.get("/health") assert health.status == 200 body = await health.json() - assert body["models"] == 1 + assert body["models"] == len(FALLBACK_CHATGPT_PASSTHROUGH_SLUGS) assert body["chatgpt_passthrough"] is True models = await shim_client.get("/v1/models") assert models.status == 200 payload = await models.json() - assert [model["id"] for model in payload["data"]] == ["gpt-5.5"] + assert sorted(model["id"] for model in payload["data"]) == sorted(FALLBACK_CHATGPT_PASSTHROUGH_SLUGS) await shim_client.close() @@ -256,6 +527,64 @@ async def test_health_and_models_hide_chatgpt_passthrough_when_auth_missing(tmp_ await shim_client.close() +@pytest.fixture +def cursor_present(monkeypatch): + def _on(**_kwargs): + return True + + for target in ( + "codex_shim.cursor_passthrough.cursor_passthrough_available", + "codex_shim.server.cursor_passthrough_available", + "codex_shim.catalog.cursor_passthrough_available", + "codex_shim.cli.cursor_passthrough_available", + ): + monkeypatch.setattr(target, _on) + + +@pytest.fixture +def cursor_missing(monkeypatch): + monkeypatch.setattr("codex_shim.cursor_passthrough.cursor_passthrough_available", lambda **_: False) + monkeypatch.setattr("codex_shim.server.cursor_passthrough_available", lambda **_: False) + monkeypatch.setattr("codex_shim.catalog.cursor_passthrough_available", lambda **_: False) + + +async def test_health_and_models_include_cursor_passthrough_when_auth_present(tmp_path, cursor_present, auth_missing): + settings = tmp_path / "settings.json" + settings.write_text(json.dumps({"customModels": []})) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + health = await shim_client.get("/health") + assert health.status == 200 + body = await health.json() + assert body["models"] == 1 + assert body["cursor_passthrough"] is True + + models = await shim_client.get("/v1/models") + payload = await models.json() + assert [model["id"] for model in payload["data"]] == ["composer-2-5"] + + await shim_client.close() + + +async def test_health_and_models_hide_cursor_passthrough_when_auth_missing(tmp_path, cursor_missing, auth_missing): + settings = tmp_path / "settings.json" + settings.write_text(json.dumps({"customModels": []})) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + health = await shim_client.get("/health") + body = await health.json() + assert body["models"] == 0 + assert body["cursor_passthrough"] is False + + models = await shim_client.get("/v1/models") + payload = await models.json() + assert payload["data"] == [] + + await shim_client.close() + + async def test_chat_routes_to_openai_normalizes_developer_role(tmp_path): captured = {} @@ -278,6 +607,7 @@ async def chat(request): "displayName": "DeepSeek Reasoner", "provider": "openai", "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", } ] } @@ -335,7 +665,676 @@ async def messages(request): assert payload["choices"][0]["message"]["content"] == "anthropic hello" assert captured["body"]["model"] == "claude-real" assert captured["headers"]["x-api-key"] == "secret" + assert "Authorization" not in captured["headers"] + + await shim_client.close() + await upstream_client.close() + + +async def test_anthropic_messages_routes_to_openai_chat(tmp_path): + captured = {} + + async def chat(request): + captured["headers"] = dict(request.headers) + captured["body"] = await request.json() + return web.json_response( + { + "id": "chatcmpl_fake", + "choices": [{"finish_reason": "stop", "message": {"role": "assistant", "content": "openai hello"}}], + "usage": {"prompt_tokens": 2, "completion_tokens": 1, "total_tokens": 3}, + } + ) + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "generic-chat-completion-api", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={ + "model": "real-openai", + "system": "System", + "messages": [{"role": "user", "content": [{"type": "text", "text": "hi"}]}], + "max_tokens": 42, + }, + ) + assert resp.status == 200 + payload = await resp.json() + assert payload["type"] == "message" + assert payload["model"] == "real-openai" + assert payload["content"] == [{"type": "text", "text": "openai hello"}] + assert payload["usage"] == {"input_tokens": 2, "output_tokens": 1} + assert captured["headers"]["Authorization"] == "Bearer secret" + assert captured["body"]["model"] == "real-openai" + assert captured["body"]["max_tokens"] == 42 + assert captured["body"]["messages"] == [{"role": "system", "content": "System"}, {"role": "user", "content": "hi"}] await shim_client.close() await upstream_client.close() + +async def test_anthropic_messages_passes_through_anthropic_upstream(tmp_path): + captured = {} + + async def messages(request): + captured["headers"] = dict(request.headers) + captured["body"] = await request.json() + return web.json_response( + { + "id": "msg_fake", + "type": "message", + "role": "assistant", + "model": "claude-upstream", + "content": [{"type": "text", "text": "anthropic hello"}], + "stop_reason": "end_turn", + "usage": {"input_tokens": 2, "output_tokens": 1}, + } + ) + + upstream = web.Application() + upstream.router.add_post("/v1/messages", messages) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "claude-upstream", + "displayName": "Claude Upstream", + "provider": "anthropic", + "baseUrl": str(upstream_client.make_url("")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "claude-upstream", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42}, + ) + assert resp.status == 200 + payload = await resp.json() + assert payload["model"] == "claude-upstream" + assert payload["content"][0]["text"] == "anthropic hello" + assert captured["body"]["model"] == "claude-upstream" + assert captured["headers"]["x-api-key"] == "secret" + assert "Authorization" not in captured["headers"] + + await shim_client.close() + await upstream_client.close() + + +async def test_anthropic_messages_streams_openai_chat_as_anthropic_sse(tmp_path): + captured = {} + + async def chat(request): + captured["body"] = await request.json() + response = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await response.prepare(request) + await response.write(b'data: {"choices":[{"delta":{"content":"hello"}}]}\n\n') + await response.write( + b'data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":4,"completion_tokens":2,"total_tokens":6}}\n\n' + ) + await response.write(b"data: [DONE]\n\n") + await response.write_eof() + return response + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "generic-chat-completion-api", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "real-openai", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42, "stream": True}, + ) + assert resp.status == 200 + text = await resp.text() + assert "[DONE]" not in text + events = _named_sse_events(text) + assert [event for event, _ in events] == [ + "message_start", + "content_block_start", + "content_block_delta", + "content_block_stop", + "message_delta", + "message_stop", + ] + assert events[2][1]["delta"] == {"type": "text_delta", "text": "hello"} + assert events[4][1]["delta"]["stop_reason"] == "end_turn" + assert events[4][1]["usage"] == {"input_tokens": 4, "output_tokens": 2} + assert captured["body"]["stream_options"] == {"include_usage": True} + + await shim_client.close() + await upstream_client.close() + + + +async def test_anthropic_messages_streams_tool_calls_as_anthropic_sse(tmp_path): + async def chat(request): + response = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await response.prepare(request) + await response.write( + b'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_1","function":{"name":"lookup","arguments":""}}]}}]}\n\n' + ) + await response.write( + b'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\\"q\\":\\"repo\\"}"}}]}}]}\n\n' + ) + await response.write( + b'data: {"choices":[{"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":5,"completion_tokens":3,"total_tokens":8}}\n\n' + ) + await response.write(b"data: [DONE]\n\n") + await response.write_eof() + return response + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "generic-chat-completion-api", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "real-openai", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42, "stream": True}, + ) + assert resp.status == 200 + text = await resp.text() + events = _named_sse_events(text) + event_names = [event for event, _ in events] + assert "message_start" in event_names + assert "content_block_start" in event_names + tool_start = next(payload for name, payload in events if name == "content_block_start" and payload.get("content_block", {}).get("type") == "tool_use") + assert tool_start["content_block"]["id"] == "call_1" + assert tool_start["content_block"]["name"] == "lookup" + tool_deltas = [payload for name, payload in events if name == "content_block_delta" and payload.get("delta", {}).get("type") == "input_json_delta"] + assert len(tool_deltas) >= 1 + message_delta = next(payload for name, payload in events if name == "message_delta") + assert message_delta["delta"]["stop_reason"] == "tool_use" + assert "message_stop" in event_names + + await shim_client.close() + await upstream_client.close() + + +async def test_anthropic_messages_streams_reasoning_as_anthropic_sse(tmp_path): + async def chat(request): + response = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await response.prepare(request) + await response.write( + b'data: {"choices":[{"delta":{"reasoning_content":"let me think"}}]}\n\n' + ) + await response.write( + b'data: {"choices":[{"delta":{"content":"the answer"}}]}\n\n' + ) + await response.write( + b'data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":1,"total_tokens":3}}\n\n' + ) + await response.write(b"data: [DONE]\n\n") + await response.write_eof() + return response + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "generic-chat-completion-api", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "real-openai", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42, "stream": True}, + ) + assert resp.status == 200 + events = _named_sse_events(await resp.text()) + thinking_starts = [p for n, p in events if n == "content_block_start" and p.get("content_block", {}).get("type") == "thinking"] + assert len(thinking_starts) == 1 + thinking_deltas = [p for n, p in events if n == "content_block_delta" and p.get("delta", {}).get("type") == "thinking_delta"] + assert len(thinking_deltas) == 1 + assert thinking_deltas[0]["delta"]["thinking"] == "let me think" + text_deltas = [p for n, p in events if n == "content_block_delta" and p.get("delta", {}).get("type") == "text_delta"] + assert len(text_deltas) == 1 + assert text_deltas[0]["delta"]["text"] == "the answer" + + await shim_client.close() + await upstream_client.close() + + +async def test_anthropic_messages_returns_anthropic_error_for_upstream_failure(tmp_path): + async def chat(request): + return web.json_response( + {"error": {"message": "invalid api key", "type": "invalid_request_error"}}, + status=401, + ) + + upstream = web.Application() + upstream.router.add_post("/v1/chat/completions", chat) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "real-openai", + "displayName": "Real OpenAI", + "provider": "generic-chat-completion-api", + "baseUrl": str(upstream_client.make_url("/v1")), + "apiKey": "bad-key", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "real-openai", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42}, + ) + assert resp.status == 401 + payload = await resp.json() + assert payload["type"] == "error" + assert payload["error"]["type"] == "authentication_error" + assert "invalid api key" in payload["error"]["message"] + + await shim_client.close() + await upstream_client.close() + + +async def test_anthropic_messages_streams_anthropic_passthrough(tmp_path): + async def messages(request): + response = web.StreamResponse(headers={"Content-Type": "text/event-stream"}) + await response.prepare(request) + await response.write(b'event: message_start\ndata: {"type":"message_start","message":{"id":"msg_1","type":"message","role":"assistant","model":"claude-upstream","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":1,"output_tokens":0}}}\n\n') + await response.write(b'event: content_block_start\ndata: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}\n\n') + await response.write(b'event: content_block_delta\ndata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hello"}}\n\n') + await response.write(b'event: content_block_stop\ndata: {"type":"content_block_stop","index":0}\n\n') + await response.write(b'event: message_delta\ndata: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":1}}\n\n') + await response.write(b'event: message_stop\ndata: {"type":"message_stop"}\n\n') + await response.write_eof() + return response + + upstream = web.Application() + upstream.router.add_post("/v1/messages", messages) + upstream_client = TestClient(TestServer(upstream)) + await upstream_client.start_server() + + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "claude-upstream", + "displayName": "Claude Upstream", + "provider": "anthropic", + "baseUrl": str(upstream_client.make_url("")), + "apiKey": "secret", + } + ] + } + ) + ) + shim_client = TestClient(TestServer(ShimServer(settings).app())) + await shim_client.start_server() + + resp = await shim_client.post( + "/v1/messages", + json={"model": "claude-upstream", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 42, "stream": True}, + ) + assert resp.status == 200 + text = await resp.text() + events = _named_sse_events(text) + event_names = [event for event, _ in events] + assert event_names == [ + "message_start", + "content_block_start", + "content_block_delta", + "content_block_stop", + "message_delta", + "message_stop", + ] + text_delta = next(payload for name, payload in events if name == "content_block_delta") + assert text_delta["delta"]["text"] == "hello" + + await shim_client.close() + await upstream_client.close() + +def _picker_settings_file(tmp_path): + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "customModels": [ + { + "model": "kimi-k26", + "displayName": "Kimi K2.6", + "provider": "openai", + "baseUrl": "http://example.invalid/v1", + "apiKey": "k", + }, + { + "model": "deepseek-v4-pro", + "displayName": "DeepSeek V4 Pro", + "provider": "openai", + "baseUrl": "http://example.invalid/v1", + "apiKey": "k", + }, + ] + } + ) + ) + return settings + + +def _stub_codex_config(monkeypatch, tmp_path, *, model: str = "kimi-k26") -> "Path": + config = tmp_path / "config.toml" + config.write_text( + f'model = "{model}"\n' + 'model_provider = "codex_shim"\n' + '\n' + '[model_providers.codex_shim]\n' + 'name = "Codex Shim"\n' + 'base_url = "http://127.0.0.1:8765/v1"\n' + 'wire_api = "responses"\n' + ) + monkeypatch.setattr(server_module, "CODEX_CONFIG_PATH", config) + return config + + +def _picker_headers(shim: ShimServer) -> dict[str, str]: + return {PICKER_TOKEN_HEADER: shim.picker_token} + + +def test_picker_html_renders_self_contained_page(): + html = _picker_html("test-token") + assert html.startswith("") + assert "/api/models" in html + assert "/api/switch" in html + assert PICKER_TOKEN_HEADER in html + assert 'const PICKER_TOKEN = "test-token";' in html + + +def test_picker_html_json_escapes_token(): + token = 'tok"\'' + html = _picker_html(token) + assert 'const PICKER_TOKEN = "tok\\"\'\\u003c/script>";' in html + assert "