feat: support Anthropic Messages vision requests

.env.example

@@ -1,7 +1,7 @@
 # Only the vision API is configured here. DeepSeek auth is still sent by Codex and passed through by the proxy.
 VISION_API_KEY=
-# Any OpenAI-compatible endpoint that supports /chat/completions with image_url works;
-# the Python client/proxy can also use /responses with input_image (see below).
+# Use an endpoint that supports OpenAI Chat Completions, OpenAI Responses, or
+# Anthropic Messages; select the protocol below.
 # Recommended options (see README):
 #   OpenRouter:        https://openrouter.ai/api/v1
 #   Aliyun DashScope:  https://dashscope.aliyuncs.com/compatible-mode/v1
@@ -10,11 +10,15 @@ VISION_BASE_URL=https://openrouter.ai/api/v1
 VISION_MODEL=google/gemini-3.6-flash
 # Vision model output language: zh=Chinese, en=English (defaults to Chinese when unset)
 LANG=zh
-# Python client/proxy protocol: chat_completions (default) or responses. Choose
-# "responses" for models exposed only through the Responses API.
+# Python client/proxy protocol: chat_completions (default), responses, or
+# anthropic. For anthropic, use a base URL ending in /v1, not /messages.
 # VISION_API_PROTOCOL=chat_completions
 # Reasoning effort sent with the responses protocol (optional).
 # VISION_REASONING_EFFORT=medium
+# Anthropic thinking mode. omit (default) sends no thinking field and has the
+# broadest compatibility. Use disabled or adaptive only when the selected model
+# documents that mode; restore omit first if the provider returns HTTP 400.
+# VISION_ANTHROPIC_THINKING=omit
 # Optional outbound User-Agent override. The default is browser-compatible to avoid
 # gateways that block Python-urllib clients.
 # VISION_USER_AGENT=custom-vision-client/1.0

AGENT_INSTALL.md

@@ -27,7 +27,7 @@
 
 - 已接入纯文本模型并能正常对话的宿主（Codex 或 Claude Code）
 - Python 3.11+
-- 一个支持 `/chat/completions` 和 `image_url` 的 OpenAI-compatible 视觉 API；Python 客户端/代理也可配置 `/responses` + `input_image`
+- 一个支持 OpenAI Chat Completions、OpenAI Responses 或 Anthropic Messages 的视觉 API；通过 `VISION_API_PROTOCOL` 选择协议
 
 ## 1. 定位并备份现有配置
 
@@ -58,13 +58,16 @@ VISION_API_KEY=...
 VISION_BASE_URL=...
 VISION_MODEL=...
 LANG=zh  # 可选：视觉模型输出语言（zh/en），不填保持默认中文
-# VISION_API_PROTOCOL=chat_completions  # 可选：Python 客户端/代理可改用 responses 协议
+# VISION_API_PROTOCOL=chat_completions  # 可选：chat_completions / responses / anthropic
 # VISION_REASONING_EFFORT=medium        # 可选：responses 协议下的推理强度
+# VISION_ANTHROPIC_THINKING=omit        # 可选：omit 兼容性最好；仅在模型明确支持时使用 disabled / adaptive
 # VISION_USER_AGENT=custom-vision-client/1.0  # 可选：覆盖默认的浏览器兼容 User-Agent
 ```
 
 不要在 env 中写入上游模型的 key（如 `DEEPSEEK_API_KEY`）。上游鉴权仍由宿主发送。
 
+`VISION_ANTHROPIC_THINKING=omit` 不发送 thinking 字段，并保留模型默认行为。`disabled` 与 `adaptive` 具有模型兼容性限制；如果提供方返回 HTTP 400，先恢复 `omit`。当前不提供手动 `enabled` + `budget_tokens`。
+
 - macOS / Linux：执行 `chmod 600 <ENV_FILE>`。
 - Windows：把 env 保留在当前用户的 `%LOCALAPPDATA%` 下，不复制到公共目录。
 

CHANGELOG.md

@@ -7,11 +7,13 @@ All notable user-facing changes to agent-vision-toolkit are documented in this f
 ### Added
 
 - Let the shared Python vision client call either Chat Completions or Responses APIs, including optional reasoning effort and explicit `store: false` data handling.
+- Add native Anthropic Messages requests with protocol-specific authentication, image sources, optional thinking control, and text-block response extraction.
 - Rewrite OpenAI Chat Completions `image_url` blocks through the existing vision-description pipeline with a host-neutral channel note.
 
 ### Fixed
 
 - Send a browser-compatible, configurable User-Agent from the shared Python vision client so Cloudflare-backed OpenAI-compatible endpoints do not reject the default `Python-urllib` signature.
+- Honor `Retry-After` and retry Anthropic 529 overload responses.
 
 ## [0.1.0] - 2026-08-07
 

README.md

@@ -137,7 +137,7 @@ When to use them, the order in which to call tools, and how to verify the result
 
 > Read https://github.com/Anionex/agent-vision-toolkit/blob/main/AGENT_INSTALL.md in full, then install the appropriate vision proxy or native extension/plugin for the agent application we are currently using. If the vision API is not configured, locate the configuration file for the current operating system and guide me through setting `VISION_API_KEY`, `VISION_BASE_URL`, and `VISION_MODEL`.
 
-All you need to prepare is an OpenAI-compatible multimodal API base URL, API key, and model name. The agent will guide you through writing them to the appropriate configuration file.
+All you need to prepare is a multimodal API supporting OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages, plus its base URL, API key, and model name. The agent will guide you through writing them to the appropriate configuration file.
 
 > After installing the optional integration and restarting the agent, paste an image directly or let the model call its built-in image tool. Pi, Oh My Pi, and OpenCode use single-file [native extensions](extensions/) rather than the proxy; see each agent's documentation.
 
@@ -152,7 +152,7 @@ VISION_BASE_URL=https://openrouter.ai/api/v1
 VISION_MODEL=google/gemini-3.6-flash
 ```
 
-Any OpenAI-compatible endpoint that supports `/chat/completions` with `image_url` works (e.g. Aliyun DashScope: `https://dashscope.aliyuncs.com/compatible-mode/v1` + `qwen-vl-max-latest`). The Python client/proxy can also use `/responses` with `input_image` by setting `VISION_API_PROTOCOL=responses`. Add `LANG=en` for English descriptions (default is Chinese).
+Any OpenAI-compatible endpoint that supports `/chat/completions` with `image_url` works (e.g. Aliyun DashScope: `https://dashscope.aliyuncs.com/compatible-mode/v1` + `qwen-vl-max-latest`). The Python client/proxy can also use `/responses` with `input_image` by setting `VISION_API_PROTOCOL=responses`, or Anthropic Messages by setting `VISION_API_PROTOCOL=anthropic` and a base URL ending in `/v1` (not `/messages`). Add `LANG=en` for English descriptions (default is Chinese).
 
 **2. Put the CLIs on your PATH:**
 
@@ -328,11 +328,12 @@ The standalone CLIs and Python proxy use these environment variables; just three
 | Variable | Required | Description |
 |---|---:|---|
 | `VISION_API_KEY` | Yes | API key of the multimodal model |
-| `VISION_BASE_URL` | Yes | OpenAI-compatible API base URL |
+| `VISION_BASE_URL` | Yes | Provider API base URL; include `/v1` but not the protocol endpoint such as `/messages` |
 | `VISION_MODEL` | Yes | Multimodal model name |
 | `LANG` | No | Vision model output language: `zh` (Chinese) or `en` (English); default `zh` |
-| `VISION_API_PROTOCOL` | No | Python client/proxy protocol: `chat_completions` (default) or `responses` |
+| `VISION_API_PROTOCOL` | No | Python client/proxy protocol: `chat_completions` (default), `responses`, or `anthropic`; Anthropic mode uses `x-api-key` and `anthropic-version` |
 | `VISION_REASONING_EFFORT` | No | Optional provider-supported reasoning effort for the Python client/proxy when using `responses` |
+| `VISION_ANTHROPIC_THINKING` | No | Anthropic thinking mode. `omit` (default) sends no thinking field and has the broadest compatibility. Use `disabled` or `adaptive` only when the selected model documents that mode; restore `omit` first if the provider returns HTTP 400. Manual `enabled` plus `budget_tokens` is not exposed. |
 | `VISION_USER_AGENT` | No | Outbound User-Agent for the Python client/proxy; defaults to a browser-compatible value and can be overridden for provider requirements |
 
 </details>
@@ -352,7 +353,7 @@ The route whose connection (TCP/TLS handshake) succeeds is kept in memory and re
 ## Prerequisites
 
 - A coding agent already working with a model, including a text-only model such as DeepSeek V4
-- An OpenAI-compatible vision API that supports `/chat/completions` and `image_url`; the Python client/proxy can also use `/responses` with `input_image` via `VISION_API_PROTOCOL=responses`
+- A vision API supporting OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages; select the latter two with `VISION_API_PROTOCOL=responses` or `VISION_API_PROTOCOL=anthropic`
 - No other configuration is required
 
 ## FAQ

README_CN.md

@@ -137,7 +137,7 @@
 
 > 完整阅读 https://github.com/Anionex/agent-vision-toolkit/blob/main/AGENT_INSTALL.md，根据我们当前使用的 agent 应用，安装适用的视觉代理或原生 extension/plugin。如果视觉 API 尚未配置，请按当前系统找到配置文件，并引导我填写 `VISION_API_KEY`、`VISION_BASE_URL` 和 `VISION_MODEL`。
 
-唯一要准备的是 OpenAI-compatible 多模态模型的 API base URL、API key 和模型名称；agent 会引导你把它们写入对应的配置文件。
+只需准备一个支持 OpenAI Chat Completions、OpenAI Responses 或 Anthropic Messages 的多模态 API，以及它的 base URL、API key 和模型名称；agent 会引导你把它们写入对应的配置文件。
 
 > 对于可选接入层，安装完成并重启后，直接粘贴图片或让模型调用内置看图工具即可。Pi、Oh My Pi、OpenCode 走的是单文件[原生 extension](extensions/) 而不是代理，可见各 agent 的文档。
 
@@ -152,7 +152,7 @@ VISION_BASE_URL=https://openrouter.ai/api/v1
 VISION_MODEL=google/gemini-3.6-flash
 ```
 
-任何支持 `/chat/completions` 与 `image_url` 的 OpenAI-compatible 端点都可以（如阿里云百炼：`https://dashscope.aliyuncs.com/compatible-mode/v1` + `qwen-vl-max-latest`）。Python 客户端/代理也可设置 `VISION_API_PROTOCOL=responses` 使用 `/responses` + `input_image`。需要英文描述时加 `LANG=en`（默认中文）。
+任何支持 `/chat/completions` 与 `image_url` 的 OpenAI-compatible 端点都可以（如阿里云百炼：`https://dashscope.aliyuncs.com/compatible-mode/v1` + `qwen-vl-max-latest`）。Python 客户端/代理也可设置 `VISION_API_PROTOCOL=responses` 使用 `/responses` + `input_image`，或设置 `VISION_API_PROTOCOL=anthropic` 使用 Anthropic Messages；此时 Base URL 应以 `/v1` 结尾，不要包含 `/messages`。需要英文描述时加 `LANG=en`（默认中文）。
 
 **2. 把 CLI 放进 PATH：**
 
@@ -325,11 +325,12 @@ Codex -> 127.0.0.1:19100 -> 用户原有的纯文本模型上游
 | 变量 | 必需 | 说明 |
 |---|---:|---|
 | `VISION_API_KEY` | 是 | 多模态模型的 API key |
-| `VISION_BASE_URL` | 是 | OpenAI-compatible API 地址 |
+| `VISION_BASE_URL` | 是 | 服务商 API Base URL；可包含 `/v1`，但不要包含 `/messages` 等协议端点 |
 | `VISION_MODEL` | 是 | 多模态模型名 |
 | `LANG` | 否 | 视觉模型输出语言：`zh`=中文，`en`=English（默认 `zh`） |
-| `VISION_API_PROTOCOL` | 否 | Python 客户端/代理的视觉 API 协议：`chat_completions`（默认）或 `responses` |
+| `VISION_API_PROTOCOL` | 否 | Python 客户端/代理的视觉 API 协议：`chat_completions`（默认）、`responses` 或 `anthropic`；Anthropic 模式使用 `x-api-key` 与 `anthropic-version` |
 | `VISION_REASONING_EFFORT` | 否 | Python 客户端/代理使用 `responses` 时可选的服务商支持推理强度 |
+| `VISION_ANTHROPIC_THINKING` | 否 | Anthropic thinking 模式。`omit`（默认）不发送 thinking 字段，兼容性最好；仅当所选模型明确支持时使用 `disabled` 或 `adaptive`，提供方返回 HTTP 400 时应先恢复 `omit`。当前不提供手动 `enabled` + `budget_tokens`。 |
 | `VISION_USER_AGENT` | 否 | Python 客户端/代理的出站 User-Agent；默认使用浏览器兼容值，也可按服务商要求覆盖 |
 
 </details>
@@ -349,7 +350,7 @@ Codex -> 127.0.0.1:19100 -> 用户原有的纯文本模型上游
 ## 前置条件
 
 - 已接入(纯文本)模型（如 DeepSeek V4）并可正常使用的 coding agent
-- 一个支持 `/chat/completions` 与 `image_url` 的 OpenAI-compatible 视觉 API；Python 客户端/代理也可通过 `VISION_API_PROTOCOL=responses` 使用 `/responses` + `input_image`
+- 一个支持 OpenAI Chat Completions、OpenAI Responses 或 Anthropic Messages 的视觉 API；后两者分别使用 `VISION_API_PROTOCOL=responses` 与 `VISION_API_PROTOCOL=anthropic`
 - 没有其他需要的配置
 
 ## 常见问题

extensions/opencode/vision.ts

@@ -16,11 +16,12 @@
  * auto-detect vision-capable primaries; set VISION_REWRITE=off in the
  * environment to disable rewriting when running a multimodal model.
  *
- * Configuration comes from the same env chain as the agent-vision-toolkit repo
+ * Configuration comes from this extension's env chain
  * (VISION_API_KEY / VISION_BASE_URL / VISION_MODEL, optional LANG=zh|en):
  * $VISION_ENV_FILE, %LOCALAPPDATA%/agent-vision-toolkit/env,
  * ~/.config/agent-vision-toolkit/env, ./.env — later files override earlier ones
- * and the process environment, matching vision_client.py.
+ * and the process environment. Unlike the Python client, this standalone
+ * extension keeps loading later fallback files after VISION_ENV_FILE.
  *
  * A sibling implementation for Pi / Oh My Pi lives at extensions/pi/vision.ts;
  * both files deliberately duplicate the small describe core so each stays a
@@ -81,7 +82,7 @@ export interface VisionConfig {
 }
 
 // ---------------------------------------------------------------------------
-// Env-chain configuration (ported from vision_client.load_default_env).
+// Env-chain configuration for this standalone extension.
 
 function parseEnvFile(path: string, into: Record<string, string>): void {
   let raw: string;
@@ -98,7 +99,7 @@ function parseEnvFile(path: string, into: Record<string, string>): void {
     let value = line.slice(eq + 1).trim();
     value = value.replace(/^["']/, "").replace(/["']$/, "");
     // The env file is the user's explicit configuration: whatever it sets
-    // wins, even over the process environment — same as vision_client.py.
+    // wins over values already collected from the process or earlier files.
     if (key) into[key] = value;
   }
 }

extensions/pi/vision.ts

@@ -15,11 +15,12 @@
  * so rewrites never touch the stored session and an in-process cache keyed on
  * (image, prompt) makes replayed turns free.
  *
- * Configuration comes from the same env chain as the agent-vision-toolkit repo
+ * Configuration comes from this extension's env chain
  * (VISION_API_KEY / VISION_BASE_URL / VISION_MODEL, optional LANG=zh|en):
  * $VISION_ENV_FILE, %LOCALAPPDATA%/agent-vision-toolkit/env,
  * ~/.config/agent-vision-toolkit/env, ./.env — later files override earlier ones
- * and the process environment, matching vision_client.py.
+ * and the process environment. Unlike the Python client, this standalone
+ * extension keeps loading later fallback files after VISION_ENV_FILE.
  *
  * A sibling implementation for OpenCode lives at extensions/opencode/vision.ts;
  * both files deliberately duplicate the small describe core so each stays a
@@ -80,7 +81,7 @@ export interface VisionConfig {
 }
 
 // ---------------------------------------------------------------------------
-// Env-chain configuration (ported from vision_client.load_default_env).
+// Env-chain configuration for this standalone extension.
 
 function parseEnvFile(path: string, into: Record<string, string>): void {
   let raw: string;
@@ -97,7 +98,7 @@ function parseEnvFile(path: string, into: Record<string, string>): void {
     let value = line.slice(eq + 1).trim();
     value = value.replace(/^["']/, "").replace(/["']$/, "");
     // The env file is the user's explicit configuration: whatever it sets
-    // wins, even over the process environment — same as vision_client.py.
+    // wins over values already collected from the process or earlier files.
     if (key) into[key] = value;
   }
 }