From 8c326f3aa9a225c79e144ce34be5469e0cda91c8 Mon Sep 17 00:00:00 2001 From: hongyi-chen Date: Mon, 3 Aug 2026 17:18:08 +0000 Subject: [PATCH] docs: document selectable custom inference endpoint API formats Custom endpoints now declare which request format they speak: OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages. The page previously described OpenAI Chat Completions as the only supported surface. Co-Authored-By: Oz --- .../inference/custom-inference-endpoint.mdx | 34 +++++++++++++------ 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/src/content/docs/agents/inference/custom-inference-endpoint.mdx b/src/content/docs/agents/inference/custom-inference-endpoint.mdx index f3d47c47..adf4f5aa 100644 --- a/src/content/docs/agents/inference/custom-inference-endpoint.mdx +++ b/src/content/docs/agents/inference/custom-inference-endpoint.mdx @@ -1,11 +1,12 @@ --- title: Custom inference endpoint description: >- - Connect Warp's agents to any OpenAI-compatible inference endpoint — - OpenRouter, LiteLLM, z.ai, or an internal gateway exposed at a public URL. + Connect Warp's agents to an inference endpoint that speaks OpenAI Chat + Completions, OpenAI Responses, or Anthropic Messages — OpenRouter, LiteLLM, + z.ai, or an internal gateway exposed at a public URL. --- -Warp supports **custom inference endpoints** for users who want to power Warp's agents with any OpenAI-compatible inference endpoint — a model router, hosted gateway, or internal infrastructure they already run. +Warp supports **custom inference endpoints** for users who want to power Warp's agents with their own inference endpoint — a model router, hosted gateway, or internal infrastructure they already run. This lets you route AI requests through your preferred provider, run inference behind your own gateway, or use a router like OpenRouter or LiteLLM, while keeping the agent experience inside Warp. @@ -17,19 +18,19 @@ Custom inference endpoints are available on Free and all eligible paid plans for ## Key features -* **OpenAI-compatible** - Works with any endpoint that implements the OpenAI Chat Completions API. -* **Provider flexibility** - Use a model router (OpenRouter, LiteLLM), a model provider with an OpenAI-compatible surface (z.ai), or your own internal gateway exposed at a public URL. +* **Three supported API formats** - Point Warp at an endpoint that implements OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages, and pick the matching format when you add the endpoint. +* **Provider flexibility** - Use a model router (OpenRouter, LiteLLM), a model provider with a compatible surface (z.ai), or your own internal gateway exposed at a public URL. * **No AI credits consumed for inference** - Inference is billed directly by your endpoint provider. On Business and Enterprise, local agent runs that route through a custom inference endpoint still consume [platform credits](/support-and-community/plans-and-billing/platform-credits/) for Warp's platform infrastructure. * **Local API key storage** - Your endpoint API key is stored **only on your device** (in your OS keychain or equivalent secure storage), never on Warp's servers. It's used to make requests to your configured endpoint. ## How it works -A custom inference endpoint expects your endpoint to implement the **OpenAI Chat Completions API** (`POST /v1/chat/completions`). Any service that exposes a compatible surface can be used as a target: +A custom inference endpoint expects your endpoint to implement one of the request formats listed in [Supported API formats](#supported-api-formats). Any service that exposes a compatible surface can be used as a target: * **OpenRouter** - Aggregates many model providers behind a single OpenAI-compatible API and consolidated billing. * **LiteLLM** - A self-hosted proxy that exposes a unified, OpenAI-compatible API across providers. * **z.ai** - A model provider with an OpenAI-compatible API surface for its models. -* **Internal gateways (exposed at a public URL)** - An in-house service that fronts model providers behind an OpenAI-compatible endpoint (for example, a corporate AI gateway with logging, redaction, or access control). The gateway must be reachable from the public internet — an internal-only service, such as a LiteLLM proxy that only resolves inside your network or VPN, won't work until it's exposed at a public URL (see [Network requirements](#network-requirements)). +* **Internal gateways (exposed at a public URL)** - An in-house service that fronts model providers behind a compatible endpoint (for example, a corporate AI gateway with logging, redaction, or access control). The gateway must be reachable from the public internet — an internal-only service, such as a LiteLLM proxy that only resolves inside your network or VPN, won't work until it's exposed at a public URL (see [Network requirements](#network-requirements)). When you configure a custom inference endpoint, your endpoint URL, model identifiers, and API key are stored **only on your device**, never on Warp's servers. Your API key is used to make requests to your configured endpoint. @@ -55,14 +56,25 @@ When a model routed through your endpoint is selected: * Costs are billed directly by your endpoint provider. * Warp doesn't retain or store your API key on any of its servers. +## Supported API formats + +Each custom endpoint declares the request format it speaks, so Warp knows how to shape requests to it. Warp supports three: + +* **OpenAI Chat Completions** - The default. Warp calls `POST /v1/chat/completions` on your base URL. Use this for OpenRouter, LiteLLM, z.ai, and most OpenAI-compatible gateways. +* **OpenAI Responses** - OpenAI's Responses API. Use this when your endpoint exposes the Responses surface rather than Chat Completions. +* **Anthropic Messages** - Anthropic's Messages API. Use this when your endpoint fronts Anthropic-format models, such as a gateway that proxies Claude models directly. + +Choose the format that matches your endpoint. Warp doesn't translate between formats, so an endpoint configured with the wrong format returns errors on every request. + ## Enabling a custom inference endpoint To enable and configure a custom inference endpoint: 1. In Warp, open **Settings** and search for `inference endpoint` to jump to the configuration. -2. Add your endpoint URL (the base URL that exposes `/v1/chat/completions`) and any required credentials (typically an API key). -3. Specify the model identifier(s) you want to route through this endpoint. -4. Save the configuration. Once added, you'll see your custom models appear in the model picker. +2. Add your endpoint URL (the base URL that exposes the API) and any required credentials (typically an API key). +3. Choose the API format your endpoint implements: **OpenAI Chat Completions**, **OpenAI Responses**, or **Anthropic Messages**. See [Supported API formats](#supported-api-formats). +4. Specify the model identifier(s) you want to route through this endpoint. +5. Save the configuration. Once added, you'll see your custom models appear in the model picker. When you explicitly select an endpoint-routed model from the model picker, Warp routes the request through your endpoint instead of consuming Warp's AI credits. @@ -128,7 +140,7 @@ Warp offers three ways to bring your own AI infrastructure. Use this table to pi | Name | Meaning | Plans | | --- | --- | --- | | **[Bring Your Own API Key](/agents/inference/bring-your-own-api-key/)** (BYOK) | Use your own API key for OpenAI, Anthropic, or Google models. Keys are stored locally on your device. | Free and all eligible paid plans | -| **Custom inference endpoint** | Connect Warp to an OpenAI-compatible endpoint such as OpenRouter, LiteLLM, z.ai, or an internal gateway. | Free and all eligible paid plans | +| **Custom inference endpoint** | Connect Warp to an endpoint that speaks OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages, such as OpenRouter, LiteLLM, z.ai, or an internal gateway. | Free and all eligible paid plans | | **[Bring Your Own LLM](/enterprise/enterprise-features/bring-your-own-llm/)** (BYOLLM) | Enterprise-managed inference through your cloud provider (AWS Bedrock and Gemini Enterprise Agent Platform (Vertex AI) today; Azure Foundry coming soon), with Warp handling routing, orchestration, governance, and observability. | Enterprise only | Platform credits may apply for local agent runs on Business and Enterprise when using BYOK, a custom inference endpoint, or BYOLLM. See [platform credits](/support-and-community/plans-and-billing/platform-credits/).