Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 23 additions & 11 deletions src/content/docs/agents/inference/custom-inference-endpoint.mdx
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
---
title: Custom inference endpoint
description: >-
Connect Warp's agents to any OpenAI-compatible inference endpoint —
OpenRouter, LiteLLM, z.ai, or an internal gateway exposed at a public URL.
Connect Warp's agents to an inference endpoint that speaks OpenAI Chat
Completions, OpenAI Responses, or Anthropic Messages — OpenRouter, LiteLLM,
z.ai, or an internal gateway exposed at a public URL.
---

Warp supports **custom inference endpoints** for users who want to power Warp's agents with any OpenAI-compatible inference endpoint — a model router, hosted gateway, or internal infrastructure they already run.
Warp supports **custom inference endpoints** for users who want to power Warp's agents with their own inference endpoint — a model router, hosted gateway, or internal infrastructure they already run.

This lets you route AI requests through your preferred provider, run inference behind your own gateway, or use a router like OpenRouter or LiteLLM, while keeping the agent experience inside Warp.

Expand All @@ -17,19 +18,19 @@ Custom inference endpoints are available on Free and all eligible paid plans for

## Key features

* **OpenAI-compatible** - Works with any endpoint that implements the OpenAI Chat Completions API.
* **Provider flexibility** - Use a model router (OpenRouter, LiteLLM), a model provider with an OpenAI-compatible surface (z.ai), or your own internal gateway exposed at a public URL.
* **Three supported API formats** - Point Warp at an endpoint that implements OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages, and pick the matching format when you add the endpoint.
* **Provider flexibility** - Use a model router (OpenRouter, LiteLLM), a model provider with a compatible surface (z.ai), or your own internal gateway exposed at a public URL.
* **No AI credits consumed for inference** - Inference is billed directly by your endpoint provider. On Business and Enterprise, local agent runs that route through a custom inference endpoint still consume [platform credits](/support-and-community/plans-and-billing/platform-credits/) for Warp's platform infrastructure.
* **Local API key storage** - Your endpoint API key is stored **only on your device** (in your OS keychain or equivalent secure storage), never on Warp's servers. It's used to make requests to your configured endpoint.

## How it works

A custom inference endpoint expects your endpoint to implement the **OpenAI Chat Completions API** (`POST /v1/chat/completions`). Any service that exposes a compatible surface can be used as a target:
A custom inference endpoint expects your endpoint to implement one of the request formats listed in [Supported API formats](#supported-api-formats). Any service that exposes a compatible surface can be used as a target:

* **OpenRouter** - Aggregates many model providers behind a single OpenAI-compatible API and consolidated billing.
* **LiteLLM** - A self-hosted proxy that exposes a unified, OpenAI-compatible API across providers.
* **z.ai** - A model provider with an OpenAI-compatible API surface for its models.
* **Internal gateways (exposed at a public URL)** - An in-house service that fronts model providers behind an OpenAI-compatible endpoint (for example, a corporate AI gateway with logging, redaction, or access control). The gateway must be reachable from the public internet — an internal-only service, such as a LiteLLM proxy that only resolves inside your network or VPN, won't work until it's exposed at a public URL (see [Network requirements](#network-requirements)).
* **Internal gateways (exposed at a public URL)** - An in-house service that fronts model providers behind a compatible endpoint (for example, a corporate AI gateway with logging, redaction, or access control). The gateway must be reachable from the public internet — an internal-only service, such as a LiteLLM proxy that only resolves inside your network or VPN, won't work until it's exposed at a public URL (see [Network requirements](#network-requirements)).

When you configure a custom inference endpoint, your endpoint URL, model identifiers, and API key are stored **only on your device**, never on Warp's servers. Your API key is used to make requests to your configured endpoint.

Expand All @@ -55,14 +56,25 @@ When a model routed through your endpoint is selected:
* Costs are billed directly by your endpoint provider.
* Warp doesn't retain or store your API key on any of its servers.

## Supported API formats

Each custom endpoint declares the request format it speaks, so Warp knows how to shape requests to it. Warp supports three:

* **OpenAI Chat Completions** - The default. Warp calls `POST /v1/chat/completions` on your base URL. Use this for OpenRouter, LiteLLM, z.ai, and most OpenAI-compatible gateways.
* **OpenAI Responses** - OpenAI's Responses API. Use this when your endpoint exposes the Responses surface rather than Chat Completions.
* **Anthropic Messages** - Anthropic's Messages API. Use this when your endpoint fronts Anthropic-format models, such as a gateway that proxies Claude models directly.

Choose the format that matches your endpoint. Warp doesn't translate between formats, so an endpoint configured with the wrong format returns errors on every request.

## Enabling a custom inference endpoint

To enable and configure a custom inference endpoint:

1. In Warp, open **Settings** and search for `inference endpoint` to jump to the configuration.
2. Add your endpoint URL (the base URL that exposes `/v1/chat/completions`) and any required credentials (typically an API key).
3. Specify the model identifier(s) you want to route through this endpoint.
4. Save the configuration. Once added, you'll see your custom models appear in the model picker.
2. Add your endpoint URL (the base URL that exposes the API) and any required credentials (typically an API key).
3. Choose the API format your endpoint implements: **OpenAI Chat Completions**, **OpenAI Responses**, or **Anthropic Messages**. See [Supported API formats](#supported-api-formats).
4. Specify the model identifier(s) you want to route through this endpoint.
5. Save the configuration. Once added, you'll see your custom models appear in the model picker.

When you explicitly select an endpoint-routed model from the model picker, Warp routes the request through your endpoint instead of consuming Warp's AI credits.

Expand Down Expand Up @@ -128,7 +140,7 @@ Warp offers three ways to bring your own AI infrastructure. Use this table to pi
| Name | Meaning | Plans |
| --- | --- | --- |
| **[Bring Your Own API Key](/agents/inference/bring-your-own-api-key/)** (BYOK) | Use your own API key for OpenAI, Anthropic, or Google models. Keys are stored locally on your device. | Free and all eligible paid plans |
| **Custom inference endpoint** | Connect Warp to an OpenAI-compatible endpoint such as OpenRouter, LiteLLM, z.ai, or an internal gateway. | Free and all eligible paid plans |
| **Custom inference endpoint** | Connect Warp to an endpoint that speaks OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages, such as OpenRouter, LiteLLM, z.ai, or an internal gateway. | Free and all eligible paid plans |
| **[Bring Your Own LLM](/enterprise/enterprise-features/bring-your-own-llm/)** (BYOLLM) | Enterprise-managed inference through your cloud provider (AWS Bedrock and Gemini Enterprise Agent Platform (Vertex AI) today; Azure Foundry coming soon), with Warp handling routing, orchestration, governance, and observability. | Enterprise only |

Platform credits may apply for local agent runs on Business and Enterprise when using BYOK, a custom inference endpoint, or BYOLLM. See [platform credits](/support-and-community/plans-and-billing/platform-credits/).
Expand Down
Loading