Skip to content

feat: add api_format option to OpenAI/Azure targets (chat vs responses) #896

@christso

Description

@christso

Problem

The Vercel AI SDK v3 (@ai-sdk/openai@3.0.41) defaults to the OpenAI Responses API (/responses), but not all OpenAI-compatible endpoints support it (e.g. GitHub Models returns 404). Currently we use a URL heuristic (baseURL.includes('api.openai.com')) to decide between .chat() and default — this works but is fragile.

Proposal

Add an api_format field to OpenAI and Azure target configs in targets.yaml:

targets:
  - name: grader
    provider: openai
    api_format: chat          # "chat" (default) | "responses"
    base_url: https://models.github.ai/inference/v1
    api_key: ${{ GH_MODELS_TOKEN }}
    model: ${{ GH_MODELS_MODEL }}

  - name: openai-direct
    provider: openai
    api_format: responses     # opt in to Responses API for actual OpenAI
    api_key: ${{ OPENAI_API_KEY }}
    model: gpt-5-mini

Behavior

api_format AI SDK method Endpoint
chat (default) openai.chat(model) /chat/completions
responses openai(model) /responses

Precedent

  • promptfoo: openai:chat:model vs openai:responses:model
  • pi-ai: each model has an api field (openai-responses or openai-completions) in its model registry

Why default to chat

  • Universally supported by all OpenAI-compatible endpoints
  • Responses API is only fully supported by api.openai.com and Azure OpenAI
  • For eval grading (our primary use case), Chat Completions is sufficient
  • GitHub Models, local proxies, and other OpenAI-compatible APIs only support /chat/completions

Implementation

  1. Add api_format to BASE_TARGET_SCHEMA in targets.ts
  2. Pass it through OpenAIResolvedConfig and AzureResolvedConfig
  3. Use it in OpenAIProvider and AzureProvider constructors to select .chat() vs default
  4. Remove the URL heuristic (isOpenAI = config.baseURL.includes('api.openai.com'))

Related: #892, PR #893

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions