Skip to content

fix(local): wire OpenAI-compatible provider into local backend - #3674

Merged
cpacker merged 1 commit into
mainfrom
cameron/local-openai-compatible-provider
Aug 4, 2026
Merged

fix(local): wire OpenAI-compatible provider into local backend#3674
cpacker merged 1 commit into
mainfrom
cameron/local-openai-compatible-provider

Conversation

@just-cameron

@just-cameron just-cameron commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a first-class local openai-compatible provider to the CLI, TUI, and provider selector
  • discover arbitrary model IDs from the configured /v1/models endpoint and use the same published model for turns
  • support keyed and keyless endpoints without sending the internal no-key sentinel as authorization

Before and after

Before, letta --backend local connect openai-compatible returned Unknown provider, so users had to configure arbitrary endpoints as LM Studio or llama.cpp.

After, the local provider has its own identity and stable openai-compatible/<model-id> handles. The base URL is required. The API key is optional.

Scope and risk

  • local backend only; the existing cloud provider flow is unchanged
  • model capabilities remain text-only and non-reasoning because the standard model-list response does not report them
  • the provider keeps the last successful model list when a refresh fails
  • unconfigured custom endpoints are not probed
  • this adds one configured OpenAI-compatible connection, not multiple named profiles

Validation

  • 93 focused provider, CLI, storage, selector, discovery, and turn tests
  • command-level smoke using the source CLI and a mock OpenAI-compatible server
  • keyed and keyless discovery and Chat Completions requests
  • bun run check
  • listener tests that failed in the full parallel unit run passed when rerun in isolation
  • full integration suite could not create its required test agents in the current test environment; CI will run it independently

Overlord (agent-c2adbf5c-8419-4211-8cd8-3740db164974)

👾 Generated with Letta Code

Letta Code (agent-c2adbf5c-8419-4211-8cd8-3740db164974)

Add a dedicated local provider so arbitrary Chat Completions endpoints can be configured and discovered without using an engine-specific alias.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta Code <noreply@letta.com>
@just-cameron
just-cameron marked this pull request as ready for review August 4, 2026 19:37
@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@just-cameron just-cameron changed the title feat(local): add OpenAI-compatible provider fix(local): wire OpenAI-compatible provider into local backend Aug 4, 2026
@cpacker
cpacker merged commit 9f81fad into main Aug 4, 2026
28 of 29 checks passed
@cpacker
cpacker deleted the cameron/local-openai-compatible-provider branch August 4, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants