Skip to content

feat: add openai_compatible LLM profile - #16

Merged
dimknaf merged 1 commit into
mainfrom
feat/openai-compatible-profile-rework
Jun 13, 2026
Merged

feat: add openai_compatible LLM profile#16
dimknaf merged 1 commit into
mainfrom
feat/openai-compatible-profile-rework

Conversation

@dimknaf

@dimknaf dimknaf commented Jun 13, 2026

Copy link
Copy Markdown
Owner

Adds an openai_compatible profile so the internal agent can talk to any OpenAI-compatible /v1 endpoint — Ollama, LM Studio, copilot-api, a remote vLLM — without introducing another provider abstraction.

LLM_PROFILE=openai_compatible
AGENT_MODEL=openai/llama3.2:3b
OPENAI_BASE_URL=http://host.docker.internal:11434/v1
OPENAI_API_KEY=                 # only if the endpoint requires auth

Why this shape

  • Reuses the existing mechanism. It's one entry in _LLM_PROFILES (model-prefix + api_key_env) plus LiteLLM's openai/ routing the vllm_* profiles already use — no new code path.
  • Env-driven base URL, scoped to this profile only. resolved_base_url special-cases openai_compatible; every other profile's base_url stays fixed in the table, so nothing can be silently re-pointed by an env var.
  • Provider-prefix env names (OPENAI_BASE_URL / OPENAI_API_KEY) to match DEEPINFRA_API_KEY, NVIDIA_NIM_API_KEY, VLLM_API_KEY — instantly recognizable to anyone who's used an OpenAI-compatible tool.

Changes

braindb/config.py (profile + scoped resolved_base_url + EMPTY-key fallback for the env endpoint), docker-compose.yml (env passthrough), .env.example, README.md, CONTRIBUTING.md, CHANGELOG.md, and tests/test_config_profiles.py.

Verification

  • New unit tests pass stack-free, including a regression that OPENAI_BASE_URL does not leak into other profiles.
  • Full suite green against the isolated test stack; docker compose config parses with the new env present.

Reworked from @WarGloom's PR #7 — thanks for the nudge and the original direction. Adjusted to the provider-prefix env convention and a profile-scoped override per the review there.

Point the internal agent at any OpenAI-compatible /v1 endpoint (Ollama,
LM Studio, copilot-api, a remote vLLM) without a new provider abstraction:
set LLM_PROFILE=openai_compatible, AGENT_MODEL=openai/<model-id>, and
OPENAI_BASE_URL (OPENAI_API_KEY only if the endpoint needs auth).

- new profile reuses the existing model-prefix + api_key_env mechanism
- the env-driven base_url is scoped to this profile ONLY; resolved_base_url
  special-cases it so no other profile (deepinfra/nim/vllm_*) can be
  silently re-pointed
- provider-prefix env names (OPENAI_*) to match DEEPINFRA_API_KEY etc.
- unit coverage incl. a regression that OPENAI_BASE_URL does not leak into
  other profiles; docker-compose env passthrough, .env.example, README,
  CONTRIBUTING, CHANGELOG

Reworked from @WarGloom's proposal in #7 (thanks for the nudge), adjusted
to the provider-prefix convention and a profile-scoped override per review.
@dimknaf
dimknaf merged commit b8be203 into main Jun 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant