feat: add openai_compatible LLM profile - #16
Merged
Conversation
Point the internal agent at any OpenAI-compatible /v1 endpoint (Ollama, LM Studio, copilot-api, a remote vLLM) without a new provider abstraction: set LLM_PROFILE=openai_compatible, AGENT_MODEL=openai/<model-id>, and OPENAI_BASE_URL (OPENAI_API_KEY only if the endpoint needs auth). - new profile reuses the existing model-prefix + api_key_env mechanism - the env-driven base_url is scoped to this profile ONLY; resolved_base_url special-cases it so no other profile (deepinfra/nim/vllm_*) can be silently re-pointed - provider-prefix env names (OPENAI_*) to match DEEPINFRA_API_KEY etc. - unit coverage incl. a regression that OPENAI_BASE_URL does not leak into other profiles; docker-compose env passthrough, .env.example, README, CONTRIBUTING, CHANGELOG Reworked from @WarGloom's proposal in #7 (thanks for the nudge), adjusted to the provider-prefix convention and a profile-scoped override per review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an
openai_compatibleprofile so the internal agent can talk to any OpenAI-compatible/v1endpoint — Ollama, LM Studio, copilot-api, a remote vLLM — without introducing another provider abstraction.Why this shape
_LLM_PROFILES(model-prefix +api_key_env) plus LiteLLM'sopenai/routing thevllm_*profiles already use — no new code path.resolved_base_urlspecial-casesopenai_compatible; every other profile'sbase_urlstays fixed in the table, so nothing can be silently re-pointed by an env var.OPENAI_BASE_URL/OPENAI_API_KEY) to matchDEEPINFRA_API_KEY,NVIDIA_NIM_API_KEY,VLLM_API_KEY— instantly recognizable to anyone who's used an OpenAI-compatible tool.Changes
braindb/config.py(profile + scopedresolved_base_url+EMPTY-key fallback for the env endpoint),docker-compose.yml(env passthrough),.env.example,README.md,CONTRIBUTING.md,CHANGELOG.md, andtests/test_config_profiles.py.Verification
OPENAI_BASE_URLdoes not leak into other profiles.docker compose configparses with the new env present.Reworked from @WarGloom's PR #7 — thanks for the nudge and the original direction. Adjusted to the provider-prefix env convention and a profile-scoped override per the review there.