Skip to content

feat: add Ollama, Azure OpenAI, and generic OpenAI-compatible providers#179

Open
mimran-khan wants to merge 1 commit into
NVIDIA:mainfrom
mimran-khan:feat/new-providers
Open

feat: add Ollama, Azure OpenAI, and generic OpenAI-compatible providers#179
mimran-khan wants to merge 1 commit into
NVIDIA:mainfrom
mimran-khan:feat/new-providers

Conversation

@mimran-khan

Copy link
Copy Markdown
Contributor

Summary

Add three new LLM providers to expand SkillSpector beyond cloud-only OpenAI/Anthropic/NVIDIA endpoints:

  • Ollama (SKILLSPECTOR_PROVIDER=ollama) — free local/offline scanning via Ollama's OpenAI-compatible API. No API key required. Supports remote instances via OLLAMA_BASE_URL.
  • Azure OpenAI (SKILLSPECTOR_PROVIDER=azure_openai) — enterprise Azure-hosted deployments using AzureChatOpenAI with deployment-based routing, api-version handling, and dedicated env vars (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT).
  • Generic OpenAI-compatible (SKILLSPECTOR_PROVIDER=openai_compatible) — Groq, Together AI, Mistral, DeepInfra, Fireworks, and any OpenAI-compatible endpoint via SKILLSPECTOR_COMPAT_API_KEY + SKILLSPECTOR_COMPAT_BASE_URL.

Each provider ships its own model_registry.yaml for accurate token budgeting.

Changes

  • src/skillspector/providers/ollama/ — provider, model registry, __init__
  • src/skillspector/providers/azure_openai/ — provider, model registry, __init__
  • src/skillspector/providers/openai_compatible/ — provider, model registry, __init__
  • src/skillspector/providers/__init__.py — register new providers in selector, update error message to mention ollama
  • tests/unit/test_new_providers.py — 35 tests covering credentials, chat model creation, metadata, selection, and error messages

Closes #173, closes #174, closes #175

Test plan

  • All 35 new tests pass (pytest tests/unit/test_new_providers.py -v)
  • All 45 existing provider tests pass (no regressions)
  • Ollama always returns credentials (no API key needed)
  • Ollama custom base URL via OLLAMA_BASE_URL
  • Azure requires both AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT
  • Azure deployment defaults to model name when AZURE_OPENAI_DEPLOYMENT unset
  • Azure API version defaults to 2024-06-01
  • Generic requires both SKILLSPECTOR_COMPAT_API_KEY and SKILLSPECTOR_COMPAT_BASE_URL
  • All three providers' model registries return correct token budgets
  • Unknown provider error message lists all new providers

Ollama provider enables free local/offline LLM scanning via Ollama's
OpenAI-compatible API at localhost:11434. No API key required.

Azure OpenAI provider supports enterprise deployments using
AzureChatOpenAI with deployment-based routing and api-version handling.

Generic OpenAI-compatible provider serves Groq, Together AI, Mistral,
DeepInfra, and other endpoints with dedicated env vars
(SKILLSPECTOR_COMPAT_API_KEY/BASE_URL) and a bundled multi-provider
model registry.

Each provider ships its own model_registry.yaml for accurate token
budgeting.

Closes NVIDIA#173, closes NVIDIA#174, closes NVIDIA#175

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Approved.

Three providers (Ollama, Azure OpenAI, generic OpenAI-compatible) following the established provider pattern. Credential resolution fails closed (Azure and the generic provider return None unless both env vars are set; Ollama supplies the required placeholder key), the Azure key is wrapped in SecretStr, and each ships a bundled model_registry.yaml. 35 tests cover credentials (positive/negative), chat-model creation, metadata, selection, and the updated error message.

Security: the configurable base URLs (OLLAMA_BASE_URL, SKILLSPECTOR_COMPAT_BASE_URL, AZURE_OPENAI_ENDPOINT) are operator-set env vars — the same trust surface as the existing OPENAI_BASE_URL — so no new attacker-controlled SSRF vector from scanned content. Non-blocking: could validate that the base URL is a well-formed http(s) URL to fail earlier on misconfiguration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants