fix(models): list env-credentialed providers in /v1/models

[njbrake]

Description

/v1/models discovery and completion routing disagreed on what a "configured provider" is. Completions work for any any-llm provider once its native credential env var is set (get_provider_kwargs returns {} for a provider absent from config.providers, so any-llm reads the env var directly), but model discovery only queried config.providers. Setting only ANTHROPIC_API_KEY therefore made Anthropic completions work while its models never appeared in GET /v1/models, the exact quick path the Railway template recommends.

Discovery now also picks up any-llm providers made callable by an env var alone: a provider whose credential env var is present and that supports live model listing is treated as discoverable under its implementation name (the same provider:model selector routing resolves), unless it is already a configured instance. This is wired into both GET /v1/models and GET /v1/models/discoverable (and the ?provider= filter), so the two surfaces agree with what completions can reach. Provider-health and the per-instance connection test are untouched; they operate on explicit or configured instances.

Also restores os.environ around each test in test_config_env_loading, whose python-dotenv loads leaked provider credentials into the process. That was harmless before, but now that discovery is env-aware it polluted an unrelated test; a pre-existing isolation gap the change surfaced.

PR Type

• New Feature
• Bug Fix
• Refactor
• Documentation
• Infrastructure / CI

Relevant issues

Fixes #221

Checklist

• I understand the code I am submitting.
• I have added or updated tests that cover my change (tests/unit, tests/integration).
• I ran the Definition of Done checks locally (make lint, make typecheck, make test).
• Documentation was updated where necessary.
• If the API contract changed, I regenerated the OpenAPI spec (no contract change; --check confirms the spec is up to date).

AI Usage

• No AI was used.
• AI was used for drafting/refactoring.
• This is fully AI-generated.

AI Model/Tool used:

Claude Code (Opus 4.8), driven by @njbrake.

Any additional AI details you'd like to share:

Implemented and tested end to end by Claude via back-and-forth with @njbrake; the reasoning and decisions are his.

NOTE:
When responding to reviewer questions, please respond yourself rather than copy/pasting reviewer comments into an AI and pasting back its answer. We want to discuss with you, not your AI :)

• I am an AI Agent filling out this form (check box if true)

[coderabbitai]

Warning

Review limit reached

@njbrake, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 36 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: fcdfe80c-7239-433e-9e79-0af32b1d1dad

📥 Commits

Reviewing files that changed from the base of the PR and between fa68234 and 2b79d3e.

📒 Files selected for processing (7)

• docs/configuration.md
• docs/models.md
• src/gateway/services/model_discovery_service.py
• tests/integration/test_config_env_loading.py
• tests/integration/test_model_discovery.py
• tests/unit/test_gateway_model_discovery.py
• tests/unit/test_model_discovery_env_providers.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/models-discovery-disagreement

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch fix/models-discovery-disagreement

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Fixes a mismatch between completion routing and model discovery by treating any-llm providers as “discoverable” when their native credential environment variable is set, even if they are not explicitly listed under config.providers. This makes /v1/models (and the operator-facing discovery/status endpoints) align with what completions can actually reach.

Changes:

• Extend model discovery to include env-credentialed any-llm providers (when they support live model listing) and apply this consistently to /v1/models, /v1/models/discoverable, and provider filtering.
• Add unit and integration regression tests covering env-only provider discovery (issue #221) and harden env isolation in config env-loading tests.
• Update documentation to reflect that env-only credentials affect both routing and model discovery.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/gateway/services/model_discovery_service.py	Adds env-var-based provider detection and uses it to build the discoverable instance list for discovery and status surfaces.
tests/unit/test_model_discovery_env_providers.py	New unit tests validating env-only provider detection, ordering, filtering, and status inclusion.
tests/unit/test_gateway_model_discovery.py	Pins existing discovery tests to configured providers by stubbing env-provider detection.
tests/integration/test_model_discovery.py	Adds an integration regression test asserting env-only providers appear in GET /v1/models.
tests/integration/test_config_env_loading.py	Restores os.environ after each test to prevent dotenv-loaded provider creds from leaking across tests.
docs/models.md	Documents that env-only provider credentials affect both routing and discovery.
docs/configuration.md	Updates model_discovery setting description to include env-only discoverable providers.

[khaledosman]

Approving. Detailed points are inline (see the follow-up review). Themes:

• _env_provider_instances dedups by instance name, not implementation — a configured instance plus its matching credential env var can double-discover the same provider.
• AnyLLM.get_all_provider_metadata() is the one unguarded external call in an otherwise defensive module; a raise there 500s /v1/models.
• Env edge cases (slash-alternatives, compound, "None", blank) verified correct; no new capability exposed. Tests are solid bar the same-implementation duplication case.

[khaledosman]

Follow-up: converting the earlier prose review into inline comments; see threads on model_discovery_service.py and the test files. No blockers.

Retraction: my earlier cross-PR concern (that this PR's Railway/PaaS promotion collides with a Secure-cookie downgrade in #384) was based on a misread of #384 against main. #384 honors X-Forwarded-Proto via request_is_https(), so the proxy case is handled and the concern does not apply. Withdrawn.