feat(bundles): add vLLM model-provider bundle (lfx-vllm) — inherits #13910#13919
Merged
Conversation
Ships vLLM as a standalone Langflow Extension Bundle built on the provider-registry seam: its extension.json declares a providers[] entry that registers a vLLM model provider into the unified model system, with zero core edits. vLLM is OpenAI-compatible, so the provider reuses ChatOpenAI / OpenAIEmbeddings and discovers served models live from /v1/models (SSRF-guarded); the same model is offered for both Language Model and Embedding Model use, and no API key is required for local servers. Inherits the original vLLM provider from #13910 by Yash Pareek (@pareek-ml), reconstructed as a bundle so it no longer edits the 12 core files that PR did. Co-authored-by: Yash Pareek <yash.pareek@usi.ch>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
✅ Test Coverage AdvisorNo source changes detected without accompanying tests. Thanks for keeping coverage up! 🎉
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## feat/model-provider-bundles #13919 +/- ##
============================================================
Coverage 58.78% 58.78%
============================================================
Files 2351 2351
Lines 225186 225186
Branches 33572 33572
============================================================
Hits 132373 132373
Misses 91267 91267
Partials 1546 1546
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
github-actions
Bot
added
enhancement
erichare
added a commit
that referenced
this pull request
Jun 30, 2026
* feat(lfx): let extension bundles register model providers Model providers were hardcoded across lfx core (MODEL_PROVIDER_METADATA, the class-import registries, LIVE_MODEL_PROVIDERS, and the credential / instantiation / live-discovery dispatch), so a third party could not add a provider without editing core -- adding vLLM (#13910) touched 12 files. Introduce a supported extension point: an extension manifest may declare a `providers[]` block, which the loader merges into the core provider tables in place via a new `provider_registry`. Every existing accessor (the lru_cached variable maps, the shared `model_provider_metadata` reference, the /api/v1/models surface) then sees a registered provider with no further edits. - new lfx/base/models/provider_registry.py: ProviderSpec + register_provider with core-wins precedence, failure isolation, and a clear() test seam; live-discovery and validator callables resolved lazily by dotted path. - manifest: declarative providers[]; allow a provider-only extension (empty bundles); require at least one bundle or provider. - loader: register manifest providers during load_extension (the startup chokepoint), failure-isolated as warnings. - dispatch seams made registry-aware: get_model_providers union, live-model discovery fallback, credential-validation fallback, api-key-optional check. - zero added cost and byte-identical behavior when no provider bundle is installed. Groundwork for shipping providers as standalone bundles (e.g. lfx-vllm, superseding #13910). * feat(lfx): apply registered providers' connection variables generically get_llm/get_embeddings resolved base_url (and attribution headers) only via hardcoded per-provider branches, so a bundle-contributed OpenAI-compatible provider could not point its client at a custom endpoint -- it would silently hit the default API. Add a generic, is_registered-gated step that applies each non-secret metadata variable to its declared langchain_param (base_url localhost-rewritten) or HTTP header, mirroring the core branches. Core providers keep their explicit branches, so behavior is unchanged. Completes the provider-bundle seam so an OpenAI-compatible provider (e.g. vLLM) works fully from a bundle with zero core edits. * feat(lfx): pass a placeholder api key for key-optional registered providers langchain_openai's ChatOpenAI / OpenAIEmbeddings raise when constructed with api_key=None, so an OpenAI-compatible provider that declared api_key_required=False (e.g. a local vLLM server without auth) would fail to instantiate even though the model system correctly skips the "API key required" error. Pass a non-empty "EMPTY" placeholder for such providers so the client constructs. Only applies to registered (bundle) providers that opted out of API keys; core providers are unaffected. * feat(lfx): apply the api key for key-optional registered embedding providers get_embeddings only passed an API key when the provider's param_mapping declared an explicit "api_key" slot. Bundle providers can omit that slot, so apply the resolved key (or the api-key-optional placeholder) under the OpenAI-compatible "api_key" kwarg for registered providers -- mirroring how get_llm already resolves the key. Lets an OpenAI-compatible embedding provider (e.g. vLLM) work from a bundle without that slot. * feat(bundles): add vLLM model-provider bundle (lfx-vllm) — inherits #13910 (#13919) feat(bundles): add vLLM model-provider bundle (lfx-vllm) Ships vLLM as a standalone Langflow Extension Bundle built on the provider-registry seam: its extension.json declares a providers[] entry that registers a vLLM model provider into the unified model system, with zero core edits. vLLM is OpenAI-compatible, so the provider reuses ChatOpenAI / OpenAIEmbeddings and discovers served models live from /v1/models (SSRF-guarded); the same model is offered for both Language Model and Embedding Model use, and no API key is required for local servers. Inherits the original vLLM provider from #13910 by Yash Pareek (@pareek-ml), reconstructed as a bundle so it no longer edits the 12 core files that PR did. Co-authored-by: Yash Pareek <yash.pareek@usi.ch> * fix(lfx): address CodeRabbit review on the provider seam - model_utils: normalize non-list live-discovery returns to [] (list[dict] contract) - provider_registry: validate lazy-import/embedding keys before mutating any global table (reject unknown or conflicting model/embedding classes and embedding_param_key collisions, so clear() can't drop a core mapping); _resolve_callable verifies the target is callable; callable resolution now catches broad import-time failures (SyntaxError, side effects) and degrades to None - credentials: normalize non-ValueError bundle-validator failures to ValueError so they can't escape as an unhandled 500 - loader: surface skipped (name-collision) provider registrations as a typed provider-skipped warning instead of returning silent success - errors: register provider-invalid / provider-skipped codes + format templates - tests: cover the register-time warning-isolation and collision paths, unknown/conflicting keys, non-callable and non-list discovery, and validator normalization * docs(bundle-api): changelog for the providers[] manifest surface Manifest providers[] (model-provider registration) and the optional 0-or-1 bundles[] (provider-only extensions) touch in-scope BUNDLE_API files (manifest.py, _orchestrator.py, errors.py); record the additive contract change + the new provider-invalid / provider-skipped codes so the changelog gate passes. No BUNDLE_API_VERSION bump (all additive). * fix(lfx): correct provider api-key resolution + provider-only discovery Addresses two review findings on the provider seam: - [P1] get_model_provider_variable_mapping() fell back to the FIRST provider variable when no *required* secret existed, so a provider whose API key is an optional secret (e.g. vLLM's VLLM_API_KEY) mapped to its required non-secret base URL. get_api_key_for_provider then resolved and forwarded the endpoint as the bearer token, skipping the "EMPTY" placeholder. Now prefer a required secret, then any secret, then the first variable. Covered by a test that exercises the real resolver (no patched stub). - [P2] discover_installed_extensions / discover_seed_extensions indexed manifest.bundles[0], crashing with IndexError on a valid provider-only manifest (bundles=[]) and breaking `lfx extension list` / registry discovery. Guard the empty-bundle case; DiscoveredExtension.bundle_name and registry.Extension.bundle_name are now str | None, and the list CLI renders a dash for provider-only extensions. BUNDLE_API.md changelog updated for the discovery/registry surface nullability. --------- Co-authored-by: Yash Pareek <yash.pareek@usi.ch>
This was referenced Jun 30, 2026
5 tasks
erichare
added a commit
that referenced
this pull request
Jul 1, 2026
…13940) * feat: add OpenAI Compatible as a first-class unified model provider Ships a new lfx-openai-compatible extension bundle that registers an "OpenAI Compatible" provider in the unified model system, so any OpenAI-compatible endpoint (OpenRouter, Together, Groq, Fireworks, self-hosted vLLM/TGI/LM Studio, ...) can be configured once in the Models pane and reused across flows, including the Agent component. Follows the provider-bundle pattern established by lfx-vllm (#13919): - OPENAI_COMPATIBLE_BASE_URL (required) + OPENAI_COMPATIBLE_API_KEY (optional, secret) variables; api_key_required=false for local servers - Reuses ChatOpenAI / OpenAIEmbeddings; no core table edits - Live model discovery from the endpoint's /v1/models route with SSRF validation; credential validation probes the same route - Ships by default via the root workspace, like the other bundles - Frontend: map the provider to the Lucide "Plug" icon Fixes #12839 * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * fix: wrap non-auth endpoint validation failures in user-facing ValueError Per CodeRabbit review on #13940: a 404/500 from the /v1/models probe previously escaped validate_openai_compatible_credentials as a raw requests.HTTPError instead of the ValueError shape used for auth, connection, and timeout failures. Catch HTTPError/RequestException and re-raise ValueError with the endpoint and status. Also add tests for the server-error path and the API-key-lookup-exception fallback in discovery. * fix: surface unconfigured live-only bundle providers in /api/v1/models Providers contributed by extension bundles (OpenAI Compatible, vLLM) ship no static catalog rows and rely entirely on live discovery, so list_models never emitted them until they were configured -- but the Model Providers dialog is where they get configured, a bootstrap dead-end that made the new provider undiscoverable in the UI. - lfx: add get_live_only_providers() -- providers with registered metadata and a live-discovery gate but no static catalog rows. Gated on live capability so metadata-only, non-live providers (Azure OpenAI, Groq) stay hidden as before. - api: union these into list_models with an empty model list (full provider metadata attached) so the dialog offers their configuration form; once configured, replace_with_live_models fills the same entry with the endpoint's discovered models. Skipped for model_name/metadata queries, which ask about concrete models. - api: re-apply the ?provider= filter after replace_with_live_models, which iterates every live-capable provider and could append providers outside the requested filter (e.g. a vacuously-configured OpenRouter in a ?provider=OpenAI response). - tests: registry coverage for the helper (lfx), endpoint coverage for the union + filter contract (backend), bundle e2e assertion. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
lfx-vllm— vLLM as a standalone Langflow model-provider bundle, built on the provider-registry extension point from #13916. PR 2 of 2; supersedes #13910.Note
Stacked on #13916 (base branch
feat/model-provider-bundles). Review/merge #13916 first — GitHub retargets this torelease-1.11.0once it lands.How it works
The bundle ships no component — it contributes a provider via a
providers[]block inextension.json, which the registry merges into the unified model system. vLLM is OpenAI-compatible, so it:ChatOpenAI/OpenAIEmbeddings(lazily imported; no new dependency),/v1/models(SSRF-guarded), tagging each with the requestedmodel_typeso the same model is offered for both Language Model and Embedding Model use,api_key_required: false; a placeholder is supplied so the OpenAI-compatible client constructs).It edits zero Langflow core files — the 12-file footprint of #13910 collapses to a self-contained
src/bundles/vllm/(onediscovery.pyfor the live fetch + credential validation, plus the declarative manifest).Credit
Inherits the original vLLM provider from #13910 by Yash Pareek (@pareek-ml). The bundle commit is authored by him (
--author+Co-authored-bytrailer) so attribution survives the squash-merge. This PR reconstructs that work on the new seam so it ships without core edits.Tests
src/bundles/vllm/tests/test_vllm_provider.py— adapted from #13910: live/v1/modelsdiscovery (OpenAI-dict and plain-list payloads,/v1dedup, bearer header, degradation paths), credential validation (missing URL, 401/403, connection error, timeout), and an end-to-end load through the real extension loader asserting vLLM registers and appears inget_model_providers(). 20 passed.Closes #13910