Add Google Vertex AI provider for Anthropic Claude models

[harche]

Summary

• Adds a new google_vertex LLM provider type that enables using Anthropic Claude models (e.g. claude-opus-4-6) through Google Cloud's Vertex AI
• Uses langchain-anthropic with a custom ChatAnthropicVertexAI subclass that routes authentication through Google ADC via the anthropic SDK's AnthropicVertex client
• Avoids langchain-google-vertexai due to an unresolvable httpx version conflict with the existing ibm-generative-ai dependency

Changes

• ols/constants.py — Added PROVIDER_GOOGLE_VERTEX constant
• ols/src/llms/providers/google_vertex.py — New provider implementation
• ols/src/llms/providers/provider.py — Added parameter definitions and mappings
• ols/app/models/config.py — Added GoogleVertexConfig model and wired into ProviderConfig
• pyproject.toml / uv.lock — Added langchain-anthropic>=0.3.12 dependency
• tests/unit/llms/providers/test_google_vertex.py — Unit tests (3 tests)
• tests/unit/llms/providers/test_providers.py — Added to registration test
• docs/ai/providers.md — Updated provider reference table
• examples/olsconfig.yaml — Added example configuration

Test plan

• Unit tests pass (pytest tests/unit/llms/ — 78 tests)
• Config model tests pass (pytest tests/unit/app/models/test_config.py — 155 tests)
• Live integration test with Vertex AI (claude-opus-4-6, us-east5 region) confirmed working
• CI pipeline validation

🤖 Generated with Claude Code

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign bparees for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[harche]

Local integration test with Vertex AI

Tested locally against Google Vertex AI using Application Default Credentials:

• Region: us-east5
• Model: claude-opus-4-6

$ uv run python -c "
from ols.app.models.config import ProviderConfig
from ols.src.llms.providers.google_vertex import GoogleVertex

provider_config = ProviderConfig({
    'name': 'vertex_test',
    'type': 'google_vertex',
    'url': 'https://us-east5-aiplatform.googleapis.com',
    'credentials_path': 'tests/config/secret/apitoken',
    'project_id': '...',
    'google_vertex_config': {
        'project': '...',
        'location': 'us-east5',
    },
    'models': [{'name': 'claude-opus-4-6', 'url': 'https://us-east5-aiplatform.googleapis.com', 'credentials_path': 'tests/config/secret/apitoken'}],
})

vertex = GoogleVertex(model='claude-opus-4-6', params={}, provider_config=provider_config)
llm = vertex.load()
response = llm.invoke('What is OpenShift? Answer in one sentence.')
print('Provider:', type(llm).__name__)
print('Response:', response.content)
"

Provider: ChatAnthropicVertexAI
Response: OpenShift is a Kubernetes-based container platform developed by Red Hat
that enables developers to build, deploy, and manage containerized applications
across hybrid and multi-cloud environments.

[mrunalp]

@xrajesh ptal

[harche]

Note on langchain-google-vertexai

I intentionally used langchain-anthropic + the anthropic SDK's AnthropicVertex client instead of ChatAnthropicVertex from langchain-google-vertexai. The latter has an unresolvable dependency conflict:

• langchain-google-vertexai>=2.1.0 requires httpx>=0.28
• ibm-generative-ai (used by the BAM provider) requires httpx<0.28
• langchain-google-vertexai<2.1.0 requires langchain-core<0.4, which conflicts with langchain-community>=0.3.81 requiring langchain-core>=1.0.0

The workaround is a thin ChatAnthropicVertexAI subclass of ChatAnthropic that overrides _client and _async_client to use the Vertex-specific clients from the anthropic SDK. Functionally equivalent, no extra dependencies needed beyond langchain-anthropic.

[harche]

Gemini on Vertex AI — blocked by ibm-generative-ai httpx pin

I looked into adding Gemini support over Vertex AI as well, but every Google SDK that supports Vertex (langchain-google-vertexai, google-cloud-aiplatform, google-genai) requires httpx>=0.28. This conflicts with ibm-generative-ai==3.0.0 which pins httpx<0.28.

ibm-generative-ai 3.0.0 is the latest version, released May 2024 with no updates since. The BAM provider is the only consumer. Dropping or replacing this dependency would unblock all Google Vertex AI packages (including Gemini).

[xrajesh]

thank you @harche .
We don't need to support BAM anymore.
I will get @raptorsun to review and take it forward . He is working on supporting Gemini models.

[openshift-ci]

@harche: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ols-cluster	6e041d4	link	true	/test e2e-ols-cluster
ci/prow/ols-evaluation	6e041d4	link	true	/test ols-evaluation

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.