diff --git a/CHANGELOG.md b/CHANGELOG.md index f36a72b..ff3fd13 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,21 @@ versions (e.g. `0.0.0.dev60+g257737d`). ## [Unreleased] +### Added +- Direct cloud-provider seam — `LLM_PROVIDER` (`ReviewerConfig.llm_provider`, + default `openai-compatible`, byte-identical to today's behaviour) selects + `anthropic` or `bedrock` to run cora with no LiteLLM/OpenAI-compatible + gateway in between: an Anthropic API key (`ANTHROPIC_API_KEY`, or + `LLM_GATEWAY_KEY` which takes precedence) or AWS credentials (Bedrock's + standard credential chain) are all that's needed. Both paths drop + non-default sampling parameters and the vLLM `enable_thinking` extra_body + toggle (current Claude models reject/no-op them) and fall back to the + provider's own response `model` field for the check-run "Backend" chip + when no `x-litellm-*` headers are present. New optional extras + `cora[anthropic]` / `cora[bedrock]`; both imports are lazy, so the default + path never requires them installed. See `docs/configuration.md` → + "Running against a cloud provider". + ## [0.1.0] - 2026-07-03 First public release. cora's development history predates this diff --git a/docs/configuration.md b/docs/configuration.md index 0028f3b..d0bfbcf 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -14,12 +14,14 @@ below are the supported adopter-facing knobs. | `LLM_GATEWAY_KEY` | API key for the OpenAI-compatible endpoint | | `LITELLM_BASE_URL` | OpenAI-compatible base URL | | `REVIEW_MODEL` | Model alias sent to the endpoint | +| `LLM_PROVIDER` | `openai-compatible` (default) / `anthropic` / `bedrock` — see below | `LITELLM_BASE_URL` is named for the LiteLLM gateway convention, but the client expects any OpenAI-compatible API — it can point at LiteLLM, vLLM, or another compatible gateway. See [Running against a cloud provider](#running-against-a-cloud-provider) -for the Anthropic/Bedrock topology. +for the Anthropic/Bedrock topology, including the gateway-less +`LLM_PROVIDER=anthropic`/`bedrock` direct-SDK paths. ## Review Mode And Budgets @@ -135,6 +137,13 @@ behind review setup: ## Running against a cloud provider +cora talks to a cloud model one of two ways: through an OpenAI-compatible +gateway (the default, works today with zero code changes), or directly +against the Anthropic API / Amazon Bedrock via `LLM_PROVIDER` — no proxy +in between. + +### Via a gateway (default `LLM_PROVIDER=openai-compatible`) + cora speaks one dialect — OpenAI Chat Completions — to whatever `LITELLM_BASE_URL` points at. Cloud models work today by letting the gateway do the translation: run a [LiteLLM proxy](https://docs.litellm.ai/) @@ -189,5 +198,57 @@ Notes for cloud deployments: caching helps: cora's agent loop has a frozen system prompt and an append-only history, so multi-turn prefixes are highly cacheable. -Direct provider SDK support (Anthropic / Bedrock without a gateway) is -planned as a provider seam in the agent factory — see the issue tracker. +### Direct — no gateway (`LLM_PROVIDER=anthropic` / `bedrock`) + +An adopter with only an Anthropic API key (or AWS credentials for +Bedrock) can skip the gateway entirely. `LLM_PROVIDER` selects the +dialect `make_review_agent` speaks; the default `openai-compatible` +value keeps every existing deployment byte-identical — this is +opt-in. + +**Anthropic:** + +```bash +export LLM_PROVIDER=anthropic +export ANTHROPIC_API_KEY=sk-ant-... # or LLM_GATEWAY_KEY, which wins if both are set +export REVIEW_MODEL=claude-sonnet-5 +export T1_MODEL=claude-opus-4-8 # optional — T1 continuation tier +``` + +Requires the `cora[anthropic]` extra (`pip install 'cora[anthropic]'`); +the import is lazy, so installing the base package alone never pulls in +the `anthropic` SDK. `LITELLM_BASE_URL` is ignored on this path unless +explicitly pointed somewhere other than its `localhost:4000` default — +set it only when routing through an Anthropic-compatible proxy that +isn't LiteLLM. + +**Bedrock:** + +```bash +export LLM_PROVIDER=bedrock +export AWS_REGION=us-east-1 # standard AWS credential chain — no LLM_GATEWAY_KEY needed +export REVIEW_MODEL=claude-opus-4-8 # the `anthropic.` Bedrock prefix is added automatically +``` + +Requires the `cora[bedrock]` extra (`pip install 'cora[bedrock]'`, +which pulls in `boto3`). Authenticates via the standard AWS credential +chain (env vars, shared profile, instance role, ...) — no API key +field is read or required. + +**Both direct paths, by design:** + +- **Sampling parameters are dropped.** cora's engine tunes + `temperature=0.2` for the gateway path; current Claude models reject + non-default `temperature`/`top_p`/`top_k` with a 400, so the direct + paths omit them entirely rather than forwarding a value that would + break every call. +- **The vLLM `enable_thinking` toggle never fires.** `AGENT_REVIEW_ENABLE_THINKING` + is a self-hosted-reasoning-backend knob (`chat_template_kwargs` is a + vLLM extra_body shape); cloud Claude models use their own adaptive- + thinking defaults instead, so it's a no-op here rather than an error. +- **The check-run "Backend" chip still works** — with no `x-litellm-*` + headers to read (there's no gateway to emit them), it falls back to + the provider response's own `model` field. +- **Deep mode is unchanged** — the agent loop, MCP attachments, and + local `grep_repo`/`git_show` tools all work identically; only the + model-construction seam differs. diff --git a/pyproject.toml b/pyproject.toml index bd6e10d..5b30de1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -52,6 +52,20 @@ otel = [ "opentelemetry-sdk==1.42.1", "opentelemetry-exporter-otlp-proto-grpc==1.42.1", ] +# Direct-SDK provider seam (`LLM_PROVIDER=anthropic` / +# `ReviewerConfig.llm_provider="anthropic"`) — gateway-less, straight +# to the Anthropic API. Only pulled in when an adopter opts in; the +# default `"openai-compatible"` path never imports the `anthropic` SDK +# (`cora.core.agent` imports it lazily inside the provider branch). +anthropic = [ + "pydantic-ai-slim[anthropic]>=1.102", +] +# Direct-SDK provider seam for Amazon Bedrock (`LLM_PROVIDER=bedrock`) +# — AWS credential chain, no gateway. Same lazy-import discipline as +# the `anthropic` extra above. +bedrock = [ + "pydantic-ai-slim[bedrock]>=1.102", +] dev = [ "pytest", "ruff", diff --git a/src/cora/config.py b/src/cora/config.py index b9febf1..dfa2830 100644 --- a/src/cora/config.py +++ b/src/cora/config.py @@ -48,6 +48,13 @@ class ReviewerConfig: llm_base_url: str = _c.DEFAULT_LITELLM_BASE llm_api_key: str | None = None model: str = _c.DEFAULT_MODEL + # Which dialect `make_review_agent` speaks. Default + # `"openai-compatible"` is byte-identical to every existing + # deployment; `"anthropic"` / `"bedrock"` are gateway-less direct-SDK + # paths (see `core.config.SUPPORTED_LLM_PROVIDERS`). `llm_base_url` + # is ignored on those paths unless explicitly pointed away from the + # `DEFAULT_LITELLM_BASE` placeholder (see `cora.core.agent`). + llm_provider: str = _c.DEFAULT_LLM_PROVIDER # ── MCP attachments (optional; deep mode) ──────────────────────── mcp_url: str = _c.DEFAULT_MCP_URL @@ -286,11 +293,29 @@ def getalias(name: str, default: str) -> str: except ValueError: t1_max_iterations = _c.DEFAULT_T1_MAX_ITERATIONS + # Direct-SDK provider seam. An unknown value fails loudly (like + # the escalation-triggers CSV below) rather than silently + # falling back to the gateway path. + llm_provider = getalias("LLM_PROVIDER", _c.DEFAULT_LLM_PROVIDER) + if llm_provider not in _c.SUPPORTED_LLM_PROVIDERS: + raise ValueError( + f"unknown LLM_PROVIDER: {llm_provider!r} " + f"(supported: {sorted(_c.SUPPORTED_LLM_PROVIDERS)})" + ) + # `llm_api_key` also accepts `ANTHROPIC_API_KEY` on the direct + # Anthropic path so `LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=…` + # runs with no gateway secret at all. Bedrock needs no API key + # (AWS credential chain) so it isn't threaded here. + llm_api_key = get("LLM_GATEWAY_KEY") + if llm_api_key is None and llm_provider == "anthropic": + llm_api_key = get("ANTHROPIC_API_KEY") + cfg = cls( repo=get("GH_REPO") or get("GITHUB_REPOSITORY") or "", pr_number=get("PR_NUMBER") or "", llm_base_url=get("LITELLM_BASE_URL", _c.DEFAULT_LITELLM_BASE), - llm_api_key=get("LLM_GATEWAY_KEY"), + llm_api_key=llm_api_key, + llm_provider=llm_provider, model=get("REVIEW_MODEL", _c.DEFAULT_MODEL), mcp_url=get("MCP_URL", _c.DEFAULT_MCP_URL), mcp_token=get("MCP_TOKEN"), diff --git a/src/cora/core/agent.py b/src/cora/core/agent.py index 664a01a..52ab87e 100644 --- a/src/cora/core/agent.py +++ b/src/cora/core/agent.py @@ -12,7 +12,12 @@ Provider-agnostic by design: the reviewer talks to whatever OpenAI-compatible endpoint sits behind `endpoint_base_url`. The reference deployment fronts its models with LiteLLM; the application -code doesn't know or care. +code doesn't know or care. `AgentConfig.provider` widens that seam to +two gateway-less direct-SDK dialects (`"anthropic"`, `"bedrock"`) for +an adopter with only a cloud API key/credentials — see +`_build_anthropic_model` / `_build_bedrock_model` and +`core.config.SUPPORTED_LLM_PROVIDERS`. The default `"openai-compatible"` +path is untouched byte-for-byte. """ from __future__ import annotations @@ -20,8 +25,11 @@ from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Callable, Literal +from cora.core import config as _c + if TYPE_CHECKING: from cora.config import ReviewerConfig + from pydantic_ai.settings import ModelSettings def _default_reviewer_config() -> "ReviewerConfig": @@ -88,6 +96,9 @@ class AgentConfig: # OpenAI-compatible endpoint (a LiteLLM gateway, vLLM-direct, # OpenRouter — anything OpenAI-compatible, per deployment). # Reviewer code stays generic via the generic `OpenAIProvider`. + # Ignored on the `"anthropic"` / `"bedrock"` provider paths unless + # explicitly pointed away from the localhost placeholder — see + # `_build_anthropic_model`. endpoint_base_url: str api_key: str @@ -102,6 +113,11 @@ class AgentConfig: # leak detector + verdict parser consume the response unchanged. system_prompt: str + # Which dialect to speak — mirrors `ReviewerConfig.llm_provider` / + # `core.config.SUPPORTED_LLM_PROVIDERS`. Default keeps the + # OpenAI-compatible path unchanged for every existing deployment. + provider: str = _c.DEFAULT_LLM_PROVIDER + # MCP servers — list of `(url, auth_headers)` tuples. Empty in # quick mode. Populated in deep mode with the three sessions # (the read-tools MCP server, the actions MCP server, the @@ -139,6 +155,134 @@ class AgentConfig: retries: int = 1 +def _build_openai_compatible_model(config: AgentConfig): + """The stock, default path — byte-identical to cora's pre-seam + behaviour. Talks OpenAI Chat Completions to whatever + `config.endpoint_base_url` points at (a LiteLLM gateway, + vLLM-direct, OpenRouter, ...).""" + from pydantic_ai.models.openai import OpenAIChatModel + from pydantic_ai.providers.openai import OpenAIProvider + + from cora.core.litellm_capture import build_capture_client + + # Custom httpx client carries a response event hook that drains + # `x-litellm-*` headers into a ContextVar so the call site can + # populate `Budget.resolved_model` / `litellm_headers` after + # `agent.run()` — the openai SDK otherwise hides them behind + # pydantic-ai's Agent wrapper. The client is owned by the + # underlying AsyncOpenAI; not explicitly closed because the + # per-PR process exits after the review lands. + openai_provider = OpenAIProvider( + base_url=config.endpoint_base_url, + api_key=config.api_key, + http_client=build_capture_client(), + ) + return OpenAIChatModel(config.model_alias, provider=openai_provider) + + +def _provider_base_url_override(config: AgentConfig) -> str | None: + """Optional base-url override for the direct-SDK provider paths. + + `config.endpoint_base_url` always carries a value (the OpenAI- + compatible path needs one structurally), so a bare gateway-less + `LLM_PROVIDER=anthropic` run — no `LITELLM_BASE_URL` set — still + arrives here holding the unmodified `DEFAULT_LITELLM_BASE` + placeholder. Forwarding that would point the direct SDK at a local + gateway that doesn't exist for this path, so it's treated as "no + override" — the SDK falls back to its own default endpoint. An + adopter who explicitly points `LITELLM_BASE_URL` somewhere else + (an Anthropic-compatible proxy, a regional endpoint, ...) gets it + passed through untouched.""" + base_url = (config.endpoint_base_url or "").strip() + if not base_url or base_url == _c.DEFAULT_LITELLM_BASE: + return None + return base_url + + +def _build_anthropic_model(config: AgentConfig): + """Direct Anthropic SDK path — no gateway in between. + + Requires the optional `cora[anthropic]` extra + (`pydantic-ai-slim[anthropic]`, which pulls in the `anthropic` + SDK); the import is lazy so the default openai-compatible path + never needs it installed.""" + try: + from pydantic_ai.models.anthropic import AnthropicModel + from pydantic_ai.providers.anthropic import AnthropicProvider + except ImportError as exc: + raise ImportError( + "LLM_PROVIDER=anthropic requires the 'anthropic' extra: " + "pip install 'cora[anthropic]'" + ) from exc + + anthropic_provider = AnthropicProvider( + api_key=config.api_key or None, + base_url=_provider_base_url_override(config), + ) + return AnthropicModel(config.model_alias, provider=anthropic_provider) + + +def _build_bedrock_model(config: AgentConfig): + """Direct Amazon Bedrock SDK path — AWS credential chain, no + gateway in between. + + Requires the optional `cora[bedrock]` extra + (`pydantic-ai-slim[bedrock]`, which pulls in `boto3`); the import + is lazy so the default openai-compatible path never needs it + installed. Bedrock model IDs carry an `anthropic.` prefix — a bare + alias like `claude-opus-4-8` (the shape every other provider + path/dashboard uses) is prefixed automatically; an already-prefixed + `model_alias` is left alone.""" + try: + from pydantic_ai.models.bedrock import BedrockConverseModel + except ImportError as exc: + raise ImportError( + "LLM_PROVIDER=bedrock requires the 'bedrock' extra: " + "pip install 'cora[bedrock]'" + ) from exc + + model_name = config.model_alias + if not model_name.startswith("anthropic."): + model_name = f"anthropic.{model_name}" + return BedrockConverseModel(model_name) + + +def build_model_settings( + cfg: "ReviewerConfig | None", + *, + max_tokens: int, + timeout_s: float, + extra: dict[str, Any] | None = None, +) -> "ModelSettings": + """Build the per-call `ModelSettings` for the configured provider. + + On the default `"openai-compatible"` path this is byte-identical + to what every call site constructed before the provider seam: + `max_tokens` + a fixed `temperature=0.2` + `timeout`, plus whatever + `extra` the caller threads (deep mode's `_thinking_extra_body` + vLLM `extra_body` toggle). + + On the direct `"anthropic"` / `"bedrock"` SDK paths, `temperature` + and `extra` are both dropped: current Claude models reject + non-default sampling parameters with a 400, the vLLM + `chat_template_kwargs.enable_thinking` shape is a no-op there + (cloud Claude's adaptive-thinking defaults are already correct), + and explicit thinking configuration isn't sent either — same + reasoning as the dropped sampling params. + """ + from pydantic_ai import ModelSettings + + provider = cfg.llm_provider if cfg is not None else _c.DEFAULT_LLM_PROVIDER + if provider == "openai-compatible": + return ModelSettings( + max_tokens=max_tokens, + temperature=0.2, + timeout=timeout_s, + **(extra or {}), + ) + return ModelSettings(max_tokens=max_tokens, timeout=timeout_s) + + def make_review_agent(config: AgentConfig, deps_type: type = Deps): """Build a Pydantic-AI Agent from the supplied config. @@ -163,29 +307,32 @@ def make_review_agent(config: AgentConfig, deps_type: type = Deps): schema. Local tools are listed before MCP toolsets, so on a name collision local wins (deep mode uses this so the PR-aware `grep_repo` / `git_show` shadow the MCP server's main-mirror copies). + + `config.provider` selects the dialect. Default + `"openai-compatible"` is this exact path, unchanged. `"anthropic"` / + `"bedrock"` build the model through pydantic-ai's direct SDK + integrations instead — see `_build_anthropic_model` / + `_build_bedrock_model`. Both require their own optional extra + (`cora[anthropic]` / `cora[bedrock]`); the import is lazy so the + default path never needs them installed. """ # Local imports keep the heavy framework dep off the module-load # path for environments where `pydantic-ai` isn't installed yet # (the smoke test handles that case with `pytest.importorskip`). from pydantic_ai import Agent - from pydantic_ai.models.openai import OpenAIChatModel - from pydantic_ai.providers.openai import OpenAIProvider - - from cora.core.litellm_capture import build_capture_client - # Custom httpx client carries a response event hook that drains - # `x-litellm-*` headers into a ContextVar so the call site can - # populate `Budget.resolved_model` / `litellm_headers` after - # `agent.run()` — the openai SDK otherwise hides them behind - # pydantic-ai's Agent wrapper. The client is owned by the - # underlying AsyncOpenAI; not explicitly closed because the - # per-PR process exits after the review lands. - provider = OpenAIProvider( - base_url=config.endpoint_base_url, - api_key=config.api_key, - http_client=build_capture_client(), - ) - model = OpenAIChatModel(config.model_alias, provider=provider) + provider = config.provider or _c.DEFAULT_LLM_PROVIDER + if provider == "openai-compatible": + model = _build_openai_compatible_model(config) + elif provider == "anthropic": + model = _build_anthropic_model(config) + elif provider == "bedrock": + model = _build_bedrock_model(config) + else: + raise ValueError( + f"unknown AgentConfig.provider: {provider!r} " + f"(supported: {sorted(_c.SUPPORTED_LLM_PROVIDERS)})" + ) toolsets: list[Any] = [] if config.mcp_servers: diff --git a/src/cora/core/budget.py b/src/cora/core/budget.py index c8a69b7..26025b2 100644 --- a/src/cora/core/budget.py +++ b/src/cora/core/budget.py @@ -185,6 +185,23 @@ def resolve_run_usage(result): return usage +def model_name_from_result(result) -> str | None: + """Best-effort `resolved_model` fallback for provider paths that + never emit `x-litellm-*` headers (the direct Anthropic/Bedrock SDK + paths have no gateway in front of them to set any). Reads + pydantic-ai's `ModelResponse.model_name` off the run's final + response — the provider's own answer to "what model actually + served this" — so the check-run "Backend" chip keeps working + without a gateway. Returns `None` on any shape mismatch (older + pydantic-ai, a test double, ...) so callers can chain it after + `resolved_model_from(captured)` without an extra guard.""" + try: + response = getattr(result, "response", None) + return getattr(response, "model_name", None) or None + except Exception: # noqa: BLE001 — diagnostic-only, never fatal + return None + + def usage_tokens(usage, *names: str) -> int: """First present, non-zero token counter from `usage`, trying `names` in order. pydantic-ai renamed `request_tokens`/`response_tokens` → diff --git a/src/cora/core/config.py b/src/cora/core/config.py index b9d5708..f63cc6a 100644 --- a/src/cora/core/config.py +++ b/src/cora/core/config.py @@ -56,6 +56,19 @@ # defaults are never exercised there. DEFAULT_LITELLM_BASE = "http://localhost:4000" DEFAULT_MODEL = "review" # LiteLLM alias + +# Which dialect `cora.core.agent.make_review_agent` speaks to the LLM. +# `"openai-compatible"` (the default) is the existing behaviour — +# talk OpenAI Chat Completions to whatever `llm_base_url` points at +# (a LiteLLM gateway, vLLM-direct, OpenRouter, ...). `"anthropic"` / +# `"bedrock"` are gateway-less direct-SDK paths for an adopter with +# only an Anthropic API key / AWS credentials — no proxy in between. +# Mirrored as `ReviewerConfig.llm_provider` (env `LLM_PROVIDER`); the +# default keeps every existing deployment byte-identical. +DEFAULT_LLM_PROVIDER = "openai-compatible" +SUPPORTED_LLM_PROVIDERS: frozenset[str] = frozenset( + {"openai-compatible", "anthropic", "bedrock"} +) DEFAULT_MCP_URL = "http://localhost:8080/mcp" # Cold-start pretrigger — which `model` aliases the warmup fires at (see diff --git a/src/cora/core/continuation.py b/src/cora/core/continuation.py index 4964776..87d0db1 100644 --- a/src/cora/core/continuation.py +++ b/src/cora/core/continuation.py @@ -40,7 +40,7 @@ _make_pydantic_ai_local_tools, ) from cora.core import config as _c -from cora.core.budget import resolve_run_usage, usage_tokens +from cora.core.budget import model_name_from_result, resolve_run_usage, usage_tokens from cora.core.mcp_probe import probe_mcp_server as _probe_mcp_server @@ -226,10 +226,10 @@ async def continue_on_t1( is unlikely if T0 succeeded, but still returns the distinct `mcp-connect-failed` reason for finalize-path consistency. """ - from pydantic_ai import ModelSettings, UsageLimits + from pydantic_ai import UsageLimits from pydantic_ai.exceptions import UsageLimitExceeded - from cora.core.agent import AgentConfig, Deps, make_review_agent + from cora.core.agent import AgentConfig, Deps, build_model_settings, make_review_agent from cora.core.loop_logging import ( PerCallTimeoutExceeded, WallTimeExceeded, @@ -278,6 +278,7 @@ async def continue_on_t1( api_key=llm_gateway_key, model_alias=t1_model_alias, system_prompt=system_prompt, + provider=cfg.llm_provider if cfg is not None else _c.DEFAULT_LLM_PROVIDER, mcp_servers=mcp_servers, mcp_allowed_tools=mcp_allowed_for_filter, local_tools=_make_pydantic_ai_local_tools( @@ -364,17 +365,17 @@ async def continue_on_t1( leadin_prompt, message_history=message_history_arg, deps=deps, - model_settings=ModelSettings( - # Same per-call cap as T0 (deep_review.py): fit the - # model's reasoning trace plus the turn's output. - # See `_c.DEEP_MAX_OUTPUT_TOKENS`. + # Same per-call cap as T0 (deep_review.py): fit the + # model's reasoning trace plus the turn's output. + # See `_c.DEEP_MAX_OUTPUT_TOKENS`. + model_settings=build_model_settings( + cfg, max_tokens=( cfg.deep_max_output_tokens if cfg is not None else _c.DEEP_MAX_OUTPUT_TOKENS ), - temperature=0.2, - timeout=timeout_s, + timeout_s=timeout_s, ), usage_limits=UsageLimits(request_limit=max_iterations), ) as agent_run: @@ -523,6 +524,7 @@ def _extend_t1_deadline(_seconds: float) -> None: budget.record_litellm_headers(captured) budget.set_resolved_model( resolved_model_from(captured) + or model_name_from_result(result) or f"unknown (T1 on {t1_model_alias}, no x-litellm-* headers)" ) diff --git a/src/cora/core/deep_review.py b/src/cora/core/deep_review.py index 20c7074..0c149e1 100644 --- a/src/cora/core/deep_review.py +++ b/src/cora/core/deep_review.py @@ -31,8 +31,10 @@ import time from typing import TYPE_CHECKING, Any, Callable +from cora.core import config as _c from cora.core.budget import ( T0_COLD_START_ALLOWANCE_S, + model_name_from_result, resolve_run_usage, usage_tokens, ) @@ -177,8 +179,6 @@ def _loaded_tool_names( "unused" denominator tied to the successfully opened server classes so it remains useful as a tool-loading signal. """ - from cora.core import config as _c - read_set = set(_c.READ_TOOLS if read_tools is None else read_tools) local_set = set( _c.LOCAL_REPO_TOOLS if local_repo_tools is None else local_repo_tools @@ -298,10 +298,10 @@ async def deep_review_call( falling back to the `agent-loop-errored` soft-fail. Flag-off and non-spiral errors follow the pre-feature path unchanged. """ - from pydantic_ai import ModelSettings, UsageLimits + from pydantic_ai import UsageLimits from pydantic_ai.exceptions import UnexpectedModelBehavior, UsageLimitExceeded - from cora.core.agent import AgentConfig, Deps, make_review_agent + from cora.core.agent import AgentConfig, Deps, build_model_settings, make_review_agent # Pre-probe the required MCP server. Failure here returns the # distinct `mcp-connect-failed` terminated_reason so the caller @@ -347,6 +347,7 @@ async def deep_review_call( api_key=llm_gateway_key, model_alias=model_alias, system_prompt=system_prompt, + provider=cfg.llm_provider if cfg is not None else _c.DEFAULT_LLM_PROVIDER, mcp_servers=mcp_servers, mcp_allowed_tools=mcp_allowed_for_filter, local_tools=_make_pydantic_ai_local_tools( @@ -413,8 +414,6 @@ async def deep_review_call( # Per-call output budget — must fit the model's `` trace AND the # turn's text/tool-call. See `_c.DEEP_MAX_OUTPUT_TOKENS`. - from cora.core import config as _c - deep_max_tokens = ( cfg.deep_max_output_tokens if cfg is not None else _c.DEEP_MAX_OUTPUT_TOKENS ) @@ -445,23 +444,23 @@ async def deep_review_call( async with agent.iter( initial_user_prompt, deps=deps, - model_settings=ModelSettings( - # Per-call output budget — must absorb the model's - # `...` reasoning block AND leave room for - # the turn's text / tool-call. Pydantic-AI raises - # `UnexpectedModelBehavior` when a turn finishes with - # `finish_reason='length'` and only thinking parts came - # back (`_agent_graph.py` L1104). This cap has climbed - # 8k → 16k → `DEEP_MAX_OUTPUT_TOKENS` as the review model's - # reasoning grew — the review reasoning model blew the whole - # 16k on turn 1 of a substantive PR. - # Config-threaded; T1 (continuation.py) uses the same cap. - # Fits the review chain's typical context windows - # (e.g. 80k T0, 256k T1). + # Per-call output budget — must absorb the model's + # `...` reasoning block AND leave room for + # the turn's text / tool-call. Pydantic-AI raises + # `UnexpectedModelBehavior` when a turn finishes with + # `finish_reason='length'` and only thinking parts came + # back (`_agent_graph.py` L1104). This cap has climbed + # 8k → 16k → `DEEP_MAX_OUTPUT_TOKENS` as the review model's + # reasoning grew — the review reasoning model blew the whole + # 16k on turn 1 of a substantive PR. + # Config-threaded; T1 (continuation.py) uses the same cap. + # Fits the review chain's typical context windows + # (e.g. 80k T0, 256k T1). + model_settings=build_model_settings( + cfg, max_tokens=deep_max_tokens, - temperature=0.2, - timeout=timeout_s, - **_thinking_extra_body( + timeout_s=timeout_s, + extra=_thinking_extra_body( cfg.enable_thinking if cfg is not None else None ), ), @@ -686,11 +685,11 @@ def _extend_t0_deadline(_seconds: float) -> None: leadin, message_history=spiral_messages, deps=deps, - model_settings=ModelSettings( + model_settings=build_model_settings( + cfg, max_tokens=cfg.spiral_recovery_max_output_tokens, - temperature=0.2, - timeout=timeout_s, - **_thinking_extra_body( + timeout_s=timeout_s, + extra=_thinking_extra_body( cfg.enable_thinking if cfg is not None else None ), ), @@ -742,7 +741,9 @@ def _extend_t0_deadline(_seconds: float) -> None: captured = drain_captured_headers() budget.record_litellm_headers(captured) budget.set_resolved_model( - resolved_model_from(captured) or "unknown (no x-litellm-* headers)" + resolved_model_from(captured) + or model_name_from_result(result) + or "unknown (no x-litellm-* headers)" ) body = result.output if isinstance(result.output, str) else str(result.output) diff --git a/src/cora/core/quick_review.py b/src/cora/core/quick_review.py index dabe67b..1f2ca93 100644 --- a/src/cora/core/quick_review.py +++ b/src/cora/core/quick_review.py @@ -18,7 +18,7 @@ from typing import TYPE_CHECKING from cora.core import config as _c -from cora.core.budget import resolve_run_usage, usage_tokens +from cora.core.budget import model_name_from_result, resolve_run_usage, usage_tokens if TYPE_CHECKING: from cora.config import ReviewerConfig @@ -46,11 +46,11 @@ async def quick_review_call( # — `("", "pydantic-ai unavailable: ...")` maps to a `cancelled` # check-run. try: - from pydantic_ai import ModelSettings + import pydantic_ai # noqa: F401 except ImportError as exc: return "", f"pydantic-ai unavailable: {exc}" - from cora.core.agent import AgentConfig, Deps, make_review_agent + from cora.core.agent import AgentConfig, Deps, build_model_settings, make_review_agent from cora.core.rate_limit import run_with_rate_limit_backoff agent_config = AgentConfig( @@ -58,6 +58,7 @@ async def quick_review_call( api_key=llm_gateway_key, model_alias=model_alias, system_prompt=system_prompt, + provider=cfg.llm_provider if cfg is not None else _c.DEFAULT_LLM_PROVIDER, # `retries=1` matches the existing leak-retry budget — no # tool-call retries because quick mode doesn't expose tools. retries=1, @@ -94,10 +95,8 @@ async def _call(): return await agent.run( initial_user_prompt, deps=deps, - model_settings=ModelSettings( - max_tokens=max_tokens, - temperature=0.2, - timeout=timeout_s, + model_settings=build_model_settings( + cfg, max_tokens=max_tokens, timeout_s=timeout_s ), ) @@ -171,7 +170,9 @@ async def _call(): captured = drain_captured_headers() budget.record_litellm_headers(captured) budget.set_resolved_model( - resolved_model_from(captured) or "unknown (no x-litellm-* headers)" + resolved_model_from(captured) + or model_name_from_result(result) + or "unknown (no x-litellm-* headers)" ) # `result.output` is a string when `output_type=str` (the factory @@ -199,9 +200,8 @@ async def _recover_quick( `spiral_recovery_max_output_tokens`. Returns the pydantic-ai result on success, or None if recovery raises (caller soft-fails). """ - from pydantic_ai import ModelSettings - from cora.core import spiral as _spiral + from cora.core.agent import build_model_settings leadin = _spiral.build_recovery_leadin( _spiral.extract_partial_reasoning( @@ -213,10 +213,10 @@ async def _recover_quick( leadin, message_history=msgs, deps=deps, - model_settings=ModelSettings( + model_settings=build_model_settings( + cfg, max_tokens=cfg.spiral_recovery_max_output_tokens, - temperature=0.2, - timeout=timeout_s, + timeout_s=timeout_s, ), ) except Exception as exc: # noqa: BLE001 — recovery is best-effort diff --git a/src/cora/review/_patches.py b/src/cora/review/_patches.py index 1b8a5dc..5f8bb7e 100644 --- a/src/cora/review/_patches.py +++ b/src/cora/review/_patches.py @@ -214,7 +214,7 @@ async def dispatch_patches(run: ReviewRun) -> None: # noqa: PLR0915 esc_t2_term, _esc_t2_tools, ) = await call_t2_alt_reviewer( - endpoint_base_url=f"{run.base_url.rstrip('/')}/v1", + endpoint_base_url=run.endpoint_base_url, llm_gateway_key=run.api_key, t2_model_alias=t2_model_alias, system_prompt=run.system_prompt, diff --git a/src/cora/review/_preflight.py b/src/cora/review/_preflight.py index e115afa..0a11c05 100644 --- a/src/cora/review/_preflight.py +++ b/src/cora/review/_preflight.py @@ -120,7 +120,10 @@ def preflight(run: ReviewRun) -> ReviewResult | None: else: print("::warning::could not derive PR head SHA; verdict check skipped") - if not run.api_key: + # Bedrock authenticates via the AWS credential chain, not an API + # key — the other providers (openai-compatible, anthropic) still + # require one. + if not run.api_key and cfg.llm_provider != "bedrock": print("::warning::LLM gateway key not set — skipping review") try: run.reporter.post_skip( diff --git a/src/cora/review/_state.py b/src/cora/review/_state.py index 562a869..df7ae47 100644 --- a/src/cora/review/_state.py +++ b/src/cora/review/_state.py @@ -172,3 +172,22 @@ def skip_result( def eval_dump(self, filename: str, content: str) -> None: _eval_dump(self.cfg, filename, content) + + @property + def endpoint_base_url(self) -> str: + """`base_url` shaped for the configured LLM provider. + + The OpenAI-compatible path (the default) always wants the + `/v1`-suffixed base the `openai` SDK expects — unchanged + behaviour. The direct Anthropic/Bedrock SDK paths use their own + default endpoints (api.anthropic.com / the AWS credential + chain's region) unless `llm_base_url` is explicitly pointed + elsewhere, so neither the `/v1` suffix nor the + `DEFAULT_LITELLM_BASE` localhost placeholder should be + forwarded as an override. See `cora.core.agent` for how the + provider branch consumes this.""" + if self.cfg.llm_provider == "openai-compatible": + return f"{self.base_url.rstrip('/')}/v1" + from cora.core import config as _c + + return self.base_url if self.base_url != _c.DEFAULT_LITELLM_BASE else "" diff --git a/src/cora/review/_tiers.py b/src/cora/review/_tiers.py index 2a0533f..c1d9652 100644 --- a/src/cora/review/_tiers.py +++ b/src/cora/review/_tiers.py @@ -192,7 +192,7 @@ async def dispatch_tiers(run: ReviewRun) -> ReviewResult | None: run.tiers_run.append(model) run.final_body, run.terminated_reason = await quick_review_call( - endpoint_base_url=f"{run.base_url.rstrip('/')}/v1", + endpoint_base_url=run.endpoint_base_url, llm_gateway_key=run.api_key, model_alias=model, system_prompt=run.system_prompt, @@ -269,7 +269,7 @@ async def dispatch_tiers(run: ReviewRun) -> ReviewResult | None: run.tools_available, t0_messages, ) = await deep_review_call( - endpoint_base_url=f"{run.base_url.rstrip('/')}/v1", + endpoint_base_url=run.endpoint_base_url, llm_gateway_key=run.api_key, model_alias=model, system_prompt=run.system_prompt, @@ -403,7 +403,7 @@ async def dispatch_tiers(run: ReviewRun) -> ReviewResult | None: tag=tag, terminated_reason=run.terminated_reason, extra={ - "endpoint_base_url": f"{run.base_url.rstrip('/')}/v1", + "endpoint_base_url": run.endpoint_base_url, "llm_gateway_key": run.api_key, "system_prompt": run.system_prompt, "budget": budget, @@ -544,7 +544,7 @@ async def second_opinion_dispatch(run: ReviewRun) -> None: return run.second_opinion_result = await run.second_opinion.dispatch( cfg=run.cfg, - endpoint_base_url=f"{run.base_url.rstrip('/')}/v1", + endpoint_base_url=run.endpoint_base_url, api_key=run.api_key, system_prompt=run.system_prompt, initial_user_prompt=run.initial_user_prompt, diff --git a/tests/test_provider_seam.py b/tests/test_provider_seam.py new file mode 100644 index 0000000..4191147 --- /dev/null +++ b/tests/test_provider_seam.py @@ -0,0 +1,390 @@ +"""Direct cloud-provider seam — provider selection, sampling-param +hygiene, `resolved_model` fallback, and lazy-import behaviour. + +Covers the `LLM_PROVIDER` / `ReviewerConfig.llm_provider` / +`AgentConfig.provider` seam added on top of the existing +OpenAI-compatible-only path (`cora.core.agent.make_review_agent`). +Every test here is offline — no network, no real Anthropic/AWS +credentials required. Tests that construct a real `AnthropicModel` / +`BedrockConverseModel` skip when the optional `anthropic` / `bedrock` +extras aren't installed, matching the existing +`pytest.importorskip("pydantic_ai.mcp")` style in `test_agent_factory.py`. +""" + +from __future__ import annotations + +import sys + +import pytest + +from cora.config import ReviewerConfig +from cora.core import config as _c +from cora.core.agent import AgentConfig, build_model_settings, make_review_agent +from cora.core.budget import model_name_from_result +from cora.providers.reporter import NullReporter +from cora.providers.retrieval import NullRetrievalProvider +from cora.review._preflight import build_run, preflight + + +pytest.importorskip("pydantic_ai") + + +# ── AgentConfig / make_review_agent provider selection ──────────────── + + +def test_agent_config_default_provider_is_openai_compatible(): + c = AgentConfig( + endpoint_base_url="http://litellm.test/v1", + api_key="dummy", + model_alias="review", + system_prompt="test", + ) + assert c.provider == "openai-compatible" + + +def test_make_review_agent_default_provider_builds_openai_chat_model(): + from pydantic_ai.models.openai import OpenAIChatModel + + c = AgentConfig( + endpoint_base_url="http://litellm.test/v1", + api_key="dummy", + model_alias="review", + system_prompt="test", + ) + agent = make_review_agent(c) + assert isinstance(agent.model, OpenAIChatModel) + + +def test_make_review_agent_unknown_provider_raises(): + c = AgentConfig( + endpoint_base_url="http://litellm.test/v1", + api_key="dummy", + model_alias="review", + system_prompt="test", + provider="bogus", + ) + with pytest.raises(ValueError, match="unknown AgentConfig.provider"): + make_review_agent(c) + + +def test_make_review_agent_anthropic_provider_builds_anthropic_model(): + pytest.importorskip("anthropic") + from pydantic_ai.models.anthropic import AnthropicModel + + c = AgentConfig( + # Unmodified localhost placeholder — must NOT be forwarded as + # a base_url override (see `_provider_base_url_override`). + endpoint_base_url=_c.DEFAULT_LITELLM_BASE, + api_key="sk-ant-test", + model_alias="claude-sonnet-5", + system_prompt="test", + provider="anthropic", + ) + agent = make_review_agent(c) + assert isinstance(agent.model, AnthropicModel) + assert agent.model._provider.base_url == "https://api.anthropic.com" # noqa: SLF001 + + +def test_make_review_agent_anthropic_respects_explicit_base_url_override(): + pytest.importorskip("anthropic") + + c = AgentConfig( + endpoint_base_url="https://anthropic-proxy.example.com", + api_key="sk-ant-test", + model_alias="claude-sonnet-5", + system_prompt="test", + provider="anthropic", + ) + agent = make_review_agent(c) + assert ( + agent.model._provider.base_url # noqa: SLF001 + == "https://anthropic-proxy.example.com" + ) + + +def test_make_review_agent_bedrock_provider_builds_bedrock_model(monkeypatch): + pytest.importorskip("boto3") + from pydantic_ai.models.bedrock import BedrockConverseModel + + monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1") + monkeypatch.setenv("AWS_ACCESS_KEY_ID", "testing") + monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "testing") + + c = AgentConfig( + endpoint_base_url=_c.DEFAULT_LITELLM_BASE, + api_key="", + model_alias="claude-opus-4-8", + system_prompt="test", + provider="bedrock", + ) + agent = make_review_agent(c) + assert isinstance(agent.model, BedrockConverseModel) + # Bare alias gets the `anthropic.` prefix Bedrock model IDs require. + assert agent.model.model_name == "anthropic.claude-opus-4-8" + + +def test_make_review_agent_bedrock_does_not_double_prefix(monkeypatch): + pytest.importorskip("boto3") + + monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1") + monkeypatch.setenv("AWS_ACCESS_KEY_ID", "testing") + monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "testing") + + c = AgentConfig( + endpoint_base_url=_c.DEFAULT_LITELLM_BASE, + api_key="", + model_alias="anthropic.claude-opus-4-8", + system_prompt="test", + provider="bedrock", + ) + agent = make_review_agent(c) + assert agent.model.model_name == "anthropic.claude-opus-4-8" + + +def test_anthropic_provider_missing_extra_raises_friendly_import_error(monkeypatch): + """Simulate the `anthropic` extra not being installed — `sys.modules` + entries set to `None` make the next `import` of that name raise + `ImportError`, regardless of what's actually installed in the test + environment. Guards the packaging contract: a helpful message + pointing at `cora[anthropic]`, not a raw traceback.""" + monkeypatch.setitem(sys.modules, "pydantic_ai.models.anthropic", None) + monkeypatch.setitem(sys.modules, "pydantic_ai.providers.anthropic", None) + + c = AgentConfig( + endpoint_base_url=_c.DEFAULT_LITELLM_BASE, + api_key="sk-ant-test", + model_alias="claude-sonnet-5", + system_prompt="test", + provider="anthropic", + ) + with pytest.raises(ImportError, match=r"cora\[anthropic\]"): + make_review_agent(c) + + +def test_bedrock_provider_missing_extra_raises_friendly_import_error(monkeypatch): + monkeypatch.setitem(sys.modules, "pydantic_ai.models.bedrock", None) + + c = AgentConfig( + endpoint_base_url=_c.DEFAULT_LITELLM_BASE, + api_key="", + model_alias="claude-opus-4-8", + system_prompt="test", + provider="bedrock", + ) + with pytest.raises(ImportError, match=r"cora\[bedrock\]"): + make_review_agent(c) + + +# ── build_model_settings: sampling-param + extra_body hygiene ───────── + + +def test_build_model_settings_openai_compatible_keeps_temperature_and_extra(): + settings = build_model_settings( + ReviewerConfig(llm_provider="openai-compatible"), + max_tokens=1234, + timeout_s=30, + extra={"extra_body": {"chat_template_kwargs": {"enable_thinking": True}}}, + ) + assert settings["max_tokens"] == 1234 + assert settings["temperature"] == 0.2 + assert settings["timeout"] == 30 + assert settings["extra_body"] == { + "chat_template_kwargs": {"enable_thinking": True} + } + + +def test_build_model_settings_none_cfg_defaults_to_openai_compatible(): + """A None cfg (legacy call shape some call sites still support) + must behave exactly like an explicit default-provider config — + the compatibility contract's byte-identical-default requirement.""" + settings = build_model_settings(None, max_tokens=999, timeout_s=10) + assert settings["temperature"] == 0.2 + assert settings["max_tokens"] == 999 + + +@pytest.mark.parametrize("provider", ["anthropic", "bedrock"]) +def test_build_model_settings_direct_providers_strip_sampling_and_extra_body(provider): + settings = build_model_settings( + ReviewerConfig(llm_provider=provider), + max_tokens=1234, + timeout_s=30, + extra={"extra_body": {"chat_template_kwargs": {"enable_thinking": True}}}, + ) + assert settings["max_tokens"] == 1234 + assert settings["timeout"] == 30 + assert "temperature" not in settings + assert "extra_body" not in settings + + +# ── ReviewerConfig.llm_provider / from_env wiring ────────────────────── + + +def test_reviewer_config_llm_provider_defaults_to_openai_compatible(): + assert ReviewerConfig().llm_provider == "openai-compatible" + + +def test_from_env_llm_provider_default_unset(): + cfg = ReviewerConfig.from_env({}) + assert cfg.llm_provider == "openai-compatible" + + +def test_from_env_anthropic_api_key_fallback(): + """`LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=…` with no + `LLM_GATEWAY_KEY` — the acceptance-criteria call shape — must + resolve `llm_api_key` from `ANTHROPIC_API_KEY`.""" + cfg = ReviewerConfig.from_env( + {"LLM_PROVIDER": "anthropic", "ANTHROPIC_API_KEY": "sk-ant-live"} + ) + assert cfg.llm_provider == "anthropic" + assert cfg.llm_api_key == "sk-ant-live" + + +def test_from_env_llm_gateway_key_wins_over_anthropic_api_key(): + cfg = ReviewerConfig.from_env( + { + "LLM_PROVIDER": "anthropic", + "LLM_GATEWAY_KEY": "gw-key", + "ANTHROPIC_API_KEY": "sk-ant-live", + } + ) + assert cfg.llm_api_key == "gw-key" + + +def test_from_env_bedrock_provider_no_api_key_required(): + cfg = ReviewerConfig.from_env({"LLM_PROVIDER": "bedrock"}) + assert cfg.llm_provider == "bedrock" + assert cfg.llm_api_key is None + + +def test_from_env_unknown_provider_raises(): + with pytest.raises(ValueError, match="unknown LLM_PROVIDER"): + ReviewerConfig.from_env({"LLM_PROVIDER": "not-a-real-provider"}) + + +# ── resolved_model fallback (budget.model_name_from_result) ──────────── + + +class _FakeResponse: + def __init__(self, model_name): + self.model_name = model_name + + +class _FakeResult: + def __init__(self, model_name): + self.response = _FakeResponse(model_name) + + +def test_model_name_from_result_reads_response_model_name(): + assert model_name_from_result(_FakeResult("claude-sonnet-5")) == "claude-sonnet-5" + + +def test_model_name_from_result_none_when_response_missing(): + class _NoResponse: + pass + + assert model_name_from_result(_NoResponse()) is None + + +def test_model_name_from_result_none_when_model_name_empty(): + assert model_name_from_result(_FakeResult("")) is None + + +def test_model_name_from_result_never_raises_on_garbage_input(): + assert model_name_from_result(object()) is None + assert model_name_from_result(None) is None + + +# ── ReviewRun.endpoint_base_url — provider-aware base URL shaping ────── + + +def _run(**cfg_overrides): + base = dict(repo="owner/repo", pr_number="1", llm_api_key="key") + base.update(cfg_overrides) + cfg = ReviewerConfig(**base) + return build_run( + cfg, + reporter=NullReporter(), + retrieval=NullRetrievalProvider(), + git=None, + second_opinion=None, + ) + + +def test_endpoint_base_url_openai_compatible_appends_v1(): + run = _run() # default provider, default llm_base_url + assert run.endpoint_base_url == f"{_c.DEFAULT_LITELLM_BASE}/v1" + + +def test_endpoint_base_url_openai_compatible_strips_trailing_slash_before_v1(): + run = _run(llm_base_url="http://litellm.test/") + assert run.endpoint_base_url == "http://litellm.test/v1" + + +def test_endpoint_base_url_anthropic_ignores_unmodified_default_placeholder(): + """No `LITELLM_BASE_URL` override — the localhost placeholder must + not leak through to the direct-SDK path (see `ReviewRun.endpoint_base_url` + and `agent._provider_base_url_override`, which both key off the + same `DEFAULT_LITELLM_BASE` sentinel).""" + run = _run(llm_provider="anthropic") + assert run.endpoint_base_url == "" + + +def test_endpoint_base_url_anthropic_respects_explicit_override(): + run = _run( + llm_provider="anthropic", + llm_base_url="https://anthropic-proxy.example.com", + ) + assert run.endpoint_base_url == "https://anthropic-proxy.example.com" + + +def test_endpoint_base_url_bedrock_ignores_unmodified_default_placeholder(): + run = _run(llm_provider="bedrock") + assert run.endpoint_base_url == "" + + +# ── preflight: bedrock doesn't require an API key ────────────────────── + + +def _preflight_skip_reasons(cfg, monkeypatch): + import cora.core.pr_context as prc_mod + + monkeypatch.setattr( + prc_mod, + "fetch_pr_metadata", + lambda pr: { + "title": "t", + "body": "b", + "labels": [], + "author": {"login": "alice", "is_bot": False}, + "baseRefName": "main", + "headRefName": "feat/x", + "isCrossRepository": False, + "additions": 1, + "deletions": 1, + "changedFiles": 1, + }, + ) + monkeypatch.setattr(prc_mod, "_pr_head_sha", lambda: None) + run = _run(**cfg) + result = preflight(run) + return result.terminated_reason if result is not None else None + + +def test_preflight_bedrock_does_not_require_api_key(monkeypatch): + reason = _preflight_skip_reasons( + {"llm_provider": "bedrock", "llm_api_key": None}, monkeypatch + ) + assert reason != "secret-missing" + + +def test_preflight_anthropic_still_requires_api_key(monkeypatch): + reason = _preflight_skip_reasons( + {"llm_provider": "anthropic", "llm_api_key": None}, monkeypatch + ) + assert reason == "secret-missing" + + +def test_preflight_openai_compatible_still_requires_api_key(monkeypatch): + """Unchanged default-path behaviour — the compatibility contract.""" + reason = _preflight_skip_reasons({"llm_api_key": None}, monkeypatch) + assert reason == "secret-missing" diff --git a/tests/test_quick_review.py b/tests/test_quick_review.py index 4d15df7..2ab4c37 100644 --- a/tests/test_quick_review.py +++ b/tests/test_quick_review.py @@ -62,3 +62,105 @@ async def run(self, prompt, *, deps, model_settings): "max_tokens": 12345, } assert budget.output_used == 2 + + +def test_quick_review_threads_llm_provider_into_agent_config(monkeypatch): + """`cfg.llm_provider` must reach `AgentConfig.provider` — this is + the one seam the direct Anthropic/Bedrock SDK paths hang off of.""" + captured_config = {} + + class FakeUsage: + request_tokens = 1 + response_tokens = 2 + total_tokens = 3 + + class FakeResult: + output = "🟢 looks good" + + def usage(self): + return FakeUsage() + + class FakeAgent: + async def run(self, prompt, *, deps, model_settings): + return FakeResult() + + def fake_make_review_agent(config): + captured_config["provider"] = config.provider + captured_config["endpoint_base_url"] = config.endpoint_base_url + return FakeAgent() + + monkeypatch.setattr( + "cora.core.agent.make_review_agent", fake_make_review_agent + ) + monkeypatch.setattr( + "cora.core.litellm_capture.drain_captured_headers", lambda: {} + ) + + budget = Budget(max_input=0, max_output=0, max_iterations=0) + asyncio.run( + quick_review_call( + endpoint_base_url="https://api.anthropic.com", + llm_gateway_key="sk-ant-test", + model_alias="claude-sonnet-5", + system_prompt="system", + initial_user_prompt="prompt", + budget=budget, + timeout_s=30, + pr_number="39", + cfg=ReviewerConfig(llm_provider="anthropic"), + ) + ) + + assert captured_config["provider"] == "anthropic" + assert captured_config["endpoint_base_url"] == "https://api.anthropic.com" + + +def test_quick_review_resolved_model_falls_back_to_response_model_name( + monkeypatch, +): + """No `x-litellm-*` headers (the direct-SDK provider paths never + have a gateway to emit them) — `Budget.resolved_model` should still + populate from the pydantic-ai response's own `model_name`.""" + + class FakeUsage: + request_tokens = 1 + response_tokens = 2 + total_tokens = 3 + + class FakeResponse: + model_name = "claude-sonnet-5" + + class FakeResult: + output = "🟢 looks good" + response = FakeResponse() + + def usage(self): + return FakeUsage() + + class FakeAgent: + async def run(self, prompt, *, deps, model_settings): + return FakeResult() + + monkeypatch.setattr( + "cora.core.agent.make_review_agent", lambda config: FakeAgent() + ) + monkeypatch.setattr( + "cora.core.litellm_capture.drain_captured_headers", lambda: {} + ) + + budget = Budget(max_input=0, max_output=0, max_iterations=0) + asyncio.run( + quick_review_call( + endpoint_base_url="https://api.anthropic.com", + llm_gateway_key="sk-ant-test", + model_alias="claude-sonnet-5", + system_prompt="system", + initial_user_prompt="prompt", + budget=budget, + timeout_s=30, + pr_number="39", + cfg=ReviewerConfig(llm_provider="anthropic"), + ) + ) + + assert budget.resolved_model == "claude-sonnet-5"