Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,21 @@ versions (e.g. `0.0.0.dev60+g257737d`).

## [Unreleased]

### Added
- Direct cloud-provider seam — `LLM_PROVIDER` (`ReviewerConfig.llm_provider`,
default `openai-compatible`, byte-identical to today's behaviour) selects
`anthropic` or `bedrock` to run cora with no LiteLLM/OpenAI-compatible
gateway in between: an Anthropic API key (`ANTHROPIC_API_KEY`, or
`LLM_GATEWAY_KEY` which takes precedence) or AWS credentials (Bedrock's
standard credential chain) are all that's needed. Both paths drop
non-default sampling parameters and the vLLM `enable_thinking` extra_body
toggle (current Claude models reject/no-op them) and fall back to the
provider's own response `model` field for the check-run "Backend" chip
when no `x-litellm-*` headers are present. New optional extras
`cora[anthropic]` / `cora[bedrock]`; both imports are lazy, so the default
path never requires them installed. See `docs/configuration.md` →
"Running against a cloud provider".

## [0.1.0] - 2026-07-03

First public release. cora's development history predates this
Expand Down
67 changes: 64 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ below are the supported adopter-facing knobs.
| `LLM_GATEWAY_KEY` | API key for the OpenAI-compatible endpoint |
| `LITELLM_BASE_URL` | OpenAI-compatible base URL |
| `REVIEW_MODEL` | Model alias sent to the endpoint |
| `LLM_PROVIDER` | `openai-compatible` (default) / `anthropic` / `bedrock` — see below |

`LITELLM_BASE_URL` is named for the LiteLLM gateway convention, but the
client expects any OpenAI-compatible API — it can point at LiteLLM, vLLM,
or another compatible gateway. See
[Running against a cloud provider](#running-against-a-cloud-provider)
for the Anthropic/Bedrock topology.
for the Anthropic/Bedrock topology, including the gateway-less
`LLM_PROVIDER=anthropic`/`bedrock` direct-SDK paths.

## Review Mode And Budgets

Expand Down Expand Up @@ -135,6 +137,13 @@ behind review setup:

## Running against a cloud provider

cora talks to a cloud model one of two ways: through an OpenAI-compatible
gateway (the default, works today with zero code changes), or directly
against the Anthropic API / Amazon Bedrock via `LLM_PROVIDER` — no proxy
in between.

### Via a gateway (default `LLM_PROVIDER=openai-compatible`)

cora speaks one dialect — OpenAI Chat Completions — to whatever
`LITELLM_BASE_URL` points at. Cloud models work today by letting the
gateway do the translation: run a [LiteLLM proxy](https://docs.litellm.ai/)
Expand Down Expand Up @@ -189,5 +198,57 @@ Notes for cloud deployments:
caching helps: cora's agent loop has a frozen system prompt and an
append-only history, so multi-turn prefixes are highly cacheable.

Direct provider SDK support (Anthropic / Bedrock without a gateway) is
planned as a provider seam in the agent factory — see the issue tracker.
### Direct — no gateway (`LLM_PROVIDER=anthropic` / `bedrock`)

An adopter with only an Anthropic API key (or AWS credentials for
Bedrock) can skip the gateway entirely. `LLM_PROVIDER` selects the
dialect `make_review_agent` speaks; the default `openai-compatible`
value keeps every existing deployment byte-identical — this is
opt-in.

**Anthropic:**

```bash
export LLM_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-ant-... # or LLM_GATEWAY_KEY, which wins if both are set
export REVIEW_MODEL=claude-sonnet-5
export T1_MODEL=claude-opus-4-8 # optional — T1 continuation tier
```

Requires the `cora[anthropic]` extra (`pip install 'cora[anthropic]'`);
the import is lazy, so installing the base package alone never pulls in
the `anthropic` SDK. `LITELLM_BASE_URL` is ignored on this path unless
explicitly pointed somewhere other than its `localhost:4000` default —
set it only when routing through an Anthropic-compatible proxy that
isn't LiteLLM.

**Bedrock:**

```bash
export LLM_PROVIDER=bedrock
export AWS_REGION=us-east-1 # standard AWS credential chain — no LLM_GATEWAY_KEY needed
export REVIEW_MODEL=claude-opus-4-8 # the `anthropic.` Bedrock prefix is added automatically
```

Requires the `cora[bedrock]` extra (`pip install 'cora[bedrock]'`,
which pulls in `boto3`). Authenticates via the standard AWS credential
chain (env vars, shared profile, instance role, ...) — no API key
field is read or required.

**Both direct paths, by design:**

- **Sampling parameters are dropped.** cora's engine tunes
`temperature=0.2` for the gateway path; current Claude models reject
non-default `temperature`/`top_p`/`top_k` with a 400, so the direct
paths omit them entirely rather than forwarding a value that would
break every call.
- **The vLLM `enable_thinking` toggle never fires.** `AGENT_REVIEW_ENABLE_THINKING`
is a self-hosted-reasoning-backend knob (`chat_template_kwargs` is a
vLLM extra_body shape); cloud Claude models use their own adaptive-
thinking defaults instead, so it's a no-op here rather than an error.
- **The check-run "Backend" chip still works** — with no `x-litellm-*`
headers to read (there's no gateway to emit them), it falls back to
the provider response's own `model` field.
- **Deep mode is unchanged** — the agent loop, MCP attachments, and
local `grep_repo`/`git_show` tools all work identically; only the
model-construction seam differs.
14 changes: 14 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,20 @@ otel = [
"opentelemetry-sdk==1.42.1",
"opentelemetry-exporter-otlp-proto-grpc==1.42.1",
]
# Direct-SDK provider seam (`LLM_PROVIDER=anthropic` /
# `ReviewerConfig.llm_provider="anthropic"`) — gateway-less, straight
# to the Anthropic API. Only pulled in when an adopter opts in; the
# default `"openai-compatible"` path never imports the `anthropic` SDK
# (`cora.core.agent` imports it lazily inside the provider branch).
anthropic = [
"pydantic-ai-slim[anthropic]>=1.102",
]
# Direct-SDK provider seam for Amazon Bedrock (`LLM_PROVIDER=bedrock`)
# — AWS credential chain, no gateway. Same lazy-import discipline as
# the `anthropic` extra above.
bedrock = [
"pydantic-ai-slim[bedrock]>=1.102",
]
dev = [
"pytest",
"ruff",
Expand Down
27 changes: 26 additions & 1 deletion src/cora/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ class ReviewerConfig:
llm_base_url: str = _c.DEFAULT_LITELLM_BASE
llm_api_key: str | None = None
model: str = _c.DEFAULT_MODEL
# Which dialect `make_review_agent` speaks. Default
# `"openai-compatible"` is byte-identical to every existing
# deployment; `"anthropic"` / `"bedrock"` are gateway-less direct-SDK
# paths (see `core.config.SUPPORTED_LLM_PROVIDERS`). `llm_base_url`
# is ignored on those paths unless explicitly pointed away from the
# `DEFAULT_LITELLM_BASE` placeholder (see `cora.core.agent`).
llm_provider: str = _c.DEFAULT_LLM_PROVIDER

# ── MCP attachments (optional; deep mode) ────────────────────────
mcp_url: str = _c.DEFAULT_MCP_URL
Expand Down Expand Up @@ -286,11 +293,29 @@ def getalias(name: str, default: str) -> str:
except ValueError:
t1_max_iterations = _c.DEFAULT_T1_MAX_ITERATIONS

# Direct-SDK provider seam. An unknown value fails loudly (like
# the escalation-triggers CSV below) rather than silently
# falling back to the gateway path.
llm_provider = getalias("LLM_PROVIDER", _c.DEFAULT_LLM_PROVIDER)
if llm_provider not in _c.SUPPORTED_LLM_PROVIDERS:
raise ValueError(
f"unknown LLM_PROVIDER: {llm_provider!r} "
f"(supported: {sorted(_c.SUPPORTED_LLM_PROVIDERS)})"
)
# `llm_api_key` also accepts `ANTHROPIC_API_KEY` on the direct
# Anthropic path so `LLM_PROVIDER=anthropic ANTHROPIC_API_KEY=…`
# runs with no gateway secret at all. Bedrock needs no API key
# (AWS credential chain) so it isn't threaded here.
llm_api_key = get("LLM_GATEWAY_KEY")
if llm_api_key is None and llm_provider == "anthropic":
llm_api_key = get("ANTHROPIC_API_KEY")

cfg = cls(
repo=get("GH_REPO") or get("GITHUB_REPOSITORY") or "",
pr_number=get("PR_NUMBER") or "",
llm_base_url=get("LITELLM_BASE_URL", _c.DEFAULT_LITELLM_BASE),
llm_api_key=get("LLM_GATEWAY_KEY"),
llm_api_key=llm_api_key,
llm_provider=llm_provider,
model=get("REVIEW_MODEL", _c.DEFAULT_MODEL),
mcp_url=get("MCP_URL", _c.DEFAULT_MCP_URL),
mcp_token=get("MCP_TOKEN"),
Expand Down
Loading
Loading