feat(cli): add FastRouter as a built-in provider#10207
Conversation
FastRouter (https://fastrouter.ai) is an OpenRouter-compatible AI gateway. The integration mirrors the existing `apertis` provider — a fully formed provider entry is injected into `ModelsDev.get()` from a public catalog (`https://go.fastrouter.ai/api/v1/models`), with the model fetcher living in `packages/opencode/src/kilocode/provider/fastrouter.ts` and the runtime loader registered through `kiloCustomLoaders`. No new SDK dependency: FastRouter reuses `@openrouter/ai-sdk-provider` since the wire format is identical. Highlights: - No `models.dev` involvement — `delete providers["fastrouter"]` runs unconditionally before injection so any stray upstream entry is always overwritten by the live FastRouter API response. - The `/models` endpoint is public, so the FastRouter provider and its catalog appear in the picker even before the user adds an API key. - API key resolution: `FASTROUTER_API_KEY` env var, `kilo auth fastrouter`, or `provider.fastrouter.options.apiKey` in `kilo.json` (matches OpenRouter). - TUI surfacing: `fastrouter` added to the Kilo `PROVIDER_PRIORITY` map so it shows under "Popular" alongside `openrouter`. - OpenRouter-compatible code paths in `transform.ts` extended to cover `model.providerID === "fastrouter"` (reasoning effort, prompt cache key). - Docs: new `ai-providers/fastrouter.md` page plus nav and gateways-index entries. Diff is intentionally minimal: 50 added / 2 changed lines across 5 shared files (each guarded by `kilocode_change` markers) plus 2 new files in kilocode-only directories that need no markers. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@johnnyeric @chrarnoldus @marius-kilocode made a new PR for the overall fastrouter integration as a provider as per the comments mentioned earlier |
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| } | ||
|
|
||
| if (fastrouterAllowed) { | ||
| const fr = yield* Effect.promise(() => ModelCache.fetch("fastrouter").catch(() => ({}))) |
There was a problem hiding this comment.
WARNING: FastRouter model fetch fires for all users, not just those who configure FastRouter
Unlike kilo (which requires a key before fetching) or apertis (which gates fetching on having an API key in fetchApertisModels), FastRouter's /models endpoint is public and is hit unconditionally on every ModelsDev.get() call for any user who hasn't explicitly added fastrouter to disabled_providers. This makes FastRouter an opt-out provider rather than opt-in — every Kilo user silently pings go.fastrouter.ai every 5 minutes.
The 5-minute TTL cache prevents hammering, but users who have never configured FastRouter and don't know about it will still generate background traffic. Consider gating the fetch on whether the user has FastRouter configured (key present) or at least documenting this opt-out behavior clearly in the code.
|
|
||
| type Models = { data?: Top[] } | ||
|
|
||
| export async function fetchFastRouterModels(): Promise<Record<string, any>> { |
There was a problem hiding this comment.
SUGGESTION: Consider tightening the return type from Record<string, any> to Record<string, unknown> to avoid silently accepting arbitrary values. The apertis fetcher uses the same loose type, but any bypasses TypeScript's type checking on every property access downstream.
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Other Observations (not in diff)Integration correctness: The implementation correctly reuses
Memory: No memory leak risk. Docs page: The Files Reviewed (9 files)
Fix these issues in Kilo Cloud Reviewed by claude-4.6-sonnet-20260217 · 2,353,556 tokens |
Addresses the kilo-code-bot warning on Kilo-Org#10207. The previous catalog injection ran for every install regardless of whether a FastRouter API key was configured, which meant every Kilo CLI start spawned an HTTP request to go.fastrouter.ai with no user intent behind it. Now the inject + ModelCache.fetch + background refresh only fire when a key resolves from one of the three documented sources: kilo.json provider.fastrouter.options.apiKey, the auth store (kilo auth login), or the FASTROUTER_API_KEY env var. Matches how kilo and apertis gate their fetches inline in models.ts. The runtime path for users with a key is unchanged. Also addresses the bot's typing suggestion: fetchFastRouterModels now returns Record<string, FastRouterModel> with a locally-defined shape type so future field-name drift is caught at compile time. A local type keeps us free of the models.ts -> model-cache.ts -> fastrouter.ts cycle a shared ModelsDev.Model import would create. Docs page updated to drop the "public catalog visible without a key" claim that no longer matches the runtime behavior. Co-authored-by: Cursor <cursoragent@cursor.com>
…ilocode into fastrouter_re_work
these changes have been made, please check again |
The previous link pointed at https://go.fastrouter.ai/ which is the API host (only /api/v1/... routes resolve). Lychee link-check on CI flagged the root path as 404. Sign-up actually lives on the marketing site at https://fastrouter.ai/, so point the docs there. Co-authored-by: Cursor <cursoragent@cursor.com>
Adds the two FastRouter URLs (https://fastrouter.ai and https://go.fastrouter.ai/api/v1) referenced from packages/opencode/src/kilocode/provider/fastrouter.ts. Fixes the extract-source-links --check CI step. Generated by `bun run script/extract-source-links.ts`. Co-authored-by: Cursor <cursoragent@cursor.com>
…ilocode into fastrouter_re_work
|
@markijbema @johnnyeric can we run the workflows again, fixed the issues with the two failing workflows |
`https://go.fastrouter.ai/api/v1` is the FastRouter inference base URL referenced by the `FASTROUTER_API` constant in source code. lychee extracts it via source-links.md and a plain GET against the bare base returns 404 (only `/v1/models`, `/v1/chat/completions`, etc. are real paths). Mirror the existing apertis exclusion right above it. Verified locally with lychee v0.23.0 (the exact version used in CI): 0 errors across packages/kilo-docs/source-links.md and packages/kilo-docs/pages. Co-authored-by: Cursor <cursoragent@cursor.com>
…ilocode into fastrouter_re_work
|
@markijbema @johnnyeric the lychee fix should be done now and i ran the other tests in my system, things were working |
|
@jatingomnet we submitted the contact form on your site to discuss this in more detail. Thank you! |
2 participants
Adds FastRouter (https://fastrouter.ai) as a built-in provider. FastRouter is
an OpenRouter-compatible gateway that fronts roughly 170 models from
Anthropic, OpenAI, Google, Mistral and others behind one API. Since the wire
format matches OpenRouter, this reuses
@openrouter/ai-sdk-provideranddoesn't pull in a new SDK package.
This is a redo of #7969 along the lines @johnnyeric suggested. I rewrote
the integration to mirror the existing
apertisprovider exactly ratherthan the more invasive
models.devpath I took the first time. One commit,no new dependencies, ~80 lines of real logic.
Maintainers
We'll watch issues filed against the FastRouter integration and loop in the
FastRouter team (
support@fastrouter.ai) if anything needs them on thebackend side.
Companion Kilo Gateway PR
Kilo-Org/cloud#3226 registers
FastRouter as a known upstream in the gateway. It's a small symbolic add
(
'fastrouter'in theProviderIdunion and aPROVIDERS.FASTROUTERentry)so anything that later sets
gateway: 'fastrouter'resolves cleanly insteadof crashing in
OpenRouterInferenceProviderIdSchema.parse.This kilocode PR doesn't depend on the gateway PR. FastRouter here is a
direct user-API-key integration with no gateway routing. The gateway PR is
the prerequisite for the eventual kilo-exclusive-models story.
What changed
The integration is the apertis pattern, point for point.
packages/opencode/src/kilocode/provider/fastrouter.tsis the new helper:it hits the public
/modelsendpoint, walks the response, and maps eachmodel into Kilo's
Modelshape. I usedglobalThis.fetch(to dodge thenamespace-shadow trap from last time), a 10s timeout, and
||instead of??for the numeric/string defaults so that0,null, and""all fallthrough to sensible values.
packages/kilo-docs/pages/ai-providers/fastrouter.mdis the new docs page,nav and gateways-index entries follow the same shape as the OpenRouter
docs.
The shared files (
model-cache.ts,models.ts,transform.ts) get a fewkilocode_change-guarded additions: afastrouterbranch infetchModels,the inject+refresh logic in
ModelsDev.get, and two more provider IDs addedto the OpenRouter-compatible conditions for
prompt_cache_keyandsmallOptions. The kilocode-only files (provider.ts,dialog-provider.tsx) get the loader registration and a priority entry soFastRouter shows up under Popular in the TUI.
Total: 10 files, +254/-2. The FastRouter icon already lives in the
upstream sprite so no asset work was needed.
The thing I cut from PR1: no
models.devinjection, no background refreshthread. Both were the root cause of the stale-data +
ProviderModelNotFoundErrorissues the bot flagged. Apertis works fine without them and so does this.
How I tested
CI checks (typecheck, knip, check-kilocode-change) pass locally on a clean
branch.
bun test test/provider/passes 288/289; the one failure(
plugin config enabled and disabled providers are honored) reproduces onplain
mainand isn't related.End-to-end:
bun run dev models fastrouterreturns the full catalog (works withoutan API key too, since
/modelsis public).fastrouter/anthropic/claude-haiku-4.5streamscorrectly and the token/cost numbers populate in the footer.
readtool. The full tool-useround trip works and both legs are visible in the FastRouter dashboard.
kilo auth login→ fastrouter, then restart withFASTROUTER_API_KEYunset. Chat still works, so the auth-store read path is exercised.
broken anything on the shared OpenRouter-shaped code paths.
I didn't add a unit test for the catalog fetcher. Apertis doesn't have one
and the previous review didn't ask, but it's a 20-line addition if you'd
like one in a follow-up.
On the plugin path
You raised plugins as the recommended alternative in #7969. I thought about
it. The reasons I went back in-tree:
precedent in the broader ecosystem for treating a popular gateway this
way.
hit a discovery gap.
If you'd still prefer this as a plugin, I have a scaffold ready and am
happy to close this PR.
Discord
The team is following up and will join soon
One callout for reviewers
The unconditional
delete providers["fastrouter"]inmodels.tsisdeliberate. It guarantees that if
models.devever ships afastrouterentry, the live FastRouter API response always wins. Behavior on a
/modelsoutage: the provider is still injected with an empty model map and
autoload: false, so it stays visible in the picker but selecting it failsuntil the next refresh fires
(automatically triggered when the catalog is
empty).
Screenshots for local run with the changes