chore(fork): update from upstream - #21
Merged
Merged
Conversation
* test(ui): make the agents route's tests markup-agnostic before migration Rewrites the two assertions that were coupled to antd's DOM and adds the missing characterisation test for agent_cost_view, so the suite describes behaviour rather than antd markup and can stay untouched across the shadcn migration. The skill selection test reached the checkbox with a querySelector on input[type=checkbox]; antd renders an input while Base UI renders a span[role=checkbox], so it now queries by role and accessible name, which both libraries derive from the wrapping label. The delete confirmation test queried role=dialog; antd Modal is a dialog while Base UI AlertDialog is an alertdialog, so it now anchors on the confirmation text and accepts either role. agent_cost_view had no test at all; it gets one covering the null render, the dollar-prefixed values, the omitted rows, and a zero cost that must not be mistaken for unset. All 55 tests pass against the current antd components. * refactor(ui): migrate agents to shadcn Replaces antd and Tremor with shadcn (base-vega) primitives across the five files the agents route exclusively owns. Markup only; no behaviour, data fetching or route structure changes. Modal becomes AlertDialog, with a plain destructive Button in the footer rather than AlertDialogAction, because that action is AlertDialog.Close and would dismiss the dialog before the delete request settles, losing the in-flight state. Alert, Tag, Spin, Space, Collapse, Descriptions, Typography and the antd icons map onto alert, badge, ui-loading-spinner, flex/grid utilities, collapsible, a definition list, semantic headings and lucide. The shadcn CLI emits alert.tsx importing cva from class-variance-authority, which this project does not depend on; it uses the cva object syntax from lib/cva.config. The generated file fails to typecheck, so the adapted copy lives in components/shared instead, per the convention that ui/ stays CLI-managed. Colour comes from tokens throughout, so the info callout is now the neutral card style rather than antd's blue, and nothing hardcodes a colour in the way of a later theme change. The 55 tests in the route pass unchanged from the previous commit. The visual gate re-baselined agents and all 34 other routes stayed pixel-identical.
…dels + Endpoints (BerriAI#34435) * refactor(ui): extract shared tab-routing helpers Every per-tab-routed page copy-pastes the same URL<->slug logic and the same active-tab/redirect engine. Extract two reusable pieces: - createTabRoutes(baseSegment, slugs) in utils/tabRoutes.ts returns { baseSegment, slugs, tabHref, slugFromPathname }, the trailing-slash href builder (via migratedHref) and the pathname->slug reader. - useTabRouting({ routes, baseTabKey, visibleKeys, ready }) derives the active tab from the pathname, redirects an unknown/forbidden slug to base once ready, and returns an onTabChange navigator. visibleKeys + ready exist so a role-gated page can pass its filtered tab set and defer the redirect until permissions resolve, rather than bouncing a user off a still-loading valid tab. Both are pure/unit-tested. No page consumes them yet. * refactor(ui): migrate Models + Endpoints onto the shared tab-routing helpers Replace the page's hand-rolled tabRoutes.ts (base segment + slug tuple + href builder + slugFromPathname) with createTabRoutes, keeping the existing named exports as thin re-exports so callers and tests are unchanged. The layout drops its local activeSlug/isKnownSlug/activeKey derivation, its redirect useEffect and its router.push onChange in favor of useTabRouting, passing the role-filtered visibleKeys and a ready flag (!teamsLoading && !uiSettingsLoading) so the permission-gated redirect behavior is preserved exactly. The antd tab bar, role-gated tab set, the refresh button and the ?model=/?team= drill-in overlay are untouched; the file's pre-existing antd import is now recorded in the suppressions baseline since editing it makes it a linted-as-changed file. The existing models-and-endpoints layout.test.tsx and tabRoutes.test.ts pass unchanged, which is the regression guarantee.
…ponses Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…param with key hash (BerriAI#34417) * feat(proxy): add overwrite_user_with_key_hash to stamp outgoing user param with key hash Adds a litellm_settings flag that forces the outgoing user param to the authenticated key's hashed token before the request is forwarded to the provider. The value overrides any caller-supplied user, so providers see a stable, tamper-proof identifier they can rate-limit or ban on, and the hash matches user_api_key_hash in spend logs for easy mapping back to the key owner. Off by default * fix(proxy): hash non-sk credentials before stamping user param UserAPIKeyAuth only hashes sk-prefixed keys and JWTs; custom-auth credentials stay raw on api_key, so stamping them directly would forward auth material to the provider. Pass through the two known hashed forms (sha256 hex, hashed-jwt-*) and hash anything else * refactor(proxy): stamp only standard virtual keys, skip jwt and custom auth A hashed JWT rotates on every token re-issue so it is useless as a stable ban id, and custom-auth credentials arrive raw on api_key. Instead of hashing whatever we hold, the stamp now applies only when api_key is the sha256 hex digest of a standard virtual key; other auth methods are explicitly out of scope until the stamped identifier is configurable * fix(proxy): gate user stamping on server-set virtual key provenance Shape alone cannot distinguish a key hash from a raw custom-auth credential that happens to be 64 hex chars. Adds via_virtual_key, a server-only marker on UserAPIKeyAuth following the mcp_admitted_user_subject pattern: stripped from all validated input so handlers and claims cannot forge it, set by post-construction assignment only at the DB virtual-key auth return. Stamping now requires the marker and the hash shape * test(proxy): prove db auth path sets via_virtual_key marker The stamping unit tests set the marker manually, so deleting the assignment in _user_api_key_auth_builder would pass every existing test; this exercises the real builder path with a mocked identity store and fails if the marker is not set * fix(proxy): stamp master-key requests with the master key alias Master-key auth substitutes LITELLM_PROXY_MASTER_KEY_ALIAS for api_key so the key and its hash never propagate; that made master-key traffic bypass stamping and pass the caller-supplied user through. The master path now sets via_virtual_key and the stamp gate accepts the alias alongside the sha256 shape, so admin traffic gets the same tamper-proof id that spend logs already record for it * fix(proxy): restore via_virtual_key marker on key-cache hits Cached PROXY_ADMIN auth objects early-return before the marked DB and master-key returns, and cache serialization drops the exclude=True marker, so cached admin traffic bypassed stamping. Key-cache entries are written only after the proxy validated a virtual key or the master key, so the cache-hit boundary restores the marker; the UI-login JWT fallback constructs its token from a decrypted blob, not this cache, and stays unmarked
…erriAI#31743) * fix(spend): resolve spend logs by request_id across all dates (LIT-3981) The /spend/logs/ui search only filtered the page already loaded, so a log id copied from another page or from outside the active date window could not be found. request_id is the primary key of LiteLLM_SpendLogs, so when it is supplied on the internal UI route the mandatory date window is dropped and the lookup resolves across all time. The date window stays required when no request_id is given, and the public /spend/logs/v2 contract is unchanged. A non-admin id lookup is gated by the same ownership check the detail endpoint uses, so the relaxed window cannot be used to read another tenant's log by id * fix(ui): send the logs request_id search to the server (LIT-3981) The "Search by Request ID" box filtered only the rows already on the current page, so an id from another page never matched. It now feeds the existing server-side request_id filter via handleFilterChange, which debounces, resets to page one, and rides the existing react-query key. The dead client-side filter and its searchTerm state are removed; the session composition and dedup logic is unchanged. The box is now an exact request_id lookup, matching its label; the incidental client-side model and user substring matching it used to do is dropped in favor of the dedicated filters * refactor(spend): model the request_id spend-log lookup as an explicit point lookup (LIT-3981) The date-window relaxation for request_id lookups rode an apply_date_window flag threaded through the date validation and parsing. Model the two intents directly instead. A UI request_id query is a point lookup on the @id primary key that drops the time window and authorizes by row ownership; every other query, including the public /spend/logs/v2 route, takes the range-scan path that still requires a window Because the ownership check fully authorizes the single row, the general user/team scoping is now skipped for id lookups rather than layered on top redundantly. The confusing `is_v2 or request_id is None` guard is gone, and moving the date requirement into the range-scan branch lets the type checker narrow the dates it parses Behavior is preserved: the v2 contract still requires dates even when a request_id is supplied, and a non-owner is still rejected with 403. A regression test covers the non-admin owner id lookup, which resolves across all time and filters by the primary key alone
…itellm_/migrate-page-memory-9b2c09
…itellm_/migrate-page-memory-9b2c09
…d_budget_enforcement (BerriAI#34429) * fix(proxy): reject request when budget reservation write fails under fail_closed_budget_enforcement With general_settings.fail_closed_budget_enforcement set to true, the read-time spend check already returns 503 when spend cannot be verified, but the atomic pre-call reservation still failed open: reserve_budget_for_request swallowed _CounterReservationUnavailable per counter and degraded to read-time-only enforcement, so concurrent requests could all pass the same under-budget read during a Redis outage and overspend past the configured budget. Now the strict flag is threaded into reserve_budget_for_request and a failed reservation write raises 503, releasing any counters that already reserved. Default behavior with the flag absent or false is unchanged. Fixes BerriAI#33923 * fix(proxy): pass 503 budget-enforcement detail as plain string
Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The key edit form seeded mcp_servers_and_groups from the key with only servers and accessGroups, but handleKeyUpdate writes mcp_toolsets from that same value, so every save posted an empty list and the backend merge applied it literally. A key granted a toolset lost the grant on any edit, including a budget change, and then got a 403 from /toolset/<name>/mcp Read toolsets in both places the form initializes from keyData, declare mcp_toolsets on KeyResponse.object_permission so a write-without-read is a type error, and carry toolsets through the create flow, which only looked at servers and accessGroups
The generated schema declares object_permission.mcp_toolsets as string[] | null; the handwritten KeyResponse shape omitted the null. ObjectPermissionsView consumes the same value, so its prop type widens with it
…eation billing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Responses API (/v1/responses) usage transform rebuilt prompt token details and dropped OpenAI's input_tokens_details.cache_write_tokens, so gpt-5.6 cache-creation tokens were never logged or billed via that route. Map it in the transform, and make PromptTokensDetailsWrapper keep cache_write_tokens and cache_creation_tokens in sync on assignment. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…es API logs On the /v1/responses path the response usage is not chat-Usage-shaped, so additional_usage_values could not derive cache tokens from response_obj.usage and the Admin UI Logs cache-creation token row stayed empty. Fall back to the normalized standard_logging usage_object's prompt_tokens_details for both the cache-read and cache-creation counts. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…itemization (BerriAI#34309) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…_usage_objects combine_usage_objects iterates prompt_tokens_details model_fields and sums each; with cache_write_tokens and cache_creation_tokens now mirroring each other via __setattr__, the pair was summed twice, doubling cache creation counts for Anthropic batch cost calc, mid-stream fallback usage merges, and realtime usage. Collapse the mirrored pair to one representative before summing.
…ite_tokens fix(cost_tracking): map OpenAI cache_write_tokens for prompt cache creation billing
…iAI#34457) * fix(proxy): restore atomic user upsert when adding team members Parallel /team/new calls naming the same not-yet-existing member were returning 500 "Unique constraint failed on the fields: (`user_id`)". The upsert in add_new_member passed an empty update branch. Prisma only compiles an upsert down to a single INSERT ... ON CONFLICT when that branch writes something; with an empty one it emits SELECT-then-INSERT instead, so concurrent requests all read "no such user" and all insert. Postgres statement logs confirm it: the empty form logs BEGIN/SELECT/INSERT/COMMIT, the non-empty form logs INSERT ... ON CONFLICT ("user_id") DO UPDATE SET. Re-state user_id in the update branch as a no-op so the native upsert path comes back. The teams append stays in the filtered update below it, so an already-existing member still cannot pick up a duplicate team id. tests/test_team.py::test_team_new failed 9 of 15 runs against a live proxy before this and 0 of 15 after. The existing unit test asserted only that upsert had been called on a mock, so it passed either way; it now pins the shape of both branches and fails when the update branch goes back to empty. * test: point the live codex tests at gpt-5.3-codex OpenAI deprecated gpt-5.2-codex, so test_openai_codex and test_openai_codex_stream started failing against the live API with model_not_found. gpt-5.3-codex is the current codex model; both tests pass on it. The remaining gpt-5.2-codex references in the suite are mocked transformation tests and are unaffected. * test(e2e): update models page specs for the shared DataTable The DataTable migration in BerriAI#34363 changed three things the models page specs were pinned to, and five tests went red. Row click no longer opens the detail view; the Model ID cell owns that now, so both specs click its `model-id-<id>` test id instead of the row. The search box placeholder switched from an ASCII "..." to a real ellipsis, so the specs use getByPlaceholder with a substring instead of an exact attribute match that punctuation can break again. The results count moved from `models-results-count` ("Showing 1 - 50 of 137 results") to the shared pagination's `pagination-range` ("Showing 1-50 of 137"). The Team-BYOK test also filtered rows on the team alias, which the Team ID column has never rendered in either the old or the new table; it filters on the team id now, which is what the column actually shows and what the assertion's own comment intends. Verified against a local proxy serving a fresh build with the seeded e2e postgres and mock upstream: all five failing tests pass, and the full suite is 82 passed / 4 skipped at CI parity (workers=1).
…ter served a request The dashboard already receives the requested model name as model_group on every spend-log row, but LogEntry dropped the field, so nothing distinguished an auto-routed request from a direct one. Surface it precisely rather than by comparing requested against resolved: model_group differs from model for plain aliases and wildcard deployments too, so a bare mismatch tags almost every row and identifies nothing. The indication is driven instead by which deployments are auto-routers, resolved from every page of /v2/model/info and shared through context. The request drawer header names the router in a badge next to the provider; the session sidebar swaps the entry's leading icon. Rows that no auto-router served render exactly as before.
…eader_layout fix(ui): keep cache leakage time range picker inline at narrow widths
…ter_logs feat(ui): show in the log drawer and session sidebar when an auto-router served a request
…ration Adds a role/text-based characterisation test for UIThemeSettings, which had none, and extends the skills panel test to cover the delete confirmation. Both are green against the current antd/Tremor components so they can prove the shadcn migration keeps behaviour identical without being edited.
…ts from antd markup Prepares the shadcn migration of these three routes by removing every assertion that depends on the current component library, so the same tests can gate the migration without being edited. FiltersButton and its OrganizationFilters consumer both asserted on the ".ant-badge" wrapper class; they now assert the active-filter indicator element itself, and FiltersButton additionally asserts that it is absent when there are no active filters. TestVectorStoreTab drove the antd Select with fireEvent.mouseDown and picked options by node; it now clicks through the combobox role and the option text, which works against any listbox implementation. The vector-stores index test relied on Tremor mounting every TabPanel at once, so it read the Manage tab's table without ever opening that tab. It now clicks the tab first, which is what a user does and what any tabs implementation supports. VectorStoreTester had no test at all, so this adds a characterisation suite covering the empty state, the blank-query guard, the search call and its rendered result, result expansion, Enter versus Shift+Enter, the failure path and clearing history. All of these pass against the current antd and Tremor components
Replaces antd and Tremor with the installed shadcn primitives on the three route-exclusive panels: Tremor tabs, buttons and text on budgets; the antd delete Modal and Tremor button on skills; the Tremor card, inputs and buttons on ui-theme. Markup only, no behaviour change. The characterisation tests added in the previous commit are untouched and stay green, and the ui-theme inputs now carry real label associations. Shared components stay on antd; they are reached by other routes and are migrated separately. The form-bearing files on these routes are left alone.
…itellm_/dazzling-gagarin-3d4c69
… before the shadcn migration Rewrite the two markup-coupled assertions off antd class selectors and onto role/text queries, and add characterisation tests for the nine route-owned components that had none. Both rewritten tests and all nine new ones are green against the current antd and Tremor components, so the migration that follows can be judged by tests it never touched.
…shadcn Moves the nine files these three routes exclusively own off antd and Tremor onto the shadcn primitives in src/components/ui. Scope came from the migration analyzer's import closure, so nothing reached by a second route is touched and every file carrying an antd Form is left alone until BerriAI#34195 lands. access-groups gets the page header, search box and the whole detail view; vector-stores gets the tab shell, the store picker and the tester panel; organizations gets the organization detail view and the three filter controls. Two changes are behavioural rather than cosmetic. The vector-stores tab strip moves from Tremor, which mounts every panel at once, to Base UI, which mounts only the active panel; that is the correct behaviour and the reworked test now opens the tab it asserts on. The antd Select on the Test Vector Store tab becomes a combobox rather than a plain select so its showSearch type-ahead survives. organization_view keeps one antd import, the ColumnsType used to build the extra columns it hands to the shared MemberTable; that is dictated by the shared component's API and goes away when MemberTable migrates. eslint-suppressions.json ratchets down accordingly: eight files lose their no-restricted-imports entry and organization_view drops from three to one. Every test passes unedited across the migration, and the visual gate reports the three migrated routes changed with the other 32 pixel-identical
…itellm_/sleepy-pascal-0e7ee6
…re the shadcn migration Establishes the regression net for the upcoming markup migration of these three routes. Every assertion here is written against the current antd and Tremor components and passes against them, so it carries no knowledge of the markup that replaces them and stays meaningful afterwards. Adds characterisation tests for the seven components that had none, and rewrites cache_dashboard's chart-card lookup to anchor on each chart's own title instead of asserting a global count of card nodes, which would break the moment another card appears on the page. No component is touched in this commit.
Greptile caught a real regression in the shadcn migration: starting to edit organization settings and switching to another tab silently discarded the unsaved input. antd Tabs and Tremor TabGroup mount a panel lazily and then keep it mounted, so a half-filled form or a search history survives leaving the tab and coming back. Base UI unmounts inactive panels instead. Its keepMounted escape hatch is not equivalent either: it mounts every panel eagerly, which renders work the user may never ask for and, on the organization view, put the organization name on screen twice. useVisitedTabs reproduces the original semantics by tracking which tabs have been opened and keeping only those mounted. It is applied to the two tab strips whose panels wrap stateful children: organization Settings, and the vector-stores Create and Test tabs, where an in-progress upload or a search history was equally exposed. The access-group detail tabs render lists derived from props, so they stay lazy. The added regression test fails without the fix and passes with it, and it also passes against the pre-migration antd component, so it pins parity rather than the new markup.
…c lag (BerriAI#34854) * test(e2e): harden harness and tests against data-plane pod churn A stage autoscaler scale-down produced a 2s window of ALB 502s that killed six budget tests on their first management call, and a freshly scaled-up pod that had not run its 30s DB object sync yet failed two MCP tests and one prometheus cardinality test. Retry transient gateway errors (502/503/504, connection errors) once at the shared e2e_http dispatch seam, poll MCP server registration to the poll deadline instead of asserting a single-shot listing, anchor the MCP guardrail full-sync wait to the later of the guardrail and server writes, and turn the prometheus alias poll into a drive-and-scrape convergence loop that re-sends traffic for missing aliases and unions results across scrapes * test(e2e): drain request body in retry stub handler so keep-alive reuse cannot misparse leftovers as requests * revert(e2e): drop the transient-502 retry seam A raw 502 during a pod scale-down is what a real client sees, so the suite retrying past it hides an availability gap instead of flagging it. The gateway-side fix is graceful drain on the deployment; until then the failures are signal * test(e2e): cap per-alias driver re-drives in the prometheus cardinality poll Bounds worst-case provider spend to 4 completions per alias while scrapes keep polling to the deadline; counters persist on whichever pod served them, so the cap costs no convergence unless that pod dies * test(e2e): drop driver re-drives from the prometheus cardinality poll The per-key cardinality contract is process-local and counters persist on whichever pod served the driver call, so unioning aliases across free scrape polls converges without re-sending billable traffic. The residual gap, a pod dying inside the poll window, is deferred to direct per-pod scraping
…4885) * Fix cache leakage card layout to keep date picker on right and prevent content overlap Removes flex-wrap and mt-3 to ensure date picker stays pinned to the right side of the card header regardless of zoom level, preventing it from covering card content below * Remove overflow-hidden from Card to allow dropdowns and overlays to display fully Fixes date picker dropdown being clipped when opened in cards like the Cache Leakage Card. By removing overflow-hidden from the Card container, popovers, dropdowns, and other overflow content can now display properly without being clipped by the card boundaries. * Make cache leakage card descriptions consistent with line clamping Adds line-clamp-2 to ensure both 'by model' and 'by virtual key' cards maintain consistent height. Removes conditional anthropic-specific text that caused height variations between dimensions.
* fix(gateway): route /a2a through the gateway component
A2A message-send runs the completion bridge, an outbound LLM call, but the
ingress only listed /v1/a2a so the serving routes at /a2a/{agent_id} fell to
the backend catch-all. Backend pods hold no provider credentials, so every
invocation died with a missing-provider-key auth error while the same call
succeeds on the gateway fleet. Adds /a2a to the ingress gateway prefixes and
the gateway route allowlist, plus a parity test so an ingress prefix that the
gateway trims can never reappear
* revert(test): drop the allowlist parity tests
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
…tract (BerriAI#34968) The cache-hit and paid rows for the two driver calls flush from different pods on independent update_spend timers, so waiting only for the cache-hit row can return a half-arrived result set where the paid-row assertion then fails on an empty list. Requiring both row kinds in the poll predicate lets the existing deadline absorb the slower flush without weakening any assertion
… paths to provider creds (BerriAI#34980)
A team's model_aliases can map a public name like gpt-4 to the internal
routing key (model_name_{team_id}_{uuid}) of a team deployment that has
since been deleted, e.g. after replacing per-team duplicates with one
gateway-level model. The pre-call rewrite then sent every request to a
name the router cannot serve, failing with "no healthy deployments for
model_name_..." even though the requested name still resolves at the
gateway level. The rewrite is now skipped when the alias target has no
live deployment in the router
delete_model also skipped the team alias scan for internal-shaped names
on the assumption they can never be alias values, which is exactly the
shape legacy team model aliases have, so deleting a legacy team model
left the stale alias behind. The scan now always runs, and a public
name that still resolves to a live router deployment (e.g. a shared
gateway-level model group) stays in team.models so the delete does not
revoke the team's access to it
…the deleted name Scrub aliases on delete only when the deleted deployment's model_name no longer resolves in the router. A legacy load-balanced team model can have several deployment rows sharing one internal name; deleting one replica must not remove aliases that still route to the survivors, in any team
…34749) * fix(proxy): warm rotate Prisma client for IAM refresh * fix(proxy): drain Prisma operations during IAM rotation * fix(proxy): bound the drain wait when retiring a replaced prisma engine A replaced engine waited indefinitely for its drain tracker to empty. Hung queries self-release via prisma's 30s default HTTP timeout, but a transaction whose owner is hard-cancelled before commit/rollback leaks its drain count forever, keeping the retired engine and its DB connection pool alive indefinitely; at one rotation per 12 minutes such engines accumulate. Cap the wait at 90 seconds, which exceeds every legitimate operation bound (30s HTTP timeout, 60s max interactive transaction timeout in this codebase), then kill the engine anyway. Work killed at the deadline degrades to the pre-drain behavior and is retried by the existing reconnect/backoff layers. --------- Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
…s fail validation
…BerriAI#34963) Bitnami retired the versioned tags under docker.io/bitnami and republished the archived builds under docker.io/bitnamilegacy, so every install and upgrade of the chart with the bundled database fails to pull docker.io/bitnami/postgresql:16.2.0-debian-12-r6. Repoint the subchart images at the bitnamilegacy copies of the exact builds those subchart versions shipped with, so the on-disk data directory layout is unchanged for existing installs. Pin the subchart dependency ranges to the versions already in Chart.lock. The current bitnami postgresql chart defaults to `tag: latest`, which is PostgreSQL 18 today, so an open-ended range turns a dependency refresh into a major-version jump on an existing volume. Refuse to render when postgresql.image.tag is empty or `latest` while the bundled database is deployed. Starting a different PostgreSQL major against an existing data directory leaves the server unable to boot with no in-place way back, which is how the reported install lost its data. Resolves LIT-4708
…passthrough_stream fix(vertex): decide rawPredict passthrough streaming from the request body
…p_thinking fix(anthropic-adapter): translate stop_sequences and disabled thinking for non-Claude targets
…on_schema ci: publish a generated JSON schema for model_prices_and_context_window.json
… reload Every model-write endpoint returned 200 off the DB write alone; a model the reload dropped (ignore_invalid_deployments, or a wholesale reload failure) stayed invisible on every channel at once, which is how the registry-leak defect went undiagnosed for three weeks. ProxyConfig.add_deployment and clear_cache now return whether the reload pass completed, and each write endpoint verifies the rows it wrote are live in this pod's router afterwards, distinguishing a deliberately environment-inactive model via the same predicate the Router's own gate uses. The access-group writers return the mutated id set instead of discarding it
…_team_route_1784693761 fix(jwt_auth): allow /v1/messages for JWT teams by default
…-cwalrj fix(proxy): skip team model aliases that point at deleted deployments
…_reload_drop fix(proxy): report when a model write does not survive the post-write reload
An auto-router deployment's litellm_params.model (auto_router/...) is the discriminator the router loads it by, but the model management endpoints accepted any client-supplied value verbatim; a doubled or stripped prefix made router init fail on the next load and ignore_invalid_deployments silently dropped the deployment. Validate writes that supply litellm_params.model at all three endpoints against the merged params and reject incoherent values with an actionable 400. Classification is extracted to router_utils/auto_router_model_naming.py so the Router predicates and the validation share one source
…ter_prefix fix(proxy): reject model writes that corrupt an auto-router pseudo-model
…medelta fix(router_strategy): serialize latency for non-chat responses in lowest-latency routing
…stream type so reasoning-first streams start with thinking (BerriAI#34433) * fix(anthropic-adapter): open first content block with the real upstream type * fix(anthropic): defer blank leading stream deltas
* fix(mcp): resolve call_tool by registry without requiring tool map Multi-worker reloads put MCP servers in the registry from the DB but do not re-run tools/list on every process. Gating call_tool on tool_name_to_mcp_server_name_mapping made cold workers 500 with Tool not found after another worker had already listed the tool. Treat a registry match on server id/name/alias as enough; upstream rejects unknown tools * test(e2e): poll MCP register, tools/list, and tools/call across multi-worker lag Stage multi-worker gateways only load MCP servers and tool maps on the process that handled the request. Poll until the server is listed, the tool appears on tools/list, and tools/call is not a cold-worker 500 so key-access and Datadog MCP e2e stop racing the LB * Revert "fix(mcp): resolve call_tool by registry without requiring tool map" This reverts commit 8b56e51. * test(e2e): tighten MCP multi-worker lag classifier Only retry tools/call on gateway shapes Tool <name> not found and server_not_found, not any 500 that mentions tool/server not found, so upstream failures are not retried until the poll deadline * test(e2e): drop unit file for MCP lag classifier The live await_call_tool polls already cover multi-worker lag; a separate string-match unit module is not worth keeping
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.