feat: multi-protocol request translation across Anthropic/OpenAI/Gemini#40
feat: multi-protocol request translation across Anthropic/OpenAI/Gemini#40ChasLui wants to merge 1 commit into
Conversation
Add a protocol-agnostic intermediate representation (IR) plus per-protocol codecs so the gateway can accept a request in one wire format and route it to an upstream that speaks another. The router parses with the inbound codec and renders with the outbound codec; when both sides share a wire format the body is passed through untouched (fast path) to preserve low overhead. Codecs implemented: Anthropic Messages, OpenAI Chat Completions, OpenAI Responses, and Gemini generateContent — covering requests, responses, and streaming (SSE) in each direction, including tool calls, reasoning/thinking, images, and usage normalization. Provider failures (non-2xx, HTTP-200 error bodies, Responses `status:"failed"`, streamed error events) are surfaced in the inbound protocol's native error shape rather than leaking as a clean success. Also adds native endpoints for OpenAI Chat Completions and Gemini, a multi-provider wildcard router, an expanded provider catalog, and docs/protocols.md describing the translation model and its limits. Response caching is intentionally excluded and will follow as a separate PR (see LiteLLM-Labs#39).
ChasLui
force-pushed
the
feat-multi-protocol-translation
branch
from
7ccbe1e to
5e2b5bf
Compare
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ccbe1ebb7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1 @@ | |||
| /Users/chao.liu/my-clone/github.com/ChasLui/litellm-rust/.claude/skills/gitnexus No newline at end of file | |||
There was a problem hiding this comment.
This committed symlink points to an author-specific /Users/chao.liu/... path, and the same pattern is used for the new .serena links. On any checkout outside that machine these entries are dangling, so tools or setup scripts that traverse .claude/skills or .serena will see broken repository contents; this also conflicts with the repo-wide CODING_STANDARDS.md guidance to keep local/tooling artifacts out of the root tree. Please remove these symlinks or generate/ignore them outside the committed tree.
Useful? React with 👍 / 👎.
|
For the record: we've decided to maintain this as an independent fork rather than upstream the work, so this PR stays closed. Thanks for the reviews on #38. |
1 participant
@codex review
This is PR 1 of the split proposed in #39, carving the multi-protocol translation foundation out of the (now-closed) #38 so it can be reviewed on its own. Response caching follows as a separate stacked PR.
What this adds
A protocol-agnostic intermediate representation (IR) plus per-protocol codecs, so the gateway can accept a request in one wire format and route it to an upstream that speaks another:
generateContent— each covering requests, responses, and streaming (SSE) in both directions.status:"failed", streamederrorevents) are surfaced in the inbound protocols native error shape rather than leaking as a clean success.docs/protocols.mddescribes the translation model and its limits.Scope / what is intentionally NOT here
cache_controlbreakpoints ride through the IR and re-emit on Anthropic render; cache token usage is normalized and echoed in each protocols native shape. The opt-in auto-injection of breakpoints moves to PR 2 with the rest of the caching feature.Verification
Each gate passes on this branch in isolation:
cargo fmt --checkcargo clippy --all-targets(no warnings)cargo test(all suites green)python3 scripts/check_code_size.py(<=300 lines/file, <=50 LOC/function)Closes part of #39.