Document and Test Provider Fallback Capabilities via Iterative Request Router - #730
Open
usize wants to merge 2 commits into
Open
Document and Test Provider Fallback Capabilities via Iterative Request Router#730usize wants to merge 2 commits into
usize wants to merge 2 commits into
Conversation
First conformance test for the AI gateway. Validates that an iterative_request_router composes correctly with responses_to_chat_completions protocol translation and credential_injection across a failover boundary. Two tests exercise the example config: - fallback_on_primary_503: primary returns 503, fallback receives the translated Chat Completions request with isolated credentials, client gets a Responses API resource. - primary_succeeds_no_fallback: primary returns 200, fallback receives no requests. Each IRR step re-runs openai_responses_format and openai_responses_validate because the IRR resets per-step metadata for credential isolation while preserving extensions. Assisted by Opus 4.6 Signed-off-by: usize <mofoster@redhat.com>
Move the metadata/extension persistence explanation into the example config header comments, where someone building a pipeline encounters it. Add a one-liner to ai-inference.md pointing to the example. Document real-world adaptation (model rewrite, TLS/SNI, env_var credentials, Host header) in the "Adapting for real providers" section. Assisted by Opus 4.6 Signed-off-by: usize <mofoster@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Provider fallbacks, along with model rewriting and api translation, are now feasible via the iterative request router. Because this functionality exists as an emergent property of our existing filters and filter pipeline, we should document it. These changes document it via a new integration test and example config.
Related issue
Ref: #249, #109
Validation
make lintChecklist
Signed-off-bytrailer.