Skip to content

Comments

fix(iflow): add connection error retry logic and silent context failure detection#139

Open
MasuRii wants to merge 3 commits intoMirrowel:devfrom
MasuRii:fix/iflow-connection-error-handling
Open

fix(iflow): add connection error retry logic and silent context failure detection#139
MasuRii wants to merge 3 commits intoMirrowel:devfrom
MasuRii:fix/iflow-connection-error-handling

Conversation

@MasuRii
Copy link
Contributor

@MasuRii MasuRii commented Feb 21, 2026

Summary

Add robust connection error handling with retry logic and silent context failure detection for the iFlow provider, preventing quota waste from transient network issues and undetected empty responses.

Changes

  • Connection error retry logic: Wrap stream_handler in a retry loop with exponential backoff (1s/2s/4s) for transient connection errors (RemoteProtocolError, ConnectError, ReadTimeout, NetworkError), retrying up to 3 times before re-raising
  • Silent context failure detection: Detect and reject HTTP 200 responses that contain no choices, zero completion tokens with non-zero prompt tokens, empty assistant messages, or streams that complete without any data chunks — raising a non-retryable context_window_exceeded error
  • Context window error detection: Surface token limit errors explicitly from HTTP error response bodies using pattern matching
  • Error classification improvement: Add httpx.RemoteProtocolError to the api_connection error type classification in the global error handler
  • Module-level constants: Extract CONNECTION_ERROR_TYPES, CONTEXT_WINDOW_ERROR_PATTERNS, MAX_CONNECTION_RETRIES, and RETRY_BACKOFF_BASE for clarity and reuse

Files Changed

File Type Impact
src/rotator_library/providers/iflow_provider.py Modified Added retry loop with backoff in stream_handler, silent context failure detection, helper methods, and module-level error constants (+219/-80 lines)
src/rotator_library/error_handler.py Modified Added httpx.RemoteProtocolError to api_connection classification tuple (+6/-1 lines)

Testing Instructions

  1. Trigger iFlow requests with a large context that exceeds the model's window — verify the proxy surfaces a clear context window error instead of silently failing
  2. Simulate a transient connection drop mid-stream — verify the proxy retries up to 3 times with backoff before failing
  3. Verify normal iFlow streaming requests still complete successfully without unnecessary retries

Important

Enhances iFlow provider with retry logic for connection errors and detection of silent context failures, updating error classifications and constants.

  • Behavior:
    • Adds retry logic with exponential backoff in stream_handler in iflow_provider.py for connection errors (RemoteProtocolError, ConnectError, ReadTimeout, NetworkError), retrying up to 3 times.
    • Detects silent context failures in iflow_provider.py by checking for HTTP 200 responses with no choices, zero completion tokens, empty messages, or streams without data, raising context_window_exceeded.
    • Explicitly surfaces context window errors from HTTP error responses using pattern matching in iflow_provider.py.
  • Error Handling:
    • Adds httpx.RemoteProtocolError to api_connection error classification in error_handler.py.
  • Constants:
    • Extracts CONNECTION_ERROR_TYPES, CONTEXT_WINDOW_ERROR_PATTERNS, MAX_CONNECTION_RETRIES, and RETRY_BACKOFF_BASE in iflow_provider.py for clarity and reuse.

This description was created by Ellipsis for a6e3024. You can customize this summary. It will automatically update as commits are pushed.

…ication

Include httpx.RemoteProtocolError in the error type tuple for
api_connection classification. This error occurs when a peer closes the
connection without sending a complete message and should be treated as a
transient connection issue rather than an unhandled exception.
…handler

Wrap stream_handler in a retry loop with exponential backoff for
transient connection errors (RemoteProtocolError, ConnectError,
ReadTimeout, NetworkError). Retries up to 3 times with 1s/2s/4s
backoff before re-raising for higher-level error handling.

Changes:
- Add CONNECTION_ERROR_TYPES tuple and CONTEXT_WINDOW_ERROR_PATTERNS
  as module-level constants for reuse and clarity
- Add MAX_CONNECTION_RETRIES and RETRY_BACKOFF_BASE configuration
- Restructure stream_handler with while-loop retry around the stream
  context manager, re-creating the stream on each retry attempt
- Add context window error detection from HTTP error response bodies
  to surface token limit issues explicitly
- Import asyncio for async sleep during backoff
- Detect empty choices array in HTTP 200 responses
- Detect zero completion_tokens with non-zero prompt_tokens
- Detect empty assistant messages (no content/reasoning/tool_calls)
- Detect streams that complete without any data chunks
- Raise non-retryable context_window_exceeded error for these cases
- Prevents quota waste from repeated failed requests
@MasuRii MasuRii requested a review from Mirrowel as a code owner February 21, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant