Skip to content

Conversation

@linw1995
Copy link
Owner

Summary

This PR implements exponential backoff retry mechanism for provider API calls (Anthropic and OpenAI), addressing issue #114.

Changes

Core Implementation

  • New retry module (src/retry.rs): Defines RetryAttempt, RetryUpdate, and RetryNotifier types for retry event tracking
  • Configuration: Added retry_max_attempts (default: 3) and retry_max_delay_ms (default: 60000ms) to RequestOptions and ModelRequestConfig
  • Retry logic: Uses the backon crate with ExponentialBuilder including jitter to prevent thundering herd problem

Provider API Clients

  • OpenAI client (crates/openai/src/client.rs):

    • Wrapped all API calls with exponential backoff retry
    • Added OpenAIRequestError with proper error classification (transport, HTTP status, decode)
    • Retryable status codes: 429 (Too Many Requests), 500 (Internal Server Error), 503 (Service Unavailable)
    • Network timeouts and connection errors are also retryable
    • Non-retryable: 400, 401, 403, 404, and decode errors
  • Anthropic client (crates/anthropic/src/client/messages.rs):

    • Same retry pattern applied to both streaming and non-streaming message APIs
    • Consistent error classification with OpenAI implementation

UI Integration

  • TUI chat component (crates/coco-tui/src/components/chat.rs):
    • Added retry status indicator showing current retry attempt and delay before next attempt
    • Displays format: "retrying in {delay} (attempt {current}/{max})"
    • Status clears when retry finishes (success or final failure)

Logging & Observability

  • Each retry attempt logs the error and delay duration using tracing::warn
  • Retry events are propagated through RetryNotifier callback system
  • Integrates with TUI event system for real-time status updates

Issue Reference

Resolves #114

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 9.77131% with 434 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.22%. Comparing base (e5977ab) to head (3be0148).
⚠️ Report is 20 commits behind head on dev.

Files with missing lines Patch % Lines
crates/anthropic/src/client/messages.rs 6.36% 147 Missing ⚠️
crates/openai/src/client.rs 6.57% 142 Missing ⚠️
src/provider.rs 0.00% 39 Missing ⚠️
crates/anthropic/src/client.rs 0.00% 35 Missing ⚠️
crates/coco-tui/src/components/chat.rs 0.00% 30 Missing ⚠️
src/provider/openai.rs 0.00% 25 Missing ⚠️
src/retry.rs 0.00% 11 Missing ⚠️
src/agent.rs 40.00% 3 Missing ⚠️
src/config/provider.rs 92.59% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #123      +/-   ##
==========================================
- Coverage   53.46%   50.22%   -3.25%     
==========================================
  Files          88       93       +5     
  Lines       21248    25033    +3785     
==========================================
+ Hits        11360    12572    +1212     
- Misses       9888    12461    +2573     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@linw1995 linw1995 merged commit a46f02c into dev Jan 22, 2026
3 checks passed
@linw1995 linw1995 added the enhancement New feature or request label Jan 22, 2026
@linw1995 linw1995 deleted the feature/retry-exponential-backoff branch January 22, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Provider API calls should support exponential backoff retry

3 participants