Skip to content

test: provider error classification — overflow detection and message extraction#375

Merged
anandgupta42 merged 4 commits intomainfrom
claude/test-provider-error-20260322-session_01XDr623NsEC3PHzfgiQVPUh
Mar 23, 2026
Merged

test: provider error classification — overflow detection and message extraction#375
anandgupta42 merged 4 commits intomainfrom
claude/test-provider-error-20260322-session_01XDr623NsEC3PHzfgiQVPUh

Conversation

@anandgupta42
Copy link
Contributor

@anandgupta42 anandgupta42 commented Mar 22, 2026

What does this PR do?\n\n1. ProviderError.parseStreamErrorsrc/provider/error.ts (8 new tests)\n\nThis function classifies SSE streaming errors from LLM providers (context overflow, quota exceeded, invalid prompt, usage_not_included). It had zero test coverage despite being on the critical error-handling path — every streaming LLM failure flows through it. Wrong classification causes the session to crash instead of gracefully compacting context, or shows the wrong error message to users.\n\nNew coverage includes:\n- context_length_exceededcontext_overflow classification\n- usage_not_included → upgrade URL message verification\n- invalid_prompt passthrough message and fallback when message is non-string\n- Boundary cases: null/undefined input, non-error events, unknown error codes\n- JSON string input parsing (AI SDK sometimes passes SSE chunks as strings)\n\n2. ProviderError.parseAPICallErrorsrc/provider/error.ts (10 new tests)\n\nThis function classifies HTTP errors from LLM provider APIs, with overflow detection via 15+ regex patterns covering Anthropic, OpenAI, Google Gemini, Groq, DeepSeek, Cerebras, Mistral, and others. Also had zero test coverage. If overflow detection fails, sessions crash instead of gracefully compacting.\n\nNew coverage includes:\n- Overflow detection: Anthropic "prompt is too long", OpenAI "exceeds the context window", HTTP 413 status code, Cerebras/Mistral "400 (no body)" and "413 (no body)" patterns\n- OpenAI-specific 404 retry override (model temporarily unavailable → retryable)\n- Non-OpenAI provider isRetryable passthrough from SDK (both true and false)\n- HTML response body detection (403 → friendly gateway message instead of raw HTML)\n- URL metadata preservation in error output\n- Empty message fallback to HTTP status code text (critical for 429 rate-limiting)\n- Plain-text responseBody concatenation\n\n### Type of change\n- [x] New feature (non-breaking change which adds functionality)\n\n### Issue for this PR\nN/A — proactive test coverage for critical error-handling path\n\n### How did you verify your code works?\n\n\nbun test test/provider/error.test.ts # 18 pass, 0 fail, 40 expect() calls\n\n\nAll tests import directly from the source module (../../src/provider/error) — no re-implementations or mocks of the functions under test. The APICallError constructor from the ai SDK is used to create realistic error objects.\n\n### Checklist\n- [x] I have added tests that prove my fix is effective or that my feature works\n- [x] New and existing unit tests pass locally with my changes\n\nhttps://claude.ai/code/session_01XDr623NsEC3PHzfgiQVPUh

Summary by CodeRabbit

  • Tests
    • Added comprehensive test suite for error handling and classification across various scenarios, including stream error parsing, API call error detection, and error message normalization.

…extraction

ProviderError.parseStreamError and parseAPICallError had zero test coverage despite
being on the critical error-handling path for every LLM API failure. Wrong
classification causes crashes instead of graceful compaction, or incorrect retry
behavior. These 18 tests cover SSE error codes, overflow regex patterns across 6+
providers, HTTP 413 detection, OpenAI 404 retry override, HTML response handling,
and status code fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01XDr623NsEC3PHzfgiQVPUh
Copy link

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review.

@coderabbitai
Copy link

coderabbitai bot commented Mar 22, 2026

📝 Walkthrough

Walkthrough

A new Bun test suite was added for ProviderError to validate error classification and handling logic across SSE stream parsing, API call error detection, and error message normalization for multiple provider scenarios.

Changes

Cohort / File(s) Summary
ProviderError Test Suite
packages/opencode/test/provider/error.test.ts
New test file (+262 lines) with comprehensive test coverage for ProviderError.parseStreamError (SSE event classification, error code mapping), ProviderError.parseAPICallError (context overflow detection, provider-specific handling, status code interpretation), error message extraction, normalization, and fallback behavior across edge cases (null inputs, HTML responses, empty messages, plain-text content).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Suggested labels

contributor

Poem

🐰 When errors flow through streams and calls,
The tests stand guard in humble halls,
Each edge case caught, each code unwound,
In Bun's swift dance, new truth is found! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding tests for provider error classification, overflow detection, and message extraction.
Description check ✅ Passed The description is comprehensive and follows the template structure with clear Summary, Test Plan sections, and completed Checklist; all required information is present.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/test-provider-error-20260322-session_01XDr623NsEC3PHzfgiQVPUh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/opencode/test/provider/error.test.ts (1)

29-105: Add a direct test for the insufficient_quota stream-error branch.

parseStreamError has an explicit insufficient_quota case, but this suite does not currently assert it. Adding one test will close that branch-coverage gap.

Proposed test addition
 describe("ProviderError.parseStreamError: SSE error classification", () => {
   test("classifies context_length_exceeded as context_overflow", () => {
@@
     expect(result!.type).toBe("context_overflow")
   })

+  test("classifies insufficient_quota as non-retryable api_error", () => {
+    const result = ProviderError.parseStreamError({
+      type: "error",
+      error: { code: "insufficient_quota", message: "quota exceeded" },
+    })
+    expect(result).toBeDefined()
+    expect(result!.type).toBe("api_error")
+    if (result!.type === "api_error") {
+      expect(result!.isRetryable).toBe(false)
+      expect(result!.message).toContain("Quota exceeded")
+    }
+  })
+
   test("classifies usage_not_included with upgrade URL", () => {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/opencode/test/provider/error.test.ts` around lines 29 - 105, Add a
new unit test in the same "ProviderError.parseStreamError: SSE error
classification" suite that sends an SSE error event with error.code
"insufficient_quota" to ProviderError.parseStreamError and asserts the result is
defined and result!.type is "insufficient_quota" (use
ProviderError.parseStreamError and the "insufficient_quota" code symbol to
locate where to add the test).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/opencode/test/provider/error.test.ts`:
- Around line 29-105: Add a new unit test in the same
"ProviderError.parseStreamError: SSE error classification" suite that sends an
SSE error event with error.code "insufficient_quota" to
ProviderError.parseStreamError and asserts the result is defined and
result!.type is "insufficient_quota" (use ProviderError.parseStreamError and the
"insufficient_quota" code symbol to locate where to add the test).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 251e318b-306c-480c-a8bb-e23fafc19bc9

📥 Commits

Reviewing files that changed from the base of the PR and between d10303c and ebf6996.

📒 Files selected for processing (1)
  • packages/opencode/test/provider/error.test.ts

@anandgupta42 anandgupta42 merged commit 27941cf into main Mar 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant