test: provider error classification — overflow detection and message extraction#375
Conversation
…extraction ProviderError.parseStreamError and parseAPICallError had zero test coverage despite being on the critical error-handling path for every LLM API failure. Wrong classification causes crashes instead of graceful compaction, or incorrect retry behavior. These 18 tests cover SSE error codes, overflow regex patterns across 6+ providers, HTTP 413 detection, OpenAI 404 retry override, HTML response handling, and status code fallback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01XDr623NsEC3PHzfgiQVPUh
📝 WalkthroughWalkthroughA new Bun test suite was added for Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/opencode/test/provider/error.test.ts (1)
29-105: Add a direct test for theinsufficient_quotastream-error branch.
parseStreamErrorhas an explicitinsufficient_quotacase, but this suite does not currently assert it. Adding one test will close that branch-coverage gap.Proposed test addition
describe("ProviderError.parseStreamError: SSE error classification", () => { test("classifies context_length_exceeded as context_overflow", () => { @@ expect(result!.type).toBe("context_overflow") }) + test("classifies insufficient_quota as non-retryable api_error", () => { + const result = ProviderError.parseStreamError({ + type: "error", + error: { code: "insufficient_quota", message: "quota exceeded" }, + }) + expect(result).toBeDefined() + expect(result!.type).toBe("api_error") + if (result!.type === "api_error") { + expect(result!.isRetryable).toBe(false) + expect(result!.message).toContain("Quota exceeded") + } + }) + test("classifies usage_not_included with upgrade URL", () => {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opencode/test/provider/error.test.ts` around lines 29 - 105, Add a new unit test in the same "ProviderError.parseStreamError: SSE error classification" suite that sends an SSE error event with error.code "insufficient_quota" to ProviderError.parseStreamError and asserts the result is defined and result!.type is "insufficient_quota" (use ProviderError.parseStreamError and the "insufficient_quota" code symbol to locate where to add the test).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@packages/opencode/test/provider/error.test.ts`:
- Around line 29-105: Add a new unit test in the same
"ProviderError.parseStreamError: SSE error classification" suite that sends an
SSE error event with error.code "insufficient_quota" to
ProviderError.parseStreamError and asserts the result is defined and
result!.type is "insufficient_quota" (use ProviderError.parseStreamError and the
"insufficient_quota" code symbol to locate where to add the test).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 251e318b-306c-480c-a8bb-e23fafc19bc9
📒 Files selected for processing (1)
packages/opencode/test/provider/error.test.ts
…01XDr623NsEC3PHzfgiQVPUh
…01XDr623NsEC3PHzfgiQVPUh
…01XDr623NsEC3PHzfgiQVPUh
What does this PR do?\n\n1.
ProviderError.parseStreamError—src/provider/error.ts(8 new tests)\n\nThis function classifies SSE streaming errors from LLM providers (context overflow, quota exceeded, invalid prompt, usage_not_included). It had zero test coverage despite being on the critical error-handling path — every streaming LLM failure flows through it. Wrong classification causes the session to crash instead of gracefully compacting context, or shows the wrong error message to users.\n\nNew coverage includes:\n-context_length_exceeded→context_overflowclassification\n-usage_not_included→ upgrade URL message verification\n-invalid_promptpassthrough message and fallback when message is non-string\n- Boundary cases: null/undefined input, non-error events, unknown error codes\n- JSON string input parsing (AI SDK sometimes passes SSE chunks as strings)\n\n2.ProviderError.parseAPICallError—src/provider/error.ts(10 new tests)\n\nThis function classifies HTTP errors from LLM provider APIs, with overflow detection via 15+ regex patterns covering Anthropic, OpenAI, Google Gemini, Groq, DeepSeek, Cerebras, Mistral, and others. Also had zero test coverage. If overflow detection fails, sessions crash instead of gracefully compacting.\n\nNew coverage includes:\n- Overflow detection: Anthropic "prompt is too long", OpenAI "exceeds the context window", HTTP 413 status code, Cerebras/Mistral "400 (no body)" and "413 (no body)" patterns\n- OpenAI-specific 404 retry override (model temporarily unavailable → retryable)\n- Non-OpenAI provider isRetryable passthrough from SDK (both true and false)\n- HTML response body detection (403 → friendly gateway message instead of raw HTML)\n- URL metadata preservation in error output\n- Empty message fallback to HTTP status code text (critical for 429 rate-limiting)\n- Plain-text responseBody concatenation\n\n### Type of change\n- [x] New feature (non-breaking change which adds functionality)\n\n### Issue for this PR\nN/A — proactive test coverage for critical error-handling path\n\n### How did you verify your code works?\n\n\nbun test test/provider/error.test.ts # 18 pass, 0 fail, 40 expect() calls\n\n\nAll tests import directly from the source module (../../src/provider/error) — no re-implementations or mocks of the functions under test. TheAPICallErrorconstructor from theaiSDK is used to create realistic error objects.\n\n### Checklist\n- [x] I have added tests that prove my fix is effective or that my feature works\n- [x] New and existing unit tests pass locally with my changes\n\nhttps://claude.ai/code/session_01XDr623NsEC3PHzfgiQVPUhSummary by CodeRabbit