Skip to content

Add typed error response#88

Merged
wpak-ai merged 6 commits into
cppalliance:mainfrom
jonathanMLDev:feat/add-typed-error-response
May 15, 2026
Merged

Add typed error response#88
wpak-ai merged 6 commits into
cppalliance:mainfrom
jonathanMLDev:feat/add-typed-error-response

Conversation

@jonathanMLDev
Copy link
Copy Markdown
Collaborator

@jonathanMLDev jonathanMLDev commented May 14, 2026

Pull Request

Core Changes

  • src/server/tool-error.ts — Zod toolErrorSchema / discriminated union on code (FLOW_GATE, VALIDATION, PINECONE_ERROR, TIMEOUT); builders flowGateToolError, validationToolError, pineconeToolError, timeoutToolError; classifyToolCatchError uses raw Error.message for the Timeout after …ms while waiting for prefix, then getToolErrorMessage for the user-facing message.
  • src/server/tool-response.ts — jsonErrorResponse(err: ToolError) validates with toolErrorSchema before stringify; still sets isError: true.
  • src/server/metadata-filter.ts — validateMetadataFilterDetailed() → { message, field } | null; validateMetadataFilter() unchanged (string wrapper).

Tools (all 9)

  • Flow gate: query, count, query_documents → flowGateToolError(namespace, message) with the required suggestion template.
    Validation: empty inputs, metadata filters → validationToolError with field; guided “namespace not in cache” → field: 'namespace'; empty index / no namespace → pineconeToolError with recoverable: true + suggestion (per plan comment).
  • Catches → classifyToolCatchError.
  • Keyword search: executeKeywordSearch returns { ok, body } | { ok, error }; KeywordSearchResponse is success-only.

Types

  • src/types.ts — QueryResponse is success-only (no status: 'error' / message).

Package surface

  • src/server.ts — exports toolErrorSchema, ToolError, ToolErrorCode, validateMetadataFilterDetailed, MetadataFilterValidationError; package doc mentions toolErrorSchema / ToolError.

Docs & tests

  • README.md — new “Error responses” section.
  • CHANGELOG.md — breaking MCP error body + types; added validateMetadataFilterDetailed.
  • src/server/tool-error.test.ts — 5 tests (all four codes + schema rejection for VALIDATION without field).
  • Tool tests updated to use assertToolError from test-helpers.ts.

Related Issue

close #84

Summary by CodeRabbit

  • New Features

    • Standardized structured tool error format with typed error codes, recoverability flags, optional suggestions, and field-specific validation info.
    • New detailed metadata-filter validator that returns message+field for precise validation errors.
  • Changed

    • Breaking: Tool error payloads now use typed error objects (returned with isError: true); old {status: 'error', message} shape removed.
    • Breaking: Response types for queries/keyword search now model success-only shapes; failures are returned as structured tool errors.
  • Documentation

    • README updated to document the new error contract.

Review Change Stack

@jonathanMLDev jonathanMLDev self-assigned this May 14, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

📝 Walkthrough

Walkthrough

Adds a Zod-validated ToolError discriminated union and helpers, introduces validateMetadataFilterDetailed returning field-attributed errors, validates MCP error payloads via jsonErrorResponse, updates tool handlers to return structured ToolErrors, updates tests to assert the new error contract, and documents the change in README/CHANGELOG and types.

Changes

Typed Error Response System

Layer / File(s) Summary
All changes (schema, validation, handlers, tests, docs)
src/server/tool-error.ts, src/server/metadata-filter.ts, src/server/tool-response.ts, src/server/tools/*, src/server/tools/*.test.ts, src/server/tools/test-helpers.ts, src/server.ts, src/types.ts, README.md, CHANGELOG.md
Defines and exports a Zod toolErrorSchema and ToolError / ToolErrorCode types; adds validateMetadataFilterDetailed returning { message, field }; updates jsonErrorResponse to accept/validate ToolError; migrates tool handlers to use validationToolError, flowGateToolError, and classifyToolCatchError; updates tests to use assertToolError; narrows response types to success-only shapes; and updates docs/changelog.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • wpak-ai
  • AuraMindNest

Poem

🐰 Errors now wear their tidy coat,
With codes and fields for every note,
A hint, a path, a nudge to try,
Recoverable or not — oh my!
Hop on, clients, with a happier float. 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 68.97% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Add typed error response' clearly and concisely summarizes the main change: introducing a typed error response system as the PR's primary objective.
Linked Issues check ✅ Passed All acceptance criteria from issue #84 are met: ToolError type with required fields (code, message, recoverable, suggestion) defined in tool-error.ts; error handling integrated across 9 tools using the new format; FLOW_GATE errors include suggestion templates; VALIDATION errors include field information; comprehensive tests added; README documented.
Out of Scope Changes check ✅ Passed All changes are directly aligned with issue #84 requirements. The PR introduces typed errors, updates tools to use them, adds tests, and documents the feature. No unrelated refactoring or feature additions observed.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

❌ Patch coverage is 90.90909% with 9 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@335bda1). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/server/metadata-filter.ts 77.77% 4 Missing ⚠️
src/server/tools/test-helpers.ts 81.81% 2 Missing ⚠️
src/server/tools/guided-query-tool.ts 92.85% 1 Missing ⚠️
src/server/tools/query-documents-tool.ts 75.00% 1 Missing ⚠️
src/server/tools/query-tool.ts 80.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #88   +/-   ##
=======================================
  Coverage        ?   75.02%           
=======================================
  Files           ?       33           
  Lines           ?     1165           
  Branches        ?      376           
=======================================
  Hits            ?      874           
  Misses          ?      291           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/types.ts (1)

86-91: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Narrow ListNamespacesResponse to success-only to match the migrated error handling.

ListNamespacesResponse still defines status: 'success' | 'error' and message?: string, but list_namespaces now uses jsonErrorResponse() for failures, which returns a TextPayload with isError: true rather than a ListNamespacesResponse object. The type should be narrowed to status: 'success' only, matching QueryResponse and reflecting the actual behavior of error responses in the MCP protocol.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/types.ts` around lines 86 - 91, Update the ListNamespacesResponse
interface to represent only successful responses (matching QueryResponse and the
MCP protocol): change status to the literal 'success' (remove the union with
'error'), drop the optional message field (and any other error-related
properties), and ensure namespaces?: string[] and count?: number remain; this
aligns ListNamespacesResponse with the fact that failed cases are returned by
jsonErrorResponse() and not as ListNamespacesResponse objects (refer to
ListNamespacesResponse and list_namespaces).
🧹 Nitpick comments (1)
src/server/tools/generate-urls-tool.test.ts (1)

21-21: ⚡ Quick win

Verify the error message in addition to the code.

The test only checks code but does not verify the error message. According to the acceptance criteria, PINECONE_ERROR responses must include a descriptive message. Other PINECONE_ERROR tests in this PR verify both fields (e.g., list-namespaces-tool.test.ts line 70, suggest-query-params-tool.test.ts line 110).

✅ Recommended enhancement to verify message
-    expect(assertToolError(raw).code).toBe('PINECONE_ERROR');
+    const err = assertToolError(raw);
+    expect(err.code).toBe('PINECONE_ERROR');
+    expect(err.message).toBe('Failed to generate URLs');
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/server/tools/generate-urls-tool.test.ts` at line 21, The test currently
only asserts the error code from assertToolError(raw); add an assertion that
also verifies the error message (access assertToolError(raw).message) equals the
expected descriptive PINECONE_ERROR message (or uses toMatch with the same
text/regex used in the other PINECONE_ERROR tests such as the ones referenced
for list-namespaces and suggest-query-params) so the test validates both code
and message per acceptance criteria.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/server/tools/guided-query-tool.ts`:
- Around line 100-116: Normalize and validate inputNamespace before deciding
error type: if inputNamespace is provided (inputNamespace !== undefined) trim it
and if the trimmed string is empty/only whitespace return a JSON validation
error for the namespace field (VALIDATION, field "namespace") instead of the
current backend/pinecone error; otherwise proceed to use the existing namespace
resolution (namespace = inputNamespace ?? ranked[0]?.namespace) and only return
the pinecone-style recoverable error when no routable namespace exists and no
explicit namespace was supplied. Reference symbols: inputNamespace, namespace,
ranked, jsonErrorResponse, and the pinecone-style error path currently using
pineconeToolError.

In `@src/server/tools/test-helpers.ts`:
- Around line 43-47: The helper assertToolError currently parses the payload
with parseToolJson and validates the body shape with toolErrorSchema but does
not enforce the envelope flag; update assertToolError to explicitly check that
the parsed raw value has isError === true (e.g. throw an error if raw?.isError
!== true) before calling toolErrorSchema.parse, or incorporate the isError
constraint into the validation step so that assertToolError always rejects
non-error envelopes; reference the function assertToolError, the parseToolJson
call, and toolErrorSchema when making the change.

---

Outside diff comments:
In `@src/types.ts`:
- Around line 86-91: Update the ListNamespacesResponse interface to represent
only successful responses (matching QueryResponse and the MCP protocol): change
status to the literal 'success' (remove the union with 'error'), drop the
optional message field (and any other error-related properties), and ensure
namespaces?: string[] and count?: number remain; this aligns
ListNamespacesResponse with the fact that failed cases are returned by
jsonErrorResponse() and not as ListNamespacesResponse objects (refer to
ListNamespacesResponse and list_namespaces).

---

Nitpick comments:
In `@src/server/tools/generate-urls-tool.test.ts`:
- Line 21: The test currently only asserts the error code from
assertToolError(raw); add an assertion that also verifies the error message
(access assertToolError(raw).message) equals the expected descriptive
PINECONE_ERROR message (or uses toMatch with the same text/regex used in the
other PINECONE_ERROR tests such as the ones referenced for list-namespaces and
suggest-query-params) so the test validates both code and message per acceptance
criteria.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 84e0bb6e-33a7-490b-a682-7c5743dde576

📥 Commits

Reviewing files that changed from the base of the PR and between b961f3d and b225fe8.

📒 Files selected for processing (28)
  • CHANGELOG.md
  • README.md
  • src/server.ts
  • src/server/metadata-filter.test.ts
  • src/server/metadata-filter.ts
  • src/server/tool-error.test.ts
  • src/server/tool-error.ts
  • src/server/tool-response.ts
  • src/server/tools/count-tool.test.ts
  • src/server/tools/count-tool.ts
  • src/server/tools/generate-urls-tool.test.ts
  • src/server/tools/generate-urls-tool.ts
  • src/server/tools/guided-query-tool.test.ts
  • src/server/tools/guided-query-tool.ts
  • src/server/tools/keyword-search-tool.test.ts
  • src/server/tools/keyword-search-tool.ts
  • src/server/tools/list-namespaces-tool.test.ts
  • src/server/tools/list-namespaces-tool.ts
  • src/server/tools/namespace-router-tool.test.ts
  • src/server/tools/namespace-router-tool.ts
  • src/server/tools/query-documents-tool.test.ts
  • src/server/tools/query-documents-tool.ts
  • src/server/tools/query-tool.test.ts
  • src/server/tools/query-tool.ts
  • src/server/tools/suggest-query-params-tool.test.ts
  • src/server/tools/suggest-query-params-tool.ts
  • src/server/tools/test-helpers.ts
  • src/types.ts

Comment thread src/server/tools/guided-query-tool.ts Outdated
Comment thread src/server/tools/test-helpers.ts Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/server/tools/test-helpers.ts`:
- Around line 34-38: parseToolJson was made error-only by checking
envelope.isError and throwing, which breaks success-path tests; revert
parseToolJson to be envelope-agnostic by removing the isError check so it simply
casts/returns the payload as a Record<string, unknown>, and move/enforce any
isError assertions into assertToolError (e.g., check isError: true inside
assertToolError instead). Update parseToolJson to not throw based on isError and
ensure assertToolError performs the error-specific validation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fa756d10-970e-4e73-9047-d0dd64fce75c

📥 Commits

Reviewing files that changed from the base of the PR and between b225fe8 and 23682ee.

📒 Files selected for processing (2)
  • src/server/tools/guided-query-tool.ts
  • src/server/tools/test-helpers.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/server/tools/guided-query-tool.ts

Comment thread src/server/tools/test-helpers.ts Outdated
@jonathanMLDev
Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review again

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 15, 2026

@jonathanMLDev Sure! I'll kick off a full review of PR #88 right away.

✅ Actions performed

Full review triggered.

@jonathanMLDev jonathanMLDev requested a review from wpak-ai May 15, 2026 15:55
@wpak-ai wpak-ai merged commit 4b78587 into cppalliance:main May 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Typed Error Responses

2 participants