Skip to content

fix(observability): demote expected chat failures out of Sentry (TAURI-RUST-68, closes #2926)#2928

Merged
graycyrus merged 1 commit into
tinyhumansai:mainfrom
graycyrus:worktree-agent-a69ea424
May 29, 2026
Merged

fix(observability): demote expected chat failures out of Sentry (TAURI-RUST-68, closes #2926)#2928
graycyrus merged 1 commit into
tinyhumansai:mainfrom
graycyrus:worktree-agent-a69ea424

Conversation

@graycyrus
Copy link
Copy Markdown
Contributor

@graycyrus graycyrus commented May 29, 2026

Summary

  • inference_test_provider_model fires error! for every simple_chat failure (401, 429, model not found, etc.), creating 1,309 duplicate Sentry events (TAURI-RUST-68). The sub-errors already have their own Sentry reports from the provider layer.
  • Added is_expected_chat_failure helper that delegates to the shared expected_error_kind classifier in core::observability. Expected conditions are demoted to warn! (local log, no Sentry event). Genuine unexpected failures (5xx, payload parse errors, I/O) still escalate.
  • 4 unit tests cover: API key missing, rate-limit (429), provider config rejection (model not found / temperature unsupported), and real errors that must not be demoted.

Pre-push hook failed due to node_modules missing in worktree (prettier not found) — this is a pre-existing environment issue, not caused by these Rust-only changes.

Test plan

  • cargo test --lib -- "inference::ops" — all 21 tests pass (17 existing + 4 new)
  • cargo check --manifest-path Cargo.toml — no errors
  • cargo fmt — applied
  • N/A: Sentry event volume drop is a post-deploy metric; unit tests validate the classifier correctly demotes expected errors

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Improved error handling for provider failures by distinguishing between expected failures (such as missing credentials or rate limits) and unexpected ones, reducing unnecessary error escalations.
  • Tests

    • Added comprehensive unit tests to verify error classification logic for common provider-related failure patterns.

Review Change Stack

…I-RUST-68, tinyhumansai#2926)

`inference_test_provider_model` fires `error!` for every `simple_chat`
failure (401 API key invalid, 429 rate-limit, model not found, etc.).
These sub-errors already have their own Sentry reports from the provider
layer, making every `[inference::ops] test_provider_model:error` event a
duplicate — 1,309 events across releases.

Fix: gate the ops-level error log on `is_expected_chat_failure`, which
delegates to the shared `expected_error_kind` classifier in
`core::observability`. Expected conditions are demoted to `warn!` (local
log, no Sentry). Genuine unexpected failures (5xx, payload parse errors,
I/O) still escalate via `error!`.

Adds four unit tests anchored on the same `expected_error_kind` classifier
to prevent the guard from drifting out of sync.

Closes tinyhumansai#2926
@graycyrus graycyrus requested a review from a team May 29, 2026 10:08
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 29, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3a23e3aa-e58c-4ed6-9a19-b48f568c2d9e

📥 Commits

Reviewing files that changed from the base of the PR and between 04286cb and 34d353e.

📒 Files selected for processing (2)
  • src/openhuman/inference/ops.rs
  • src/openhuman/inference/ops_tests.rs

📝 Walkthrough

Walkthrough

The PR introduces a provider chat error classifier (is_expected_chat_failure) that checks whether inference failures match known, expected error patterns via the observability framework. Expected failures are demoted from error to warn logging in inference_test_provider_model, preventing duplicate Sentry escalation while preserving error visibility for unexpected failures.

Changes

Expected Error Classification

Layer / File(s) Summary
Expected failure classifier definition and tests
src/openhuman/inference/ops.rs (lines 32–56), src/openhuman/inference/ops_tests.rs (lines 319–389)
is_expected_chat_failure checks whether a provider chat error matches the framework's expected error kind classification set. Tests verify detection of API key missing, rate limiting, and provider config rejection patterns, and confirm unrelated error strings (500 errors, unexpected payloads, I/O failures) are not classified as expected.
Error handling branch in inference provider model
src/openhuman/inference/ops.rs (lines 207–222)
inference_test_provider_model error handling branches on is_expected_chat_failure: expected failures log at warn! level, while unexpected failures log as error!, suppressing duplicate Sentry reporting for known user/provider-state failures.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related issues

  • tinyhumansai/openhuman#2926: The new is_expected_chat_failure classifier and warn-level demotion directly address the duplicate Sentry events issue by preventing expected provider failures from escalating as errors.

Possibly related PRs

  • tinyhumansai/openhuman#2830: The new is_expected_chat_failure helper relies on the expanded core::observability::expected_error_kind classification rules added in that PR.
  • tinyhumansai/openhuman#2899: Both PRs connect through expected_error_kind, with this PR using it to demote expected chat/rate-limit failures to warn level while the related PR expands rate-limit pattern detection for the same framework.
  • tinyhumansai/openhuman#2692: Both implement the same error-demotion pattern by classifying provider failures as expected via core::observability to suppress escalation.

Suggested labels

rust-core, sentry-traced-bug, bug

Suggested reviewers

  • M3gA-Mind
  • oxoxDev

🐰 A classifier born, to sort the known from strange,
Expected failures whisper, no alerts to exchange,
Sentry stays silent when the pattern's understood,
Warn logs keep the record, demotion's doing good!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: adding a helper to demote expected chat failures from error to warn level in observability/Sentry, addressing the core problem of duplicate events.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. sentry-traced-bug Bug identified via Sentry triage bug labels May 29, 2026
Copy link
Copy Markdown
Contributor

@oxoxDev oxoxDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. is_expected_chat_failure delegates to the shared expected_error_kind classifier, so the ops-level test_provider_model wrapper stops duplicating Sentry events the provider layer already reported (or that are deterministically expected). Polarity verified: genuine 500s, unexpected payloads, local I/O errors, and empty strings are NOT demoted. CI green. Approving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. sentry-traced-bug Bug identified via Sentry triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants