Skip to content

fix(search): keep searchMessages cooperative and bounded (fixes main-thread freezes/timeouts on large folders)#161

Draft
GoeppertVB wants to merge 1 commit into
TKasperczyk:mainfrom
GoeppertVB:fix/search-main-thread-scan
Draft

fix(search): keep searchMessages cooperative and bounded (fixes main-thread freezes/timeouts on large folders)#161
GoeppertVB wants to merge 1 commit into
TKasperczyk:mainfrom
GoeppertVB:fix/search-main-thread-scan

Conversation

@GoeppertVB

Copy link
Copy Markdown

Problem

searchMessages (the default, non-searchBody path) can freeze the Thunderbird UI and time out — and occasionally crash TB — on large mailstores. In testing against an IMAP account with a ~191k-message INBOX and a ~219k-message All Mail folder, most searches returned Bridge error: Request to Thunderbird timed out; scoping to a small folder returned instantly.

Root cause

The search is a synchronous linear scan on TB's main thread (extension/mcp_server/api.js, searchFolderdb.enumerateMessages()), recursing every subfolder. Cost scales with folder size, not match count:

  • maxResults only trims the returned slice — it doesn't bound the scan.
  • startDate/endDate just continue past non-matches — they don't prune enumeration.
  • The only early exit is SEARCH_COLLECTION_CAP (10 000 matches), so a rare-term query never trips it and reads every header (MIME-decoding subject/author/recipients each).
  • A no-folder search recurses every account's root folder = the entire mailstore.
  • refreshImapFolderSync calls folder.updateFolder() on every folder it visits, so an unscoped search also fires an updateFolder storm across the whole tree.

Because the loop never yields, the UI is frozen the whole time; the bridge gives up at its 30 s REQUEST_TIMEOUT while TB keeps churning, and opening a 200k-message msgDatabase + building a large result array can tip a long-lived instance into an OOM/hang.

Fix

Make the scan cooperative and bounded, without changing matching semantics:

  • Yield to the event loop every SEARCH_YIELD_EVERY (2000) enumerated headers via Services.tm.dispatchToMainThread, so the UI stays responsive.
  • Time budget: abort after SEARCH_TIME_BUDGET_MS (20 s, safely under the 30 s bridge timeout) and return partial results flagged { truncated: true, message } instead of hanging or erroring. The same flag is now surfaced when the pre-existing 10 000-match collection cap is hit (previously a silent truncation).
  • Stop the updateFolder storm: only force an IMAP sync on the explicitly requested folder, not on every folder of a recursive/whole-account scan.

searchMessages becomes async (its sole caller already awaits it).

Compatibility

  • Matching behavior is unchanged.
  • Result shape is unchanged for complete searches (bare array without offset, paginated object with offset). Only incomplete searches change shape — they return { messages, truncated: true, message, ... } so the partial-result signal always rides with the data.

Testing

  • node --test test/*.cjs388 tests, 0 failures (17 skipped, pre-existing).
  • node --check clean on api.js and mcp-bridge.cjs.
  • ⚠️ Not runtime-tested inside Thunderbird (the API module needs the live TB/Gloda/msgDatabase environment). Please exercise a large-folder and an unscoped search before merging. Marked draft for that reason.

Possible follow-ups (not in this PR)

  • Route unscoped/large-folder searches through the Gloda index for O(index) instead of O(folder) — deferred because Gloda full-text is token-based and would change substring-match semantics; wanted a behavior-preserving reliability fix first.
  • The same blocking enumerateMessages() pattern appears in getRecentMessages and a few other handlers; they could take the same cooperative treatment.
  • Make the budget/yield thresholds configurable via prefs.

🤖 Generated with Claude Code

searchMessages enumerated message headers synchronously on Thunderbird's
main thread and recursed every subfolder, so a large folder (or an
unscoped whole-account search) froze the UI and ran past the bridge's 30s
request timeout — occasionally taking TB down with it. Cost scaled with
folder size, not match count: maxResults only trimmed the final slice and
date filters didn't prune the scan, so a rare-term query over a 200k-message
folder read every header.

Make the scan cooperative:
- yield to the event loop every SEARCH_YIELD_EVERY (2000) headers via
  Services.tm.dispatchToMainThread so the UI stays responsive
- abort after SEARCH_TIME_BUDGET_MS (20s, under the 30s bridge timeout) and
  return partial results flagged { truncated: true, message } instead of
  hanging or erroring
- only force an IMAP updateFolder sync on the explicitly-requested folder,
  not on every folder of a recursive/whole-account scan (updateFolder storm)

Matching semantics and the result shape on complete searches (bare array /
paginated object) are unchanged. Existing test suite passes (388 tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant