Skip to content

fix(ui): remember a scoped count walk that failed closed, so a broken scope stops re-walking every 30s - #206

Open
andrei-hasna wants to merge 1 commit into
mainfrom
fix/41529344-mailbox-count-runaway
Open

fix(ui): remember a scoped count walk that failed closed, so a broken scope stops re-walking every 30s#206
andrei-hasna wants to merge 1 commit into
mainfrom
fix/41529344-mailbox-count-runaway

Conversation

@andrei-hasna

@andrei-hasna andrei-hasna commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What this is, and what it is not

#201/#202 fixed the mailbox-count runaway. This is not a second fix for that — it is the residual those PRs left behind, measured.

Bounding the walk made one walk finite. It did not make the sequence of walks finite. Only a walk that completes reaches the scopedCountsCache.set at the end of the walk body, so a scope that trips the bound is remembered nowhere: the TUI catches the throw into lastError, reschedules the sidebar 30s later, and pays the whole walk again.

Measured, against the pre-fix code

The regression added here, run before the fix:

Expected: <= 1
Received: 201

201 further requests on every 30s refresh, indefinitely.

This is the steady state on the real mailbox, not an edge case

Measured on production (emails.hasna.xyz, shipped 1.3.9, no local patch):

emails inbox status
  -> 174,493 total / 174,120 inbox            (server aggregate, 1 request)

emails inbox list --to andrei@hasna.com --offset 99000 --limit 1 --json
  -> rc=1, "scanned 99579 rows over 200 requests without completing", 78s

CONTROL, rare address:
emails inbox list --to zz-no-such-mailbox-41529344@example.invalid --limit 1 --json
  -> rc=0, rows returned: 0, 1.88s

The control is what makes 99,579 meaningful. Had the serve ignored ?to=, the rare address would have walked the whole store for ~78s too. It returned in 1.88s, so ?to= is honoured server-side and those rows are genuinely one address's mail — still arriving at ~498 per page when the walk bound stopped it 421 rows short of MAX_SCAN_ROWS.

So the primary address cannot complete a scoped count. This path always fails, and before this PR it re-failed at full cost twice a minute.

The change

  • Remember a failed walk in scopedCountsFailureCache, keyed by scope like the success cache.
  • 15 minutes, against the 60s a success gets. The failure is structural — the store's size against a compile-time constant — so it cannot resolve on a count's timescale, and re-deriving it costs the maximum walk the bound allows rather than a typical one.
  • Fenced by the same write generation as the success cache, and cleared by invalidate(), so a write that makes the scope countable again is reflected at once rather than after the window.
  • The original error object is re-thrown rather than a reworded one: its text already names both causes and both remedies, and that advice is exactly as true the second time.

Steady state: 201 requests/30s -> 201 requests/15min, about 30x.

Tests

Three added, all in SelfHostedMailDataSource — scoped mailboxCounts scan budget:

test guards
does not re-walk a scope whose count walk already failed closed the defect. Fails on pre-fix code with Received: 201
retries an exhausted scope once the failure has aged out the remedy does not become a permanent lockout
re-counts a failed scope immediately after a write the write fence

The third passed before the fix too (nothing was cached), so it was mutation-tested rather than trusted: removing scopedCountsFailureCache.clear() from invalidate() turns it red (1 fail), and it was restored.

bun test src/lib/self-hosted-mail-data-source.test.ts -> 127 pass, 0 fail (baseline before this branch: 124 pass, 0 fail).
bunx tsc --noEmit -> rc=0.

What this does NOT do

It makes the broken state cheap; it does not un-break the counts. A single bounded walk is still 201 requests, so any 3-minute window containing a retry still sees ~201 — the steady-state average lands at ~40 per 3 min, but the retry window does not.

Removing the walk needs a server-side recipient filter. /v1/messages/counts and /v1/messages/groups already accept ?domain= and answer from an exact aggregate; they accept no ?to=/?from=, which is precisely why an address scope walks. A domain scope has that aggregate available today and the client walks anyway.

Filed with full evidence as todos 3ae0181e.

Refs: todos 41529344


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

… scope stops re-walking every 30s

#201/#202 bounded the scoped folder-count walk, which made ONE walk finite. It
did not make the SEQUENCE of walks finite. Only a walk that COMPLETES reaches
the cache write at the end of the walk body, so a scope that trips the bound was
remembered nowhere: the TUI catches the throw into `lastError`, reschedules the
sidebar 30s later, and pays the whole walk again.

Measured by the regression added here, against the pre-fix code:

    Expected: <= 1
    Received: 201

That is 201 further requests on every 30s refresh, indefinitely.

It is the STEADY STATE on the real mailbox rather than an edge case. Measured on
production (emails.hasna.xyz, shipped 1.3.9, no local patch):

    emails inbox status                          174,493 total / 174,120 inbox
    emails inbox list --to andrei@hasna.com \
      --offset 99000 --limit 1                   rc=1, "scanned 99579 rows over
                                                 200 requests without completing"
    CONTROL, rare address:
    emails inbox list --to zz-no-such-mailbox-41529344@example.invalid --limit 1
                                                 rc=0, 0 rows, 1.88s

The control is what makes 99,579 meaningful: had the serve ignored `?to=`, the
rare address would have walked the whole store too. It returned in 1.88s against
78s, so `?to=` is honoured server-side and those rows are genuinely one address's
mail, still arriving ~498 per page when the walk stopped 421 rows short of
MAX_SCAN_ROWS. So the primary address cannot complete a scoped count, and this
path always fails.

The failure is remembered for 15 minutes rather than the 60s a success gets: it
is structural — the store's size against a compile-time constant — so it cannot
resolve on a count's timescale, and re-deriving it costs the MAXIMUM walk the
bound allows rather than a typical one. It is fenced by the same write generation
as the success cache and cleared by invalidate(), so a write that makes the scope
countable again is reflected at once rather than after the window.

Steady state goes from 201 requests/30s to 201 requests/15min, about 30x.

WHAT THIS DOES NOT DO: it makes the broken state cheap, it does not un-break the
counts. A single bounded walk is still 201 requests, so any 3-minute window
containing a retry sees ~201. Removing the walk needs a server-side recipient
filter on /v1/messages/counts, which already accepts ?domain= and no ?to=/?from=;
filed as todos 3ae0181e.

Refs: todos 41529344

Agent: agent-ceo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant