fix(ui): remember a scoped count walk that failed closed, so a broken scope stops re-walking every 30s - #206
Open
andrei-hasna wants to merge 1 commit into
Open
fix(ui): remember a scoped count walk that failed closed, so a broken scope stops re-walking every 30s#206andrei-hasna wants to merge 1 commit into
andrei-hasna wants to merge 1 commit into
Conversation
… scope stops re-walking every 30s #201/#202 bounded the scoped folder-count walk, which made ONE walk finite. It did not make the SEQUENCE of walks finite. Only a walk that COMPLETES reaches the cache write at the end of the walk body, so a scope that trips the bound was remembered nowhere: the TUI catches the throw into `lastError`, reschedules the sidebar 30s later, and pays the whole walk again. Measured by the regression added here, against the pre-fix code: Expected: <= 1 Received: 201 That is 201 further requests on every 30s refresh, indefinitely. It is the STEADY STATE on the real mailbox rather than an edge case. Measured on production (emails.hasna.xyz, shipped 1.3.9, no local patch): emails inbox status 174,493 total / 174,120 inbox emails inbox list --to andrei@hasna.com \ --offset 99000 --limit 1 rc=1, "scanned 99579 rows over 200 requests without completing" CONTROL, rare address: emails inbox list --to zz-no-such-mailbox-41529344@example.invalid --limit 1 rc=0, 0 rows, 1.88s The control is what makes 99,579 meaningful: had the serve ignored `?to=`, the rare address would have walked the whole store too. It returned in 1.88s against 78s, so `?to=` is honoured server-side and those rows are genuinely one address's mail, still arriving ~498 per page when the walk stopped 421 rows short of MAX_SCAN_ROWS. So the primary address cannot complete a scoped count, and this path always fails. The failure is remembered for 15 minutes rather than the 60s a success gets: it is structural — the store's size against a compile-time constant — so it cannot resolve on a count's timescale, and re-deriving it costs the MAXIMUM walk the bound allows rather than a typical one. It is fenced by the same write generation as the success cache and cleared by invalidate(), so a write that makes the scope countable again is reflected at once rather than after the window. Steady state goes from 201 requests/30s to 201 requests/15min, about 30x. WHAT THIS DOES NOT DO: it makes the broken state cheap, it does not un-break the counts. A single bounded walk is still 201 requests, so any 3-minute window containing a retry sees ~201. Removing the walk needs a server-side recipient filter on /v1/messages/counts, which already accepts ?domain= and no ?to=/?from=; filed as todos 3ae0181e. Refs: todos 41529344 Agent: agent-ceo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is, and what it is not
#201/#202fixed the mailbox-count runaway. This is not a second fix for that — it is the residual those PRs left behind, measured.Bounding the walk made one walk finite. It did not make the sequence of walks finite. Only a walk that completes reaches the
scopedCountsCache.setat the end of the walk body, so a scope that trips the bound is remembered nowhere: the TUI catches the throw intolastError, reschedules the sidebar 30s later, and pays the whole walk again.Measured, against the pre-fix code
The regression added here, run before the fix:
201 further requests on every 30s refresh, indefinitely.
This is the steady state on the real mailbox, not an edge case
Measured on production (
emails.hasna.xyz, shipped 1.3.9, no local patch):The control is what makes 99,579 meaningful. Had the serve ignored
?to=, the rare address would have walked the whole store for ~78s too. It returned in 1.88s, so?to=is honoured server-side and those rows are genuinely one address's mail — still arriving at ~498 per page when the walk bound stopped it 421 rows short ofMAX_SCAN_ROWS.So the primary address cannot complete a scoped count. This path always fails, and before this PR it re-failed at full cost twice a minute.
The change
scopedCountsFailureCache, keyed by scope like the success cache.invalidate(), so a write that makes the scope countable again is reflected at once rather than after the window.Steady state: 201 requests/30s -> 201 requests/15min, about 30x.
Tests
Three added, all in
SelfHostedMailDataSource — scoped mailboxCounts scan budget:does not re-walk a scope whose count walk already failed closedReceived: 201retries an exhausted scope once the failure has aged outre-counts a failed scope immediately after a writeThe third passed before the fix too (nothing was cached), so it was mutation-tested rather than trusted: removing
scopedCountsFailureCache.clear()frominvalidate()turns it red (1 fail), and it was restored.bun test src/lib/self-hosted-mail-data-source.test.ts-> 127 pass, 0 fail (baseline before this branch: 124 pass, 0 fail).bunx tsc --noEmit-> rc=0.What this does NOT do
It makes the broken state cheap; it does not un-break the counts. A single bounded walk is still 201 requests, so any 3-minute window containing a retry still sees ~201 — the steady-state average lands at ~40 per 3 min, but the retry window does not.
Removing the walk needs a server-side recipient filter.
/v1/messages/countsand/v1/messages/groupsalready accept?domain=and answer from an exact aggregate; they accept no?to=/?from=, which is precisely why an address scope walks. A domain scope has that aggregate available today and the client walks anyway.Filed with full evidence as todos
3ae0181e.Refs: todos
41529344Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.