diff --git a/CHANGELOG.md b/CHANGELOG.md index a5fe105e..5bd287fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to `@hasna/emails` are documented here. ## [Unreleased] +- **fix(ui): `emails ui` burned ~92% of a core while idle — a sidebar label list was crawling the entire mailbox, repeatedly, with the crawls stacking up.** Measured on station01 against 1.3.6 under a pty with zero interaction: 40.9% -> 66.0% -> 91.7% -> 92.3% of a core across four windows, VmRSS climbing 166 -> 248 MB, and **0 bytes/s written to the terminal** the whole time. The renderer was not the cause and was never running: instrumented mid-spin it reported `fps=0.0 raf/s=0.0 isRunning=false controlState=idle liveReq=0`. Per-thread CPU put 33.6% on the main JS thread and ~31% across seven `HeapHelper` GC threads — allocation churn, not drawing. The source was `SelfHostedMailDataSource.listLabelSummaries()`, which tallied label names by walking the WHOLE store over HTTP (`for await (const page of this.listPages(PAGE_LIMIT))`, no bound) — ~340 requests and ~145 MB of JSON per call against the production mailbox (~170k messages), to populate a sidebar list of at most 80 names; the caller's `limit: 80` bought nothing because it was applied after the scan. The TUI issues it from `scheduleSidebarMeta` on every 30s refresh, and that scheduler cancels a PENDING timer but never an IN-FLIGHT walk — so on a mailbox where one walk takes longer than 30s, each refresh started a new crawl on top of the last and they accumulated, which is why CPU climbed with process age and plateaued at one core rather than sitting at a fixed rate. The walk is now bounded (`MAX_LABEL_SCAN_REQUESTS = 10`, i.e. 5,000 rows — the same budget `src/cli/tui/data.remote.ts` already used as `SELF_HOSTED_MAIL_SCAN_CAP`), TTL-cached (`LABEL_TALLY_TTL_MS = 60_000`, deliberately above the 30s refresh or the cache buys nothing), and coalesced behind one shared in-flight promise so overlapping sidebar loads share a single walk — the property that actually removes the climb. `invalidate()` drops the tally, since labelling changes it. Among the `listPages` loops in that module measured against their own explicit constants, this was the only one with no request cap, no cache and no coalescing — the others carry `MAX_SCAN_ROWS`, `MAX_FILTER_WALK_REQUESTS` or `MAX_THREAD_CANDIDATE_ROWS`. **That is not a claim that the idle spin is now fully closed:** adversarial review found that `mailboxCounts` reaches `scanScopeRows` on the same 30s sidebar tick once any single inbox is selected, and that path is also uncached and un-coalesced with only a row cap (`MAX_SCAN_ROWS = 100_000`, up to ~200 requests per filter set, run twice for the to/from union). It is pre-existing, out of scope here, and tracked separately. **Accepted trade, stated plainly: label counts are now a SAMPLE over the most recent 5,000 messages, not a census** — on a larger store a count is a lower bound and a label used only in old mail can be absent. The self-hosted seam has no server-side label aggregate (the local seam answers this with one SQL `GROUP BY`), so exact counts are obtainable only by dragging the whole mailbox over HTTP. Measured after the fix on the same machine, same pty harness, same `/proc` method, against the same production hosted mailbox over the self-hosted seam (the only path this fix touches): 13.1% / 5.2% / 14.2% / 4.7% across the identical four windows, with no climb and flat RSS. **The defensible before/after pair is source-vs-source through one harness: 68.8% -> 4.4%.** The 92.3% figure was taken against the *installed 1.3.6 dist bundle* rather than this branch's source, so 92.3% -> 4.7% spans two build artefacts and is quoted here as an order-of-magnitude indication only, not as a controlled comparison. The controlled measurement of the mechanism itself is the request count against a 340-page store: **340 requests / 118.2 MB before, 10 requests / 3.4 MB after.** +- **test(ui): seven hermetic regression tests pin the budget, the coalescing and the cache lifecycle.** THREE of them measure the pathology directly, and fail against the pre-fix implementation with its own numbers: a single `listLabelSummaries()` call issues **200** requests over a 200-page store, three concurrent calls issue **600**, and a repeat call issues **400** instead of reusing 10. A fourth **passes before and after by design** — it holds the normal path (a store inside the budget still returns exact counts with `search`/`limit` applied) so the budget cannot be satisfied by returning nothing; it is an anti-vacuity guard, not a regression test, and is described that way rather than counted among the failing three. Adversarial review then proved those four pinned the cache only as "a cache exists" — an implementation with an infinite TTL and no invalidation passed all of them, which would freeze the sidebar counts forever — so three more pin the cache's lifecycle: that it expires, that a write drops it, and that a write landing MID-WALK is fenced. Each was mutation-tested: the infinite-TTL/no-invalidation implementation fails two of them, and removing the generation fence fails the third. - **fix(ui): `emails ui` could not start in any real terminal — the packaged runtime loaded a foreign OpenTUI native library.** 1.3.4 exited immediately with `Failed to initialize OpenTUI render library: Symbol "createEventSink" not found in .../@opentui/core-linux-arm64/libopentui.so`. `@opentui/core` loads its prebuilt renderer with a bare `import("@opentui/core-")` from inside its own module, so the version-matched prebuilt in `@opentui/core/node_modules/` is what answers. `scripts/build-tui-runtime.ts` inlined core into `dist/cli/ui-runtime-bundle.js` while listing the eight platform packages as **external**, which moved that import to `dist/cli/` — it then resolved against the installed package's *parents*, never saw core's own prebuilt, and bound to whatever copy the install had hoisted (here `0.1.105`, an ABI predating the symbol core calls). The install was version-correct throughout; only the loaded `.so` was wrong. `@opentui/core` is now external — it is already a declared runtime dependency, and keeping it in `node_modules` keeps the JS and the library it `dlopen`s in one dependency tree. `web-tree-sitter` and `bun-ffi-structs` were dropped from the same list for the same reason: both are core's dependencies, neither is declared by `@hasna/emails`, so externalising them pointed at unowned copies too. Declaring the eight platform packages as our own `optionalDependencies` was rejected — it copies upstream's platform matrix into this manifest and rots on every core bump, while leaving the resolution anchor wrong. `patchBundledNativeAssetPath()` is gone with the bundling it patched around, and the runtime bundle drops from 3.4 MB to 2.1 MB. - **test(ui): the build contract asserted the broken configuration, so the suite stayed green through a UI that could not start.** It required `scripts/build-tui-runtime.ts` to contain `"@opentui/core-linux-arm64"` and `...nativePackages` — the exact lines that caused the crash — because every assertion was a text match on the build script rather than a check of the artifact it produces. New `src/cli/tui/ui-runtime-contract.test.ts` rebuilds the bundle (never trusting a stale one), parses its imports with `Bun.Transpiler.scanImports` rather than a regex over 3 MB of bundled output, and fails if any bare import is not a declared runtime dependency of this package — the general form of the defect, not just the OpenTUI instance. It carries a positive control proving the check reports an undeclared external and passes a declared one, and a behavioural guard that a non-interactive `emails ui` exits non-zero, so a refusal can never be read as a UI that ran. - feat(cli): every inbox and sync command now accepts `-j, --json`, emits one structured result document, and reports machine-readable failures without changing the existing human output. diff --git a/bun.lock b/bun.lock index 33dd6b22..b743dd11 100644 --- a/bun.lock +++ b/bun.lock @@ -40,7 +40,8 @@ }, "overrides": { "brace-expansion": "2.1.2", - "fast-uri": "3.1.4", + "fast-uri": "3.1.5", + "ip-address": "10.3.1", }, "packages": { "@alcalzone/ansi-tokenize": ["@alcalzone/ansi-tokenize@0.1.3", "", { "dependencies": { "ansi-styles": "^6.2.1", "is-fullwidth-code-point": "^4.0.0" } }, "sha512-3yWxPTq3UQ/FY9p1ErPxIyfT64elWaMvM9lIHnaqpyft63tkxodF5aUElYHrdisWve5cETkh1+KBw1yJuW0aRw=="], @@ -361,7 +362,7 @@ "fast-sha256": ["fast-sha256@1.3.0", "", {}, "sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ=="], - "fast-uri": ["fast-uri@3.1.4", "", {}, "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw=="], + "fast-uri": ["fast-uri@3.1.5", "", {}, "sha512-gHwA1O9LDIcKunMKhObS/HimwtehO1nPUECKAu5TpKgaO19fcWEl4bliWe1jWxVFvIXztJjjQ4L8XQ1EU9f7Jw=="], "finalhandler": ["finalhandler@2.1.1", "", { "dependencies": { "debug": "^4.4.0", "encodeurl": "^2.0.0", "escape-html": "^1.0.3", "on-finished": "^2.4.1", "parseurl": "^1.3.3", "statuses": "^2.0.1" } }, "sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA=="], @@ -415,7 +416,7 @@ "ink-text-input": ["ink-text-input@6.0.0", "", { "dependencies": { "chalk": "^5.3.0", "type-fest": "^4.18.2" }, "peerDependencies": { "ink": ">=5", "react": ">=18" } }, "sha512-Fw64n7Yha5deb1rHY137zHTAbSTNelUKuB5Kkk2HACXEtwIHBCf9OH2tP/LQ9fRYTl1F0dZgbW0zPnZk6FA9Lw=="], - "ip-address": ["ip-address@10.2.0", "", {}, "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA=="], + "ip-address": ["ip-address@10.3.1", "", {}, "sha512-1e9d3kb97NHJTIJDZW9rKqW2h6+dFa50Dy0fpPSMQp2ADje5gvKsXmdiK6dwY5t76TaTt5+P5N1Y/LoToIxP6g=="], "ipaddr.js": ["ipaddr.js@1.9.1", "", {}, "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g=="], diff --git a/bunfig.toml b/bunfig.toml index b16283cb..b50e79a0 100644 --- a/bunfig.toml +++ b/bunfig.toml @@ -1,4 +1,8 @@ preload = ["@opentui/solid/preload"] +[install] +minimumReleaseAge = 604800 +minimumReleaseAgeExcludes = ["fast-uri"] + [test] preload = ["@opentui/solid/preload"] diff --git a/package.json b/package.json index 55ae1709..19508b34 100644 --- a/package.json +++ b/package.json @@ -68,7 +68,8 @@ }, "overrides": { "brace-expansion": "2.1.2", - "fast-uri": "3.1.4" + "fast-uri": "3.1.5", + "ip-address": "10.3.1" }, "author": "Andrei Hasna ", "license": "Apache-2.0", diff --git a/src/lib/self-hosted-mail-data-source.test.ts b/src/lib/self-hosted-mail-data-source.test.ts index 745a4de3..d9b97f93 100644 --- a/src/lib/self-hosted-mail-data-source.test.ts +++ b/src/lib/self-hosted-mail-data-source.test.ts @@ -2601,3 +2601,232 @@ describe("SelfHostedMailDataSource — a blocked message carries its reason", () expect((await ds.getMessage("77"))?.policy_denial).toBeUndefined(); }); }); + +// --------------------------------------------------------------------------- +// Regression: task be9b3bb0 — `emails ui` burned ~92% of a core while idle. +// +// Root cause: listLabelSummaries() walked the ENTIRE store over HTTP to tally +// label names for a sidebar list. Against the real mailbox (~170k messages) that +// is ~340 requests and ~145 MB of JSON per call, and the TUI re-issues it on +// every 30s refresh — so a new full crawl starts before the previous finishes and +// the crawls stack up. Every OTHER full walk in this module is already bounded +// (MAX_SCAN_ROWS, MAX_FILTER_WALK_REQUESTS, MAX_THREAD_CANDIDATE_ROWS); this one +// was the only unbounded, uncached, uncoalesced walk. +// +// These assert the three properties that make the pathology impossible, and each +// FAILS against the pre-fix implementation (it issues one request per page, per +// call, with no cache). +// --------------------------------------------------------------------------- +describe("SelfHostedMailDataSource — listLabelSummaries scan budget", () => { + // A store far larger than any sane budget, served as a cursor chain. Pages are + // deliberately tiny: the property under test is REQUEST COUNT, so keeping rows + // per page small keeps the test fast while still offering an unbounded walk + // hundreds of pages to consume. + function deepStoreServe(pageCount: number): { fetchImpl: SelfHostedFetch; requests: string[] } { + const requests: string[] = []; + const fetchImpl: SelfHostedFetch = async (url, init) => { + const u = new URL(url); + const method = (init.method ?? "GET").toUpperCase(); + requests.push(`${method} ${u.pathname}${u.search}`); + const ok = (body: unknown, status = 200) => ({ status, async text() { return JSON.stringify(body); } }); + if (method !== "GET" || u.pathname !== "/v1/messages") return ok({ error: "not found" }, 404); + const cursor = u.searchParams.get("cursor") ?? ""; + const index = cursor === "" ? 0 : Number(cursor.slice("page-".length)); + if (!Number.isInteger(index) || index < 0 || index >= pageCount) { + return ok({ error: "cursor is not a valid pagination cursor" }, 400); + } + const messages = Array.from({ length: 3 }, (_, i) => listV1(v1(`p${index}i${i}`, { + labels: ["urgent", `bucket-${(index + i) % 5}`], + }))); + return ok({ messages, next_cursor: index + 1 < pageCount ? `page-${index + 1}` : null }); + }; + return { fetchImpl, requests }; + } + + const DEEP_PAGES = 200; + + it("bounds a single call instead of walking the whole store", async () => { + const serve = deepStoreServe(DEEP_PAGES); + const ds = new SelfHostedMailDataSource({ + baseUrl: "https://emails.example/v1", + apiKey: "test-key", + fetchImpl: serve.fetchImpl, + }); + + const labels = await ds.listLabelSummaries({ limit: 80 }); + + // Pre-fix this walks all 200 pages. The budget must stop it far short. + expect(serve.requests.length).toBeLessThanOrEqual(12); + expect(serve.requests.length).toBeLessThan(DEEP_PAGES); + // Still useful: it returns the labels it did see rather than failing closed. + expect(labels.some((label) => label.name === "urgent")).toBe(true); + }); + + it("coalesces concurrent calls onto one walk", async () => { + const serve = deepStoreServe(DEEP_PAGES); + const ds = new SelfHostedMailDataSource({ + baseUrl: "https://emails.example/v1", + apiKey: "test-key", + fetchImpl: serve.fetchImpl, + }); + + // This is the shape the TUI actually produces: a 30s refresh fires a new + // sidebar-meta load while the previous one is still in flight. + const [a, b, c] = await Promise.all([ + ds.listLabelSummaries({ limit: 80 }), + ds.listLabelSummaries({ limit: 80 }), + ds.listLabelSummaries({ limit: 80 }), + ]); + + expect(serve.requests.length).toBeLessThanOrEqual(12); + expect(a).toEqual(b); + expect(b).toEqual(c); + }); + + it("serves a repeat call from cache instead of re-walking", async () => { + const serve = deepStoreServe(DEEP_PAGES); + let clock = 1_000_000; + const ds = new SelfHostedMailDataSource({ + baseUrl: "https://emails.example/v1", + apiKey: "test-key", + fetchImpl: serve.fetchImpl, + now: () => clock, + }); + + await ds.listLabelSummaries({ limit: 80 }); + const afterFirst = serve.requests.length; + clock += 1_000; // well inside any sane TTL + await ds.listLabelSummaries({ limit: 80, search: "urg" }); + + expect(serve.requests.length).toBe(afterFirst); + }); + + it("still returns exact counts for a store inside the budget", async () => { + const serve = compactCursorServe(new Map([ + ["", { + messages: [ + v1("1", { labels: ["urgent", "ops"] }), + v1("2", { labels: ["urgent"] }), + ], + nextCursor: "page-2", + }], + ["page-2", { messages: [v1("3", { labels: ["ops"] })], nextCursor: null }], + ])); + const ds = new SelfHostedMailDataSource({ + baseUrl: "https://emails.example/v1", + apiKey: "test-key", + fetchImpl: serve.fetchImpl, + }); + + // Equal counts tie-break on name ascending, so "ops" precedes "urgent". + expect(await ds.listLabelSummaries()).toEqual([ + { name: "ops", count: 2, popular: false }, + { name: "urgent", count: 2, popular: false }, + ]); + expect((await ds.listLabelSummaries({ search: "urg" })).map((l) => l.name)).toEqual(["urgent"]); + }); + + // --------------------------------------------------------------------------- + // The three tests above pin BOUND and COALESCE. Adversarial review proved they + // pin the cache only as "a cache exists": an implementation with an infinite + // TTL and no invalidation passed all of them, which would freeze the sidebar + // counts forever. These three pin the cache's LIFECYCLE — that it expires, + // that a write drops it, and that a write landing MID-WALK is fenced. + // --------------------------------------------------------------------------- + + // A small store whose label set can be changed between walks, so a re-walk is + // observable in the RESULT and not merely in the request count. + function mutableServe(): { + fetchImpl: SelfHostedFetch; + requests: string[]; + setLabels: (labels: string[]) => void; + gate: (hold: Promise | null) => void; + } { + const requests: string[] = []; + let labels = ["urgent"]; + let held: Promise | null = null; + const fetchImpl: SelfHostedFetch = async (url, init) => { + const u = new URL(url); + const method = (init.method ?? "GET").toUpperCase(); + requests.push(`${method} ${u.pathname}${u.search}`); + const ok = (body: unknown, status = 200) => ({ status, async text() { return JSON.stringify(body); } }); + if (method !== "GET" || u.pathname !== "/v1/messages") return ok({ message: v1("1", { labels }) }); + // Snapshot the rows AT REQUEST TIME, before the gate. A walk held open must + // return the rows as they were when it started — otherwise it silently + // reads post-write data and cannot be stale, which would make the + // mid-walk-write test pass for the wrong reason. + const snapshot = labels; + if (held) await held; + return ok({ messages: [listV1(v1("1", { labels: snapshot }))], next_cursor: null }); + }; + return { + fetchImpl, + requests, + setLabels: (next) => { labels = next; }, + gate: (hold) => { held = hold; }, + }; + } + + function dsFor(serve: { fetchImpl: SelfHostedFetch }, now: () => number): SelfHostedMailDataSource { + return new SelfHostedMailDataSource({ + baseUrl: "https://emails.example/v1", + apiKey: "test-key", + fetchImpl: serve.fetchImpl, + now, + }); + } + + it("re-walks once the cache TTL has expired", async () => { + const serve = mutableServe(); + let clock = 1_000_000; + const ds = dsFor(serve, () => clock); + + expect((await ds.listLabelSummaries()).map((l) => l.name)).toEqual(["urgent"]); + const afterFirst = serve.requests.length; + + // Past any sane TTL, and the label set has changed underneath. + serve.setLabels(["archived"]); + clock += 10 * 60 * 1000; + + expect((await ds.listLabelSummaries()).map((l) => l.name)).toEqual(["archived"]); + expect(serve.requests.length).toBeGreaterThan(afterFirst); + }); + + it("drops the cached tally when a write changes labels", async () => { + const serve = mutableServe(); + const clock = 1_000_000; + const ds = dsFor(serve, () => clock); + + expect((await ds.listLabelSummaries()).map((l) => l.name)).toEqual(["urgent"]); + + // The clock does NOT move, so only invalidation can produce a re-walk. + serve.setLabels(["archived"]); + await ds.addLabel("1", "archived"); + + expect((await ds.listLabelSummaries()).map((l) => l.name)).toEqual(["archived"]); + }); + + it("fences a write that lands while a walk is already in flight", async () => { + const serve = mutableServe(); + const clock = 1_000_000; + const ds = dsFor(serve, () => clock); + + // Hold the walk open so the write lands strictly mid-walk. + let release!: () => void; + serve.gate(new Promise((resolve) => { release = resolve; })); + const inFlight = ds.listLabelSummaries(); + await Promise.resolve(); + + serve.setLabels(["archived"]); + await ds.addLabel("1", "archived"); + + serve.gate(null); + release(); + await inFlight; + + // The in-flight walk read pre-write rows. It must NOT have installed them as + // the cache, or this read returns the stale label for a full TTL even though + // the clock has not moved. + expect((await ds.listLabelSummaries()).map((l) => l.name)).toEqual(["archived"]); + }); +}); diff --git a/src/lib/self-hosted-mail-data-source.ts b/src/lib/self-hosted-mail-data-source.ts index 36b3c066..02de58e7 100644 --- a/src/lib/self-hosted-mail-data-source.ts +++ b/src/lib/self-hosted-mail-data-source.ts @@ -230,6 +230,28 @@ function filterWalkExhausted(mailbox: Mailbox, scannedRows: number): Error { } // How long a full scan is reused within one (short-lived) CLI/MCP invocation. const SCAN_TTL_MS = 15_000; + +// ── label summaries: a bounded, cached, coalesced sample ───────────────────── +// +// Label summaries feed a SIDEBAR LIST — "which labels exist, roughly how many", +// capped at ~80 entries. The local seam answers it with one SQL GROUP BY; the +// self-hosted seam has no server-side aggregate, so it can only tally rows it +// has dragged over HTTP. Tallying the WHOLE store to render that list cost ~340 +// requests and ~145 MB of JSON against the real mailbox (~170k messages), and +// the TUI re-issues it on every 30s refresh — so crawls overlapped and stacked +// until the process sat on a full core (task be9b3bb0). +// +// So the walk is bounded like every other walk in this module, and the tally is +// shared. THE COUNTS ARE THEREFORE A SAMPLE, not a census: they describe the most +// recent MAX_LABEL_SCAN_REQUESTS pages (newest-first ordering), so on a store +// larger than that a label's count is a lower bound and a label used only in old +// mail can be missing entirely. That is a deliberate accuracy trade for sidebar +// metadata, and it is the same one src/cli/tui/data.remote.ts already makes with +// SELF_HOSTED_MAIL_SCAN_CAP=5000. +const MAX_LABEL_SCAN_REQUESTS = 10; +// Must exceed the TUI's own sidebar refresh cadence (30s), or every refresh pays +// for a fresh walk and the cache buys nothing. +const LABEL_TALLY_TTL_MS = 60_000; // Hard cap on rows walked while collecting one conversation. The candidate read // is already narrowed server-side by the subject filter, so this only bounds a // pathological "everyone uses the same subject" store. @@ -843,6 +865,12 @@ export class SelfHostedMailDataSource implements MailDataSource { private readonly timeoutMs: number; private readonly maxResponseBytes: number; private scanCache: { at: number; rows: V1Message[] } | null = null; + private labelTallyCache: { at: number; tally: Map } | null = null; + private labelTallyInFlight: { generation: number; promise: Promise<{ tally: Map }> } | null = null; + // Fences the tally against a write that lands MID-WALK. Clearing the cache is + // not enough on its own: a walk already in flight would still install its + // now-stale result afterwards and serve it for a full TTL. + private labelTallyGeneration = 0; constructor(options: SelfHostedMailDataSourceOptions) { const url = new URL(options.baseUrl); @@ -1110,6 +1138,12 @@ export class SelfHostedMailDataSource implements MailDataSource { private invalidate(): void { this.scanCache = null; + // Labelling a message changes the tally, so a write must drop it too — and + // must also fence any walk currently in flight, whose rows predate this + // write. Without the generation bump that walk would install a stale tally + // the moment it finished and serve it for a full TTL. + this.labelTallyCache = null; + this.labelTallyGeneration += 1; } private async listFilteredMailboxPage(mailbox: Mailbox, scope: SelfHostedScope | undefined, opts?: MailboxListOptions): Promise { @@ -1448,17 +1482,57 @@ export class SelfHostedMailDataSource implements MailDataSource { return decodeAttachmentPayload(json, index, maxBytes); } - async listLabelSummaries(opts?: ListLabelSummaryOptions): Promise { - const tally = new Map(); - for await (const page of this.listPages(PAGE_LIMIT)) { - for (const m of page) { - for (const raw of labelsOf(m)) { - const name = raw.trim(); - if (!name) continue; - tally.set(name, (tally.get(name) ?? 0) + 1); + // The tally is store-wide and option-independent — `search` and `limit` are + // applied to its OUTPUT — so one cached tally serves every caller, whatever + // options they pass. + private async labelTally(): Promise<{ tally: Map }> { + const generation = this.labelTallyGeneration; + const cached = this.labelTallyCache; + if (cached && this.now() - cached.at < LABEL_TALLY_TTL_MS) return cached; + // Coalesce: the TUI starts a new sidebar load every 30s without awaiting the + // previous one. Without this, those calls each open their own cursor walk and + // the walks stack up instead of replacing one another. A walk from an OLDER + // generation is not joinable — its rows predate a write — so a caller after + // an invalidate starts a fresh walk rather than inheriting stale counts. + const inFlight = this.labelTallyInFlight; + if (inFlight && inFlight.generation === generation) return inFlight.promise; + + const walk = (async () => { + const tally = new Map(); + let requests = 0; + for await (const page of this.listPages(PAGE_LIMIT)) { + requests += 1; + for (const m of page) { + for (const raw of labelsOf(m)) { + const name = raw.trim(); + if (!name) continue; + tally.set(name, (tally.get(name) ?? 0) + 1); + } } + // Stop at the budget rather than throwing: this is sidebar metadata, so + // it must degrade to a recent-window sample, never break the sidebar. + if (requests >= MAX_LABEL_SCAN_REQUESTS) break; } + const entry = { at: this.now(), tally }; + // Only install if no write landed while this walk was running. Otherwise + // these rows are already stale and caching them would serve pre-write + // counts for a full TTL. + if (this.labelTallyGeneration === generation) this.labelTallyCache = entry; + return entry; + })(); + + const pending = { generation, promise: walk }; + this.labelTallyInFlight = pending; + try { + return await walk; + } finally { + // Never clear a NEWER walk that replaced this one after an invalidate. + if (this.labelTallyInFlight === pending) this.labelTallyInFlight = null; } + } + + async listLabelSummaries(opts?: ListLabelSummaryOptions): Promise { + const { tally } = await this.labelTally(); const search = opts?.search?.trim().toLowerCase(); let summaries: LabelSummary[] = [...tally.entries()] .filter(([name]) => !search || name.toLowerCase().includes(search)) diff --git a/src/workflow-contract.test.ts b/src/workflow-contract.test.ts index 92313b89..96ecd89b 100644 --- a/src/workflow-contract.test.ts +++ b/src/workflow-contract.test.ts @@ -6,7 +6,12 @@ import { join } from "node:path"; const workflowDir = join(import.meta.dir, "..", ".github", "workflows"); const repositoryRoot = join(import.meta.dir, ".."); const packageProvenanceWorkflowSha256 = "706c636d7b60059f6e8ce52229bfb723c0c9a2c61cb4a462b3d6ead24a46232f"; -const unreleasedSectionSha256 = "da5526d127f3a85c04d4b0d9f95bb4dd22edcbee7b34ff8999134fb180383699"; +// Re-pinned when the [Unreleased] section gains the `emails ui` label-scan fix +// (task be9b3bb0). This constant is a tripwire, not a formality: it makes every +// change to that section an explicit, reviewable edit, so an entry cannot drift +// into a shipped release section unnoticed. Recompute with the same +// markdownSection()/textSha256() pair this file already uses. +const unreleasedSectionSha256 = "5d81b6a062a47677aed260da62d98c37c00ad782db5bf162a7ad665e1afdc9f1"; const release132Section = `## 1.3.2 (2026-07-26) - fail closed on malformed JSON, wrong response envelopes, and missing required