Skip to content

fix(web): page the whole patch catalog so >200 patches can be approved (#3157) - #3195

Merged
ToddHebebrand merged 4 commits into
mainfrom
fix/3157-patches-paginate-all
Aug 7, 2026
Merged

fix(web): page the whole patch catalog so >200 patches can be approved (#3157)#3195
ToddHebebrand merged 4 commits into
mainfrom
fix/3157-patches-paginate-all

Conversation

@ToddHebebrand

Copy link
Copy Markdown
Collaborator

Problem

The patches page issued exactly one GET /patches?limit=200 request. 200 is MAX_PAGE_LIMIT in apps/api/src/routes/patches/helpers.ts — the API's per-page ceiling — and PatchList filters, sorts and paginates entirely client-side over whatever that single response contained.

Everything past the first 200 rows therefore never reached the browser at all: it could not be searched, sorted, paged to, selected, or approved. A reporter with 333 Linux patches saw a single page of 200 and was left with 133 permanently unapproved.

Not Linux-specific. The same code path serves every OS; it surfaces on Linux first because apt/dnf catalogs produce far more discrete patch entries per fleet than Windows does.

Where the ceiling was not

Worth stating explicitly, because the reporter's framing ("should display several pages") pointed at the UI pager and the fix could easily have landed in the wrong layer:

  • Not an API page cap. GET /patches and GET /update-rings/:id/patches both already return pagination: { page, limit, total } and honour ?page=.
  • Not a bulk-approve cap. bulkApproveSchema puts no .max() on patchIds, and the handler loops the whole array.
  • Not the client-side pager. It pages correctly — over a list that was already truncated before it got there.

The ceiling was purely the client's single-page fetch.

Fix

  • PatchesPage walks every page until the reported total is exhausted, for both the all-rings and ring-scoped endpoints. Page 1 keeps its bare ?limit=200 URL, so the request is byte-identical for any catalog that fits in one page.
  • The walk stops at 25 pages (5,000 patches) and renders a notice naming the shown and total counts. An unbounded walk would hammer the API and put an arbitrarily large array in memory — but an invisible cap is the bug being fixed here, so the new cap is never silent.
  • A generation counter discards a superseded walk. Switching rings mid-walk could otherwise let the abandoned walk finish last and repaint the list with the previous ring's patches. This race pre-existed, but a walk is now up to 25 sequential requests instead of one, which widens the window enough to matter.
  • "Select all N matching". The header checkbox only ever covered the visible page, so a multi-page result set needed one select-all click per page (14 of them at the default page size of 25). The bulk toolbar now offers a one-click select over the full filtered set. Added to all 7 locales.
  • Stale comments in PatchList.tsx and routes/patches/list.ts that described the old fixed-200 fetch are updated — both are load-bearing warnings for whoever later pushes sorting server-side.

Verification

Suite Result
apps/web patches components (9 files) 101 passed
apps/web i18n parity + gates 196 passed
apps/api routes/patches/ (5 files) 72 passed
astro check (1528 files) 0 errors
tsc --noEmit (api) clean

New coverage:

  • PatchesPage paging — all-rings walk, ring-scoped walk, cap + truncation notice, single request when the response carries no pagination metadata, a later-page failure surfacing an error rather than a partial list, and the superseded-walk discard.
  • PatchList select-all-matching — appears only past one page, feeds the full id set to bulk approve, respects the active filter, absent when everything fits on one page.
  • API bulk-approve at scale — 333 ids in one request asserting all 333 upserts actually fire, the audit entry records the full count, and per-id failures are reported rather than aborting the batch. This pins the server side so a well-meaning .max(200) on the schema can't silently re-create the ceiling after the client fix.

All seven behavioural tests were confirmed to fail against the pre-fix code (reverted the walk and the matching-id wiring, re-ran: 7 failed / 39 passed).

Follow-up (not in this PR)

The scalable end state is server-side filter/sort/select-all, which would let the page walk go away entirely. The API already accepts sortBy/sortDir; the web still doesn't send them. Note the severity-sort divergence flagged in PatchList.tsx before wiring that up — the client sorts severity semantically, the API alphabetically.

Closes #3157

🤖 Generated with Claude Code

#3157)

The patches page issued exactly one `GET /patches?limit=200` request. 200 is
MAX_PAGE_LIMIT in apps/api/src/routes/patches/helpers.ts, the API's per-page
ceiling, and PatchList filters, sorts and paginates entirely client-side over
whatever that single response contained. Anything past the first 200 rows
therefore never reached the browser: it could not be searched, sorted, paged
to, selected, or approved. A reporter with 333 Linux patches saw one page of
200 and was stuck with the remaining 133 unapproved.

Neither server endpoint was at fault. `GET /patches` and
`GET /update-rings/:id/patches` both return `pagination: { page, limit, total }`,
and `bulkApproveSchema` puts no max on `patchIds` — the ceiling was purely the
client's single-page fetch.

- PatchesPage walks every page until the reported total is exhausted, for both
  the all-rings and ring-scoped endpoints. Page 1 keeps its bare `?limit=200`
  URL so the request is unchanged for catalogs that fit in one page.
- The walk stops at 25 pages (5,000 patches) and renders a notice naming the
  shown and total counts. An invisible cap is the bug being fixed here, so the
  new cap is never silent.
- A generation counter discards a superseded walk. Switching rings mid-walk
  could otherwise let the abandoned walk finish last and repaint the list with
  the previous ring's patches — a much wider window now that a walk is up to 25
  sequential requests.
- The header checkbox only ever covered the visible page, so a multi-page
  result set needed one select-all click per page. The bulk toolbar now offers
  "Select all N matching" over the full filtered set (all 7 locales).

Tests: PatchesPage paging (all-rings, ring-scoped, cap + notice, no-pagination
single request, later-page failure surfaces an error rather than a partial
list, superseded-walk discard); PatchList select-all-matching (appears only
past one page, feeds bulk approve, respects the active filter); API
bulk-approve with 333 ids asserting all 333 upserts fire, the audit count is
right, and per-id failures are reported rather than aborting the batch. All
seven fail against the pre-fix code.

Closes #3157

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 6, 2026

Copy link
Copy Markdown

Deploying breeze with  Cloudflare Pages  Cloudflare Pages

Latest commit: 04aee7a
Status: ✅  Deploy successful!
Preview URL: https://843b1d4c.breeze-9te.pages.dev
Branch Preview URL: https://fix-3157-patches-paginate-al.breeze-9te.pages.dev

View logs

Todd Hebebrand and others added 2 commits August 6, 2026 13:50
…prove (#3157)

Addresses the /pr-review-toolkit:review-pr findings on the page-walk fix.

CRITICAL — offset paging over a tied sort key. Three reviewers independently
flagged that `GET /patches` and `GET /update-rings/:id/patches` order by
`created_at` alone. `patches.created_at` is `defaultNow()` and agent ingest
runs inside `db.transaction`, so Postgres `now()` gives every patch in one
scan report a byte-identical timestamp — the reporter's 333 Linux patches are
almost certainly one such block. LIMIT/OFFSET over a key with mass ties has no
defined order between two queries, so the new walk could return some rows twice
and never return others: #3157's exact symptom, hidden behind a correct-looking
row count. Both endpoints now append `patches.id` as a unique tiebreaker.
Harmless before this PR (only page 1 was ever fetched); load-bearing now.

Also from review:

- Dedupe rows by id during the walk, and flag the list as short whenever fewer
  distinct rows arrive than the server's reported total. This one check now
  covers the page cap, duplicate rows, and a mid-walk anomaly.
- A missing-pagination or empty page is only treated as "that's the whole
  answer" on page 1. From page 2 on, page 1 already promised more, so it's an
  anomaly and the list is reported as incomplete rather than silently short.
- PatchList's error state only rendered when the list was EMPTY, and a failed
  walk deliberately keeps the previous array — so a failed refresh (switching
  rings, Refresh, a mid-walk failure) showed stale rows, possibly from another
  ring, with no indication at all. Added a persistent banner with retry for the
  stale-list case.
- Batch bulk-approve at 200 ids per request. The route upserts sequentially, so
  one request for a 5,000-id selection scales linearly with it; a proxy timeout
  would abandon a partly-committed batch, and the route writes its audit entry
  after the loop, so those approvals would land with no audit record. Batching
  bounds each request and gives each one an audit row.
- A 200 whose body can't be parsed no longer optimistically marks every
  submitted id approved — that asserted a result we don't have. Reported as
  indeterminate instead.
- The truncation notice told users to narrow with filters; every filter on this
  page is client-side over the already-truncated array, so that could never
  surface the missing tail. Reworded honestly across all 7 locales.
- Log per-id failures in the bulk-approve loop. The bare `catch` made an RLS
  denial, a conflict-target mismatch and a dropped connection indistinguishable
  to an operator asked why 40 of 333 patches failed.
- Comment fixes: the "walks every page" claims now name the 5,000 cap, the
  severity sort-divergence warning notes that past the cap the client ranks a
  different subset, and `usePatchSelection`'s first parameter is renamed
  `pageIdList` to match what callers pass.

Tests: end-to-end 333-patch walk → select-all-matching → approve asserting all
333 distinct ids reach the API in two batches (the only test spanning all three
legs); dedupe; page-2 anomaly; source counts; stale-list error banner; and two
API tests pinning the id tiebreaker for both default and explicit sorts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ing at (#3157)

Second review round. The completeness check added in the previous commit fires
for three causes but reused the page-cap wording for all of them, and the two
new causes are the common ones: under `createdAt DESC` new rows land at the
front, so any agent ingesting patches mid-walk slides the offset window and the
dedupe drops a few rows. A tenant with 800 patches would then read "the rest
can't be reached from this view — use the patches API" for a gap that a reload
fixes, sending an operator out of the product for nothing.

- The notice now carries its cause. `cap` keeps the API-referral copy (a reload
  genuinely cannot help); `shifted` says the catalog changed while loading and
  offers a reload button. New `patchesPage.shiftedNotice` in all 7 locales.
- Stop clearing the notice at walk start. It's committed alongside `patches`
  now, so a failed refresh leaves the notice consistent with the stale list it
  describes instead of dropping "this list is incomplete" while keeping the
  incomplete list on screen.
- A later batch aborting no longer discards per-id rejections that earlier
  batches already reported — "40 of your first 200 were refused" was vanishing
  behind the generic abort message.

Tests: the dedupe and page-2-anomaly cases now assert cause `shifted` plus the
reload affordance, the cap case asserts cause `cap` and the absence of that
affordance, and a new test covers rejections surviving a mid-run batch abort.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ToddHebebrand

Copy link
Copy Markdown
Collaborator Author

Review run: /pr-review-toolkit:review-pr — round 1 with code-reviewer, pr-test-analyzer, silent-failure-hunter, comment-analyzer (parallel); round 2 with code-reviewer scoped to the round-1 fix commit.

Findings: 12 consequential in round 1 + 3 in round 2 → all addressed in e8ee308c8 and 9ebac0e64; 0 outstanding.

The one that mattered — three agents independently flagged the same critical bug in my original fix. Both endpoints order by created_at alone with LIMIT/OFFSET. patches.created_at is defaultNow() and agent ingest runs inside db.transaction, so Postgres now() gives an entire scan report one identical timestamp — the reporter's 333 Linux patches are almost certainly a single such block. Paging over a key with mass ties has no defined order between queries, so the page walk could return some rows twice and never return others: #3157's exact symptom, hidden behind a correct-looking row count. Harmless before this PR (only page 1 was ever fetched), load-bearing the moment the client pages. Fixed by appending patches.id as a unique tiebreaker in both routes/patches/list.ts and routes/updateRings.ts, pinned by two API tests.

Also fixed from review:

  • PatchList's error state only rendered when the list was empty, and a failed walk deliberately keeps the previous array — so a failed refresh showed stale rows, possibly from a different ring, with no indication at all. Added a stale-list banner with retry.
  • Bulk-approve is now batched at 200 ids/request. The route upserts sequentially and writes its audit entry after the loop, so a proxy timeout on a 5,000-id request would have committed approvals with no audit record.
  • A missing-pagination or empty page is only "that's everything" on page 1; from page 2 it's an anomaly and the list is reported incomplete.
  • Rows are deduped by id, and a short list is never presented as complete.
  • The incomplete-list notice now distinguishes its cause: cap (a reload can't help → use the API) vs shifted (the catalog changed mid-walk → reload button). The original copy told everyone to "narrow with filters", which can never surface the tail since every filter here is client-side over the already-loaded array.
  • A 200 whose body can't be parsed no longer optimistically marks every id approved; per-id rejections from earlier batches survive a later batch aborting; the bare catch in the API loop now logs.
  • Comment fixes: the "walks every page" claims name the 5,000 cap, and the severity sort-divergence warning notes that past the cap the client ranks a different subset.

Tests: apps/web patches + i18n parity + mutation gates — 304 passed (17 files). apps/api patches + updateRings — 89 passed (6 files). astro check 0 errors (1528 files); tsc --noEmit clean. New coverage includes an end-to-end test (333 patches walk in → select-all-matching → all 333 distinct ids reach the API in two batches) that the test analyst correctly noted was the missing regression net — the three suites each covered one leg and never met. All behavioural tests were confirmed to fail against the pre-fix code.

Deliberately not in scope (flagged by review, worth separate issues):

  • handleBulkDecline still issues one POST /patches/:id/decline per id — "Select all N matching" can now put thousands of sequential browser requests behind one click. A bulk-decline endpoint mirroring bulk-approve would close the asymmetry.
  • Every page of the walk recomputes count(*) plus a GROUP BY source over the whole catalog; page 1's total already sizes the walk.
  • Server-side filter/sort/select-all remains the scalable end state that would remove the page walk entirely.

CI: the pull_request trigger dropped for this branch — gh pr checks showed only a Cloudflare Pages entry with zero Actions runs, which reads deceptively green. Dispatched manually: run 31127388127 against 9ebac0e64. Confirm that run is green before merging rather than trusting the checks list.

Status: review-clean, local verification green, awaiting maintainer merge.

@ToddHebebrand ToddHebebrand reopened this Aug 7, 2026
ToddHebebrand added a commit that referenced this pull request Aug 7, 2026
…ivy on all open PRs (#3212)

## Why

**GHSA-5p4m-2wfm-xmqj / CVE-2026-59870** — quadratic CPU consumption in
js-yaml's `!!omap` resolution (3.x and 4.x), rated **HIGH**, fixed in
**4.3.1** / 3.15.1.

The advisory entered Trivy's vulnerability DB at ~02:25 UTC on
2026-08-07. From that moment every PR whose scan resolved after the DB
refresh went red on **both** `Trivy Filesystem Scan` and `Trivy Image
Scan` (Web image). PRs scanned before the refresh are still green, which
is why the failure looked selective rather than global — it is not. This
blocks all six remaining v0.104 PRs (#3184, #3185, #3186, #3194, #3195,
#3196) and will block every future PR and main until it lands.

These are genuine scan-step failures, not the GitHub Actions outage that
hit the earlier batch — the jobs fail at `Run blocking Trivy filesystem
scan` / `Scan Web image`, with every prior step green.

## What

One `pnpm.overrides` entry. js-yaml 4.3.0 arrives transitively through
the Astro / expressive-code docs toolchain; nothing in the repo depends
on it directly.

The override is **upper-bounded to `<5.0.0`**. Without that bound it
resolves to 5.2.2, a major-version jump for `@astrojs/markdown-remark`,
`@astrojs/starlight` and `@expressive-code/core`. Bounding it keeps the
change a 4.3.0 → 4.3.1 patch bump, matching the convention already used
for `undici` and `@babel/core`.

The tree's other two js-yaml copies need no action: **3.15.1** is
already the fixed 3.x release named in the advisory, and **5.2.2** is
unaffected.

## Note on the diff

The lockfile carries one hunk unrelated to js-yaml: `anymatch@3.1.3`'s
picomatch pin moves 4.0.5 → 4.0.4. That is pre-existing drift between
`package.json` and the committed lockfile on main which a fresh resolve
normalizes — not something this change introduces. Left as the resolver
produced it rather than hand-editing the lockfile into an inconsistent
state.

Co-authored-by: Todd Hebebrand <todd@lanternops.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@ToddHebebrand
ToddHebebrand merged commit 732bbce into main Aug 7, 2026
55 checks passed
@ToddHebebrand
ToddHebebrand deleted the fix/3157-patches-paginate-all branch August 7, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Breeze 0.102 doesn't allow to approve more than 200 patches on Linux

1 participant