Skip to content

fix(search): self-describing pagination cursors so load-more can't diverge from first-load#57

Merged
tompscanlan merged 4 commits into
flo-bit:mainfrom
tompscanlan:fix/om-7dbs-self-describing-cursor
Jul 7, 2026
Merged

fix(search): self-describing pagination cursors so load-more can't diverge from first-load#57
tompscanlan merged 4 commits into
flo-bit:mainfrom
tompscanlan:fix/om-7dbs-self-describing-cursor

Conversation

@tompscanlan

@tompscanlan tompscanlan commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Problem

loadMoreEvents picked its pagination backend implicitly: if a search term was set and a Meili backend was configured it paginated via Meilisearch (cursor = a numeric offset), otherwise via D1 listRecords (cursor = a base64url keyset).

That breaks whenever a page's first load comes from one backend but its load-more resolves to the other — the two cursor formats are incompatible:

  • D1 keyset → Meili: Number(base64url)NaN → offset 0 → a relevance-reordered duplicate of page 1.
  • Meili offset → D1 listRecords: the offset is ignored and the discoverable / time-bound filters the first page applied get silently dropped, so later pages drift into past / non-discoverable events.

The topics page tripped exactly this; the search page only avoided it by being end-to-end consistent (Meili+Meili, or cursor:null in its D1 fallback).

Fix — self-describing cursors

Tag every client-facing cursor with the backend that issued it, and route load-more by that tag instead of re-deriving the backend from request shape.

  • New lib/contrail/cursor.tstagCursor(backend, raw) / parseCursor(cursor). Prefix scheme meili: / d1:; the : separator is collision-free because a base64url keyset and a decimal offset both exclude :. A null/empty raw cursor stays null (never manufactures a cursor). Untagged input → { backend: null, raw } for the legacy fallback.
  • events-load-more.ts — routes on the cursor's tag. d1-tagged → D1 pipeline even under search + configured Meili; meili-tagged → Meili; untagged → the old inference (for cursors in flight across the deploy). A meili-tagged cursor with no usable search context fails safe to { events: [], cursor: null } rather than restarting page 1 on the wrong backend. D1 output cursors are d1-tagged; the inbound tag is stripped before the D1 read.
  • search/server/query.tshydrateToPage emits a meili:-tagged offset; parseOffsetCursor accepts a tagged or bare offset and maps a d1-tagged (or NaN) cursor to offset 0 (clean restart, never NaN-from-base64url).
  • First-page routes (events, p/[actor]/hosting, p/[actor]/past-events) — emit d1-tagged first-page cursors and untag any inbound deep-link cursor before the D1 read.
  • search + topics — keep cursor:null (their fetchParams carry no pipeline, so a resumable D1 cursor would still drop the discoverable + startsAtMin filters). Deep pagination for these is deferred to a separate task. Stale "re-routes to Meili whenever a backend is configured" comments were rewritten to describe tag routing.

Verification

  • pnpm test (apps/web, vitest): 183 passed / 4 skipped (187 total), 21 files — green.
  • pnpm run check (svelte-check): 0 errors (7 pre-existing style warnings in unrelated files).
  • New unit tests cover: tag round-trip for both cursor kinds; both divergence directions (D1 first page + search + Meili → stays D1 with filters intact; Meili first page → stays Meili); the two fail-safe cases; and the legacy untagged-cursor path.

Branch is 4 commits ahead of main, 0 behind; merges cleanly.

Introduce tagCursor/parseCursor: a cursor handed to the client is prefixed
with the backend that issued it (meili: / d1:). The base64url D1 keyset and a
decimal Meili offset both exclude ':', so the first ':' is an unambiguous tag
separator. Untagged values parse to backend:null so callers can fall back to
the old inference for in-flight pre-deploy cursors.
runEventSearchPage/runNearMePage now emit a meili:-tagged offset, and
parseOffsetCursor accepts both the tagged cursor and a bare legacy offset. A
d1-tagged keyset reaching here resolves to offset 0 (clean restart) instead of
Number(base64url)->NaN. First-page search results therefore emit a tagged
cursor for free.
…e (om-7dbs)

runLoadMoreEvents now honors the cursor's own backend tag: a meili: cursor stays
on Meili, a d1: cursor stays on D1 even when a search term + configured Meili
backend are present (PR flo-bit#49's topics trip case). A meili: cursor with no usable
Meili context fails safe by ending pagination rather than restarting page 1 on
D1 or NaN-parsing. Untagged legacy cursors fall back to the old
search-set-AND-Meili-configured inference. The D1 keyset returned to the client
is now d1:-tagged.
…ents (om-7dbs)

Home events, profile hosting/past-events first pages now emit d1:-tagged cursors
and untag any inbound (deep-link) cursor before the D1 read, so first-load and
load-more can't diverge. Search D1 fallback and topics pages keep their null
cursor (no pipeline in fetchParams to resume through); their comments are
updated to describe the new tag-based routing and defer re-enabling pagination
to om-47ak.
@tompscanlan tompscanlan marked this pull request as ready for review July 4, 2026 22:50
@tompscanlan tompscanlan merged commit 79ef7e2 into flo-bit:main Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant