Skip to content

benchmark: client scenarios #7631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Sheraff wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

benchmark: client scenarios #7631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
aee0167
benchmark: client scenarios
Sheraff Jun 14, 2026
c9ac1b8
restructure apps
Sheraff Jun 14, 2026
5d68edf
format
Sheraff Jun 14, 2026
d25c48c
remove sharding
Sheraff Jun 14, 2026
adf893e
some cleanup
Sheraff Jun 14, 2026
e0f0428
i have fully lost control at this point
Sheraff Jun 14, 2026
bd2f943
Merge branch 'main' into benchmark-client-nav-scenarios
Sheraff Jun 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
51 changes: 51 additions & 0 deletions benchmarks/client-nav/FEATURES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Client CPU Benchmark Feature Inventory

This document records the client-side router features considered for CPU benchmark coverage while expanding `@benchmarks/client-nav`. It is not an implementation plan. Scenario implementation plans live in `plan/`.

## Current Coverage

The current `benchmarks/client-nav` benchmark is one mixed navigation loop per framework. It already exercises basic client navigation, `Link` clicks, imperative `router.navigate`, route matching, params parsing and stringifying, search validation, search middleware, one loader with `loaderDeps`, one `beforeLoad`, active link state, route hooks, selectors, and `scrollRestoration: true`.

The weakness is attribution. A regression in any of those areas moves one benchmark, so the future benchmark suite should keep the current mixed loop as a baseline and add isolated scenario apps like `benchmarks/ssr/scenarios/*`.

## Included Feature Areas

| Feature area | Scenario plan | Why it gains from CPU coverage |
| ---------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Existing mixed client navigation | `01-baseline-navigation.md` | Preserves CodSpeed continuity and keeps a broad smoke signal while isolated scenarios are added. |
| Route matching and path params | `02-route-matching.md` | Matching large route trees, static/dynamic/splat precedence, params parse/stringify, and not-found fallthrough are hot in every navigation. |
| Location building, links, active state, and match checks | `03-location-building-links.md` | Large menus repeatedly call `buildLocation`, resolve relative paths, compute active state, and run `MatchRoute` checks. |
| Search params, validation, middleware, and structural sharing | `04-search-params.md` | URL search state is a core TanStack Router feature with parse, stringify, validation, deep equality, and middleware costs. |
| `beforeLoad`, route context, and context invalidation | `05-before-load-context.md` | Client navigation serializes `beforeLoad` work and merges context top-down before loaders and components consume it. |
| Loader cache, `loaderDeps`, invalidation, and stale reload modes | `06-loader-cache.md` | Client loader caching is a major runtime responsibility and has cache-hit, cache-miss, stale, background reload, and blocking reload paths. |
| Preloading and preload promotion | `07-preloading.md` | Manual and link-driven preloads perform matching, `beforeLoad`, loader work, dedupe, cache retention, and promotion to navigation. |
| Hook subscribers, selectors, stores, and structural sharing | `08-subscribers-selectors.md` | Framework bindings depend on fine-grained subscriptions to keep render work bounded during frequent location changes. |
| Nested outlets, route lifecycle callbacks, and remount deps | `09-outlets-remounts.md` | Deep route rendering and remount decisions affect client CPU independently from route matching and loaders. |
| Client control-flow paths | `10-control-flow.md` | Redirects, not-found, validation errors, loader errors, and error boundaries alter match resolution and commit paths. |
| Interrupted navigations and abort handling | `11-interrupted-navigations.md` | Superseded navigations stress abort controllers, pending match cleanup, and loader resolution ordering. |
| Scroll restoration and hash navigation | `12-scroll-restoration.md` | Scroll restoration tracks scroll containers, cache keys, reset behavior, and hash/top scrolling during navigation. |
| Route masking, rewrites, basepath, and trailing slash handling | `13-masking-rewrites.md` | Public/internal href transforms and temp-location state add location-building and navigation overhead. |
| Client head management | `14-head-management.md` | Nested `head`, `links`, and `scripts` evaluation plus DOM dedupe happens on client navigation. |
| Deferred loader data and `Await` | `15-deferred-await.md` | Deferred promises exercise non-blocking loader results, suspense/await resolution, and follow-up route rendering. |
| History integration, router events, and navigation blockers | `16-history-events-blockers.md` | Back/forward, event subscribers, and blockers are client-only runtime paths not isolated by the existing mixed loop. |
| Client hydration and resume | `17-hydration-resume.md` | Hydrated matches, dehydrated loader data, skipped first-load work, deferred placeholders, and first post-hydration navigation are client CPU paths not covered by SSR request loops. |

## Considered But Not Included As Standalone Scenarios

| Feature | Decision | Reason |
| ----------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| TanStack Start SSR, server functions, server routes, middleware, streaming, and serialization | Excluded | These are server-side responsibilities already covered by `benchmarks/ssr` or memory benchmarks. |
| Memory retention shapes such as mount/unmount, unique-location churn, and loader-data retention | Excluded from CPU plans | They are already covered by `benchmarks/memory/client`; CPU scenarios should avoid duplicating memory-only signals unless the same feature has meaningful CPU work. |
| Type-safety helpers, `linkOptions`, route type utilities, and TypeScript inference | Excluded | They are compile-time features, not client runtime CPU work. |
| File-route generation, virtual route generation, and bundler plugin transforms | Excluded | They are build-time features. Runtime file-route output can still be used by scenario apps. |
| Cold dynamic import and automatic code-splitting cost | Excluded as a standalone CPU scenario | Node module caching and Vite chunk loading make cold import timing noisy under repeated benchmark invocations. Deterministic warm lazy route and component preload paths are included in `07-preloading.md`. |
| External libraries such as TanStack Query, Zod, Valibot, ArkType, and UI libraries | Excluded | Their CPU costs belong mostly to those libraries. Router scenarios may use lightweight in-house validators instead. |
| Devtools integration | Excluded | Devtools are development-only and should not affect production client CPU benchmarks. |
| Real browser painting, layout, CSS, assets, and image loading | Excluded | The client benchmark runs in jsdom and should not pretend to measure browser rendering. |
| `document.startViewTransition` and view-transition animation work | Excluded | The API is browser-dominated and unavailable or synthetic in jsdom; correctness tests are a better fit. |
| `reloadDocument`, external URL navigation, and full page unload | Excluded | They intentionally leave the SPA router runtime and are not useful CPU loops. |
| Native `beforeunload` prompts and `window.confirm` UX | Excluded | Browser dialogs cannot be benchmarked meaningfully in jsdom. Logical blockers are included in `16-history-events-blockers.md`. |
| Protocol/security warnings for dangerous URLs | Excluded | Rare defensive paths should stay in correctness tests. |
| Real IntersectionObserver viewport behavior | Excluded as a browser API | The preloading scenario can use a deterministic polyfill or direct preload calls to cover router preload work without depending on real viewport observation. |
| Accessibility focus management | Excluded | Not a primary TanStack Router runtime responsibility and difficult to measure accurately in jsdom. |
| Actual network fetch latency | Excluded | Scenarios should use deterministic synchronous or controlled async loaders so router CPU dominates. |
73 changes: 73 additions & 0 deletions benchmarks/client-nav/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,59 @@ Cross-framework client-side navigation benchmarks for:
- `react/` - React benchmark + Vitest config
- `solid/` - Solid benchmark + Vitest config
- `vue/` - Vue benchmark + Vitest config
- `scenarios/<scenario>/<framework>/` - isolated scenario apps added after the shared harness lands
- `bench-utils.ts` - stable tinybench options and deterministic seeded helpers
- `benchmark.ts` - shared workload contract for scenario bench files
- `lifecycle.ts` - shared mount/action/wait/teardown harness for scenario setup files

## Scenario Contract

The top-level `react/`, `solid/`, and `vue/` apps remain the baseline benchmarks
for CodSpeed continuity. New feature coverage should live under
`scenarios/<scenario>/<framework>/` and use stable project names of the form
`@benchmarks/client-nav-<scenario>-<framework>`.

Each scenario app exports `mountTestApp(container)` and returns
`{ router, unmount }`; `unmount` should be safe to call twice. Scenario setup
files import the built app from `./dist/app.js` with
`await import(/* @vite-ignore */ appModulePath)` and can use
`createClientNavLifecycle` from `#client-nav/lifecycle` for idempotent mount,
navigation waits, cleanup registration, and teardown.

Scenario `vite.config.ts` files should set their root to the framework directory,
build library mode from `./src/app.tsx`, and write `./dist/app.js`. Scenario
projects should expose `build:client`, `build:client:flame`, `test:flame`, and
`test:types:client` targets; flame targets should set `parallelism: false`.

Scenario `speed.bench.ts` files must run `await workload.sanity()` before
`describe(...)`, then register both Vitest `beforeAll`/`afterAll` hooks and
tinybench `setup`/`teardown` options so CodSpeed and plain `vitest bench` both
exercise setup and teardown.

## Scenario Responsibilities

The current aggregate targets include the baseline apps and every completed
scenario through plan 17.

| Scenario | Client-side responsibility |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| Baseline apps | Broad mixed navigation loop preserving current CodSpeed continuity. |
| `route-matching` | Large route-tree matching, route sorting, params parse/stringify, splat and optional segments. |
| `location-building-links` | `buildLocation`, `Link`, active state, relative targets, hash/search/state preservation, and match checks. |
| `search-params` | Search parse/stringify, validation, middleware, deep equality, custom serialization, and structural sharing. |
| `before-load-context` | Serial `beforeLoad`, context merge, context consumers, and context invalidation. |
| `loader-cache` | Loaders, `loaderDeps`, fresh/stale cache paths, `shouldReload`, stale reload modes, and invalidation. |
| `preloading` | Manual/link preloads, dedupe, component preload hooks, and preload-to-navigation promotion. |
| `subscribers-selectors` | Framework hook subscriptions, selectors, router stores, and structural sharing. |
| `outlets-remounts` | Deep outlets, lifecycle callbacks, and remount deps. |
| `control-flow` | Redirects, not-found, validation errors, loader errors, error boundaries, and unmatched routes. |
| `interrupted-navigations` | Superseded navigations, abort signals, pending match cleanup, and final commit ordering. |
| `scroll-restoration` | Scroll cache bookkeeping, nested scroll areas, hash scrolling, and reset behavior. |
| `masking-rewrites` | Route masks, URL rewrites, basepath, public hrefs, and trailing slash handling. |
| `head-management` | Nested route head evaluation, DOM head updates, and dedupe. |
| `deferred-await` | Deferred loader data, fallback rendering, controlled promise resolution, and follow-up renders. |
| `history-events-blockers` | History push/back/forward, router events, and logical navigation blockers. |
| `hydration-resume` | Client hydration of dehydrated matches, skipped first-load loader work, deferred placeholders, and first post-hydration navigation. |

## Run

Expand All @@ -28,6 +81,14 @@ CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:solid --output
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:vue --outputStyle=stream --skipRemoteCache
```

Build a framework's baseline app and completed scenario apps:

```bash
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:build:react --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:build:solid --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:build:vue --outputStyle=stream --skipRemoteCache
```

Run framework-specific flame benchmarks (10 second loop, profiled with `@platformatic/flame`, forced to `NODE_ENV=production`):

```bash
Expand All @@ -41,3 +102,15 @@ Typecheck benchmark sources:
```bash
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:types --outputStyle=stream --skipRemoteCache
```

Run a completed scenario target directly by project name when debugging a single
scenario:

```bash
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-route-matching-react:build:client --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-route-matching-react:test:types:client --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-route-matching-react:test:flame --outputStyle=stream --skipRemoteCache
```

Replace `route-matching` and `react` in the project name with any completed
scenario and framework pair listed in the responsibility table.
18 changes: 18 additions & 0 deletions benchmarks/client-nav/bench-utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
export const clientNavBenchOptions = {
warmupIterations: 100,
time: 10_000,
throws: true,
}

export function createDeterministicRandom(seed: number) {
let state = seed >>> 0

return () => {
state = (state * 1664525 + 1013904223) >>> 0
return state / 0x100000000
}
}

export function randomSegment(random: () => number) {
return Math.floor(random() * 1_000_000_000).toString(36)
}
7 changes: 7 additions & 0 deletions benchmarks/client-nav/benchmark.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export interface ClientNavWorkload {
name: string
before: () => Promise<void> | void
run: () => Promise<void> | void
sanity: () => Promise<void> | void
after: () => Promise<void> | void
}
Loading
Loading