Skip to content

[P2-33] Establish reproducible rendering performance baselines and gates #38

Description

@phranck

Problem

Current performance tests use absolute Date thresholds on shared CI runners, often print statistics without enforceable regression criteria, and do not exercise a full AppRunner/RenderLoop path. ForEach benchmarks currently measure empty output. Claimed write reductions and syscall behavior are not tied to reproducible fixtures.

Timing the incorrect String/ANSI renderer before the correctness migration would lock in misleading baselines.

Proposed solution

  • Run release-build benchmark fixtures only after correctness/output assertions pass.
  • Record deterministic work counters for body, measure, render, commit, surface cells, allocations, terminal writes, and wakeups.
  • Add representative small/large keyed collections, Unicode/style compositing, resize, focus/input, cached trees, images, and idle-app scenarios.
  • Use repeated samples with median/P95 and versioned baselines on controlled or scheduled runners.
  • Prefer algorithmic work/allocation counters in pull requests; reserve noisy wall-clock gates for stable benchmark infrastructure.

Acceptance criteria

  • No benchmark can pass while rendering empty or incorrect output.
  • Baselines cover full runtime commits, not only isolated helpers.
  • Idle CPU/wakeup count, memory growth, large keyed collection scaling, layout/compositing, and terminal-write volume have explicit regression limits.
  • Results record platform, architecture, compiler, build mode, fixture revision, and variance.
  • Performance claims in documentation link to a reproducible benchmark.
  • Release benchmarks and all Swift 6.0 quality gates complete without warnings/errors.

Dependencies

Runs after correctness/runtime #6/#8-#15, surface #11, keyed/lazy rendering #12/#25, cache #14, and scheduler #26.

Parallelization

Fixture design and counters can be prepared earlier. Enforced baselines must wait for the final correctness architecture to avoid benchmarking known-broken paths.

Commit structure

  • Split this issue into small, thematic, independently revertible commits wherever the work can remain coherent.
  • Every commit must build and keep its applicable tests/gates green. Do not commit an intentionally failing regression test; use local/known-issue characterization or land the test with the smallest fix.
  • Keep characterization/fixtures, mechanical renames or moves, semantic changes, and documentation/migration updates separate when each step remains green.
  • The issue boundary is not a commit boundary; multiple commits are expected for independently reversible changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: mediumImportant follow-up after critical and high dependencies

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions