Skip to content

Optimize benchmarks#1760

Merged
preinlein merged 1 commit intomainfrom
paul.reinlein/optimize-benchmarks
Feb 13, 2026
Merged

Optimize benchmarks#1760
preinlein merged 1 commit intomainfrom
paul.reinlein/optimize-benchmarks

Conversation

@preinlein
Copy link
Contributor

@preinlein preinlein commented Feb 11, 2026

What does this PR do?

Improves benchmark performance and accuracy by implementing iter_batched for all payload generation benchmarks. This approach properly separates setup costs from the actual benchmark measurements, ensuring more accurate throughput metrics.

Motivation

The previous benchmark implementation was mixing setup costs (RNG initialization, payload generator creation, buffer allocation) with the actual serialization costs being measured. Using BatchSize::PerIteration with iter_batched ensures we're only measuring the actual serialization performance.

Additional Notes

  • Reduced setup benchmark measurement time from 10s to 5s to speed up the benchmark suite
  • Removed 1000 MiB benchmarks for some of the more complex payload types (OpenTelemetry, Fluent, Trace Agent) to keep benchmark runtime reasonable
  • Renamed benchmark groups from *_all to *_throughput to better reflect what's being measured
  • Fixed a trailing semicolon inconsistency in the setup benchmarks

Copy link
Contributor Author

preinlein commented Feb 11, 2026

@preinlein
Copy link
Contributor Author

The stack can be validated by looking at the final Claude hunt results here: https://drive.google.com/file/d/1mbgFAFuC5s2HwM3IhoNCgqZcmt7Ho7a2/view?usp=sharing

@preinlein preinlein marked this pull request as ready for review February 11, 2026 18:18
@preinlein preinlein requested a review from a team as a code owner February 11, 2026 18:18
@preinlein preinlein force-pushed the paul.reinlein/optimize-benchmarks branch from 573530c to f0dff40 Compare February 12, 2026 16:09
@preinlein preinlein force-pushed the paul.reinlein/split-benchmarks branch from 5485811 to fe00bff Compare February 12, 2026 16:09
Base automatically changed from paul.reinlein/split-benchmarks to main February 13, 2026 13:59
@preinlein preinlein force-pushed the paul.reinlein/optimize-benchmarks branch from f0dff40 to 7316dc0 Compare February 13, 2026 14:00
@preinlein preinlein enabled auto-merge (squash) February 13, 2026 14:01
@preinlein preinlein merged commit ab9ef9e into main Feb 13, 2026
31 checks passed
@preinlein preinlein deleted the paul.reinlein/optimize-benchmarks branch February 13, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants