Skip to content

Add more project to benchmarks set - #14

Merged
blishko merged 1 commit into
mainfrom
add-benchmarks
Aug 5, 2026
Merged

Add more project to benchmarks set #14
blishko merged 1 commit into
mainfrom
add-benchmarks

Conversation

@rodiazet

Copy link
Copy Markdown
Contributor

Safe, ERC-4337 EntryPoint, SP1 verifier, Permit2, Universal Router, Aave v3-origin, and Optimism contracts-bedrock

Extends the compile-time benchmark suite with 7 real-world projects extracted directly (2 via Hardhat build-info, 5 via forge extract), rounding out coverage of wallet/account-abstraction, zk-verifier, and large monorepo shapes not previously represented.

Comment thread benchmark_data/benchmarks.toml
@rodiazet
rodiazet requested a review from blishko July 30, 2026 07:21

@msooseth msooseth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. But we probably do want to cut down on stuff eventually, or our benchmark set becomes too slow. Not sure what to cut down on, but we will need to do it :)

@blishko

blishko commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

LGTM. But we probably do want to cut down on stuff eventually, or our benchmark set becomes too slow. Not sure what to cut down on, but we will need to do it :)

I think at the moment we don't want to cut anything. We want to have comprehensive evaluation.
However, we also need a fast subset to get quick feedback.

@rodiazet

rodiazet commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

LGTM. But we probably do want to cut down on stuff eventually, or our benchmark set becomes too slow. Not sure what to cut down on, but we will need to do it :)

I think at the moment we don't want to cut anything. We want to have comprehensive evaluation. However, we also need a fast subset to get quick feedback.

15 fast tier tests are added in the following PR which extract fast subsets of already added projects by this PR

…er, Permit2, Universal Router, Aave v3-origin, and Optimism contracts-bedrock

Extends the compile-time benchmark suite with 7 real-world projects
extracted directly (2 via Hardhat build-info, 5 via forge extract),
rounding out coverage of wallet/account-abstraction, zk-verifier,
and large monorepo shapes not previously represented.
@blishko
blishko merged commit f60f6ca into main Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants