bench: Add MVP design and implementation outline#197
Conversation
|
|
||
| Before program termination, the data structure is logged. | ||
|
|
||
| #### Variant 2: `zkevm-proverjs` helpers cannot maintain state |
There was a problem hiding this comment.
We should be able to maintain state during the execution. E.g. see https://github.com/0xPolygonHermez/zkevm-proverjs/blob/07d92991778be82f66d8be55a3862d2843007221/src/sm/sm_main/sm_main_exec.js#L2751
| Histograms allow to get a quick overview of which instructions and labels are executed how often. The histograms will be generated as SVG files using the [`svg` crate](https://crates.io/crates/svg). Having visualizations as SVGs provides a number of advantages: | ||
|
|
||
| - No dependencies required as SVG files can be viewed in a web browser. | ||
| - They can be embedded in Github comments, markdown files and other documents. |
There was a problem hiding this comment.
Another requirement here would be to make it easy to experiment with different visualizations. Looking through the API of SVG crate, it is quite low-level. So we would need to build tools on top to do even simple visualizations like histograms.
A simpler alternative might be to take a dedicated visualization library e.g. Seaborn/Matplotlib in Python and use the high-level API to build the visualizations we want. For example, snippet for histogram is very short: https://seaborn.pydata.org/examples/faceted_histogram.html
To make this graph easy to view on GitHub, you can save it as "png" right from Seaborn: https://www.statology.org/save-seaborn-plot/
There was a problem hiding this comment.
Also, in case you are not very familiar with Seaborn/Matplotlib, llms are very effective in writing code using this libraries (=
aborg-dev
left a comment
aborg-dev
left a comment
There was a problem hiding this comment.
Looks good to me! Left 2 suggestions
3 participants
Adds a document describing the benchmarking MVP design and giving an overview of how to implement it.
This is rather a temporary document which is not intended to be merged into
masterand hence the writing and structure are more on the level of a draft.Reviews are requested to find out if:
Any other feedback is welcome as well.
This PR is opened against the
mooori/bench-mvpbranch. My plan would be to have that as temporary branch where the MVP is built.Related to #182. After building and using the MVP we should be in a better position to finalize the design of the benchmarking infrastructure.