Conversation
b0d2fd4 to
35de010
Compare
Benchmark results Main vs HEAD.Base
Head
Base
Head
Base
Head
Base
Head
Base
Head
Base
Head
|
Benchmarking resultsBenchmark for program
|
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
11.333 ± 0.051 | 11.240 | 11.397 | 5.73 ± 0.13 |
cairo-native (embedded AOT) |
1.979 ± 0.045 | 1.928 | 2.072 | 1.00 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
2.041 ± 0.025 | 2.002 | 2.094 | 1.03 ± 0.03 |
Benchmark for program dict_snapshot
Open benchmarks
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
532.5 ± 5.3 | 526.1 | 542.6 | 1.00 |
cairo-native (embedded AOT) |
1655.2 ± 14.6 | 1630.8 | 1671.2 | 3.11 ± 0.04 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
1761.4 ± 12.3 | 1739.5 | 1779.5 | 3.31 ± 0.04 |
Benchmark for program factorial_2M
Open benchmarks
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
4.940 ± 0.040 | 4.898 | 5.023 | 2.35 ± 0.03 |
cairo-native (embedded AOT) |
2.100 ± 0.022 | 2.072 | 2.131 | 1.00 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
2.163 ± 0.012 | 2.146 | 2.186 | 1.03 ± 0.01 |
Benchmark for program fib_2M
Open benchmarks
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
4.827 ± 0.053 | 4.785 | 4.961 | 2.91 ± 0.05 |
cairo-native (embedded AOT) |
1.658 ± 0.022 | 1.625 | 1.688 | 1.00 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
1.709 ± 0.017 | 1.688 | 1.735 | 1.03 ± 0.02 |
Benchmark for program linear_search
Open benchmarks
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
587.5 ± 10.2 | 574.7 | 608.3 | 1.00 |
cairo-native (embedded AOT) |
1702.8 ± 28.3 | 1669.6 | 1744.9 | 2.90 ± 0.07 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
1825.3 ± 19.2 | 1799.1 | 1869.3 | 3.11 ± 0.06 |
Benchmark for program logistic_map
Open benchmarks
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
Cairo-vm (Rust, Cairo 1) |
488.9 ± 5.6 | 482.4 | 501.5 | 1.00 |
cairo-native (embedded AOT) |
1821.8 ± 18.8 | 1800.3 | 1855.3 | 3.73 ± 0.06 |
cairo-native (embedded JIT using LLVM's ORC Engine) |
1985.7 ± 17.9 | 1957.4 | 2020.0 | 4.06 ± 0.06 |
35de010 to
77ffb9b
Compare
orizi
left a comment
There was a problem hiding this comment.
@orizi reviewed 3 files and all commit messages, and made 1 comment.
Reviewable status: 3 of 9 files reviewed, 1 unresolved discussion (waiting on TomerStarkware).
Makefile line 194 at r1 (raw file):
pull-external-projects: python3 ./scripts/pull_external_projects.py v${CAIRO_2_VERSION} python3 ./scripts/extract_e2e_sierra.py
this sounds not like a part of this
Code quote:
python3 ./scripts/extract_e2e_sierra.py
orizi
left a comment
There was a problem hiding this comment.
@orizi made 3 comments.
Reviewable status: 3 of 9 files reviewed, 4 unresolved discussions (waiting on TomerStarkware).
src/types/circuit.rs line 114 at r1 (raw file):
info: WithSelf<InfoOnlyConcreteType>, ) -> Result<Type<'ctx>> { if metadata.get::<ReallocBindingsMeta>().is_none() {
get_or_insert_with?
src/types/circuit.rs line 218 at r1 (raw file):
info: WithSelf<InfoOnlyConcreteType>, ) -> Result<Type<'ctx>> { if metadata.get::<ReallocBindingsMeta>().is_none() {
get_or_insert_with?
src/types/circuit.rs line 313 at r1 (raw file):
info: WithSelf<InfoOnlyConcreteType>, ) -> Result<Type<'ctx>> { if metadata.get::<ReallocBindingsMeta>().is_none() {
get_or_insert_with?
orizi
left a comment
There was a problem hiding this comment.
@orizi reviewed 1 file and made 1 comment.
Reviewable status: 4 of 9 files reviewed, 5 unresolved discussions (waiting on TomerStarkware).
tests/tests/e2e_libfuncs.rs line 13 at r1 (raw file):
if !sierra_dir.exists() { eprintln!( "Skipping e2e libfunc tests: {} not found. Run 'make pull-external-projects' first.",
add e2e_sierra as a non-phony target in the makefile as well.
orizi
left a comment
There was a problem hiding this comment.
@orizi made 1 comment.
Reviewable status: 4 of 9 files reviewed, 6 unresolved discussions (waiting on TomerStarkware).
tests/tests/e2e_libfuncs.rs line 28 at r1 (raw file):
.sort_by_file_name() .into_iter() {
already happens implicitly.
Suggestion:
for entry in walkdir::WalkDir::new(&sierra_dir)
.sort_by_file_name()
{77ffb9b to
f9de09e
Compare
TomerStarkware
left a comment
There was a problem hiding this comment.
@TomerStarkware made 6 comments.
Reviewable status: 4 of 9 files reviewed, 6 unresolved discussions (waiting on orizi).
Makefile line 194 at r1 (raw file):
Previously, orizi wrote…
this sounds not like a part of this
Done.
src/types/circuit.rs line 114 at r1 (raw file):
Previously, orizi wrote…
get_or_insert_with?
Done.
src/types/circuit.rs line 218 at r1 (raw file):
Previously, orizi wrote…
get_or_insert_with?
Done.
src/types/circuit.rs line 313 at r1 (raw file):
Previously, orizi wrote…
get_or_insert_with?
Done.
tests/tests/e2e_libfuncs.rs line 13 at r1 (raw file):
Previously, orizi wrote…
add e2e_sierra as a non-phony target in the makefile as well.
Why not phony like pull-external-projects:
tests/tests/e2e_libfuncs.rs line 28 at r1 (raw file):
Previously, orizi wrote…
already happens implicitly.
Done.
f9de09e to
085f051
Compare
orizi
left a comment
There was a problem hiding this comment.
@orizi reviewed 6 files and all commit messages, made 1 comment, and resolved 4 discussions.
Reviewable status: 8 of 54 files reviewed, 3 unresolved discussions (waiting on TomerStarkware).
test_data/e2e_libfuncs_raw/array line 2 at r2 (raw file):
//! > array_new libfunc
?
085f051 to
1e1f6f1
Compare
orizi
left a comment
There was a problem hiding this comment.
@orizi reviewed 1 file and all commit messages, made 2 comments, and resolved 1 discussion.
Reviewable status: 9 of 54 files reviewed, 2 unresolved discussions (waiting on TomerStarkware).
test_data/e2e_libfuncs_raw/array line 2 at r2 (raw file):
Previously, orizi wrote…
?
assuming all this files where added by mistake - so add their dir to .gitignore - as well as put them in a more temporarily named dir.
tests/tests/e2e_libfuncs.rs line 13 at r1 (raw file):
Previously, TomerStarkware wrote…
Why not phony like pull-external-projects:
i don't understand what you mean - i just mean that these should be generated by requirement.
Added e2e tests from cairo repo
Closes #NA
Introduces Breaking Changes?
No.
This change is