Skip to content

run benchmarks serially#1368

Closed
penelopeysm wants to merge 1 commit into
mainfrom
py/bench
Closed

run benchmarks serially#1368
penelopeysm wants to merge 1 commit into
mainfrom
py/bench

Conversation

@penelopeysm
Copy link
Copy Markdown
Member

@penelopeysm penelopeysm commented Apr 24, 2026

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 24, 2026

Benchmark Report

  • this PR's head: 74e9c8bf26ebb029a494496e34618ed1d62e589c
  • base branch: 81a245a63bcb9192ffb4ed936e9ca6612137c5e2

Computer Information

Julia Version 1.11.9
Commit 53a02c0720c (2026-02-06 00:27 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 9V74 80-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

┌───────────────────────┬───────┬─────────────┬────────┬───────────────────────────────┬─────────────────────────────┬───────────────────────────────────┐
│                       │       │             │        │       t(eval) / t(ref)        │      t(grad) / t(eval)      │         t(grad) / t(ref)          │
│                       │       │             │        │ ─────────┬──────────┬──────── │ ───────┬──────────┬──────── │ ──────────┬─────────────┬──────── │
│                 Model │   Dim │  AD Backend │ Linked │     base │  this PR │ speedup │   base │  this PR │ speedup │      base │     this PR │ speedup │
├───────────────────────┼───────┼─────────────┼────────┼──────────┼──────────┼─────────┼────────┼──────────┼─────────┼───────────┼─────────────┼─────────┤
│               Dynamic │    10 │    mooncake │   true │   269.90 │   263.16 │    1.03 │   6.38 │     7.17 │    0.89 │   1721.26 │     1887.11 │    0.91 │
│                   LDA │    12 │ reversediff │   true │  2502.48 │  2388.05 │    1.05 │   2.09 │     2.14 │    0.97 │   5225.75 │     5120.92 │    1.02 │
│   Loop univariate 10k │ 10000 │    mooncake │   true │ 30298.94 │ 28773.63 │    1.05 │   6.77 │     7.10 │    0.95 │ 205194.75 │   204333.84 │    1.00 │
├───────────────────────┼───────┼─────────────┼────────┼──────────┼──────────┼─────────┼────────┼──────────┼─────────┼───────────┼─────────────┼─────────┤
│    Loop univariate 1k │  1000 │    mooncake │   true │  3797.22 │  3798.11 │    1.00 │   5.45 │     5.42 │    1.01 │  20680.67 │    20567.99 │    1.01 │
│      Multivariate 10k │ 10000 │    mooncake │   true │ 40962.92 │ 41711.67 │    0.98 │   6.99 │     6.86 │    1.02 │ 286376.82 │   285979.73 │    1.00 │
│       Multivariate 1k │  1000 │    mooncake │   true │  3948.51 │  4071.77 │    0.97 │   7.89 │     7.58 │    1.04 │  31150.43 │    30845.03 │    1.01 │
├───────────────────────┼───────┼─────────────┼────────┼──────────┼──────────┼─────────┼────────┼──────────┼─────────┼───────────┼─────────────┼─────────┤
│ Simple assume observe │     1 │ forwarddiff │  false │     0.84 │     0.84 │    1.00 │  11.24 │    11.29 │    1.00 │      9.42 │        9.46 │    1.00 │
│           Smorgasbord │   201 │ forwarddiff │  false │   890.18 │   864.41 │    1.03 │  81.69 │ 24473.65 │    0.00 │  72718.49 │ 21155313.82 │    0.00 │
│           Smorgasbord │   201 │      enzyme │   true │  1192.99 │  1203.18 │    0.99 │   4.57 │     4.55 │    1.00 │   5449.62 │     5472.70 │    1.00 │
├───────────────────────┼───────┼─────────────┼────────┼──────────┼──────────┼─────────┼────────┼──────────┼─────────┼───────────┼─────────────┼─────────┤
│           Smorgasbord │   201 │ forwarddiff │   true │  1269.96 │  1639.39 │    0.77 │  76.72 │    57.52 │    1.33 │  97425.73 │    94296.93 │    1.03 │
│           Smorgasbord │   201 │    mooncake │   true │  1233.61 │  1204.12 │    1.02 │   4.49 │     4.57 │    0.98 │   5540.97 │     5505.09 │    1.01 │
│           Smorgasbord │   201 │ reversediff │   true │  1253.56 │  1245.38 │    1.01 │ 119.97 │   120.08 │    1.00 │ 150384.76 │   149545.44 │    1.01 │
├───────────────────────┼───────┼─────────────┼────────┼──────────┼──────────┼─────────┼────────┼──────────┼─────────┼───────────┼─────────────┼─────────┤
│              Submodel │     1 │    mooncake │   true │     0.84 │     0.83 │    1.01 │  30.62 │    33.61 │    0.91 │     25.67 │       27.98 │    0.92 │
└───────────────────────┴───────┴─────────────┴────────┴──────────┴──────────┴─────────┴────────┴──────────┴─────────┴───────────┴─────────────┴─────────┘

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.35%. Comparing base (81a245a) to head (74e9c8b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1368   +/-   ##
=======================================
  Coverage   82.35%   82.35%           
=======================================
  Files          49       49           
  Lines        3502     3502           
=======================================
  Hits         2884     2884           
  Misses        618      618           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@penelopeysm
Copy link
Copy Markdown
Member Author

That obviously did not help at all with benchmark stability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant