Skip to content

benchmark.md

Latest commit

 

History

History
98 lines (68 loc) · 2.93 KB

benchmark.md

File metadata and controls

98 lines (68 loc) · 2.93 KB

Performance Benchmarks

Test Environment

  • OS: Windows 11
  • Python: 3.13.9
  • CPU: Intel i7
  • RAM: Available
  • Test Date: October 26, 2025

Measured Performance

Generation Speed by Configuration

Configuration Frames Cells Resolution Time
Small 50 9 128×128 0.92s
Medium 100 25 256×256 3.20s
Large 200 49 512×512 20.40s

Preset Performance

All presets tested with: 50 frames, 9 cells, 128×128 resolution

  • clean: 0.58s
  • low_noise: 0.57s
  • medium_noise: 0.67s
  • high_noise: 0.70s
  • very_high_noise: 0.56s

Observation: Preset choice has minimal impact on generation time. Optical effects are fast.

Important Caveats

Performance Will Vary

These timings are specific to one machine at one point in time. Your performance will depend on:

  • CPU model and speed - Primary factor
  • Available RAM - Affects caching
  • Disk speed - Affects file writing (MP4, TIF)
  • Other running processes - System load
  • Python version - May differ from 3.13.9
  • Dependency versions - NumPy, SciPy versions matter

Linear Scaling Does Not Hold

Do NOT assume that 2x more frames = 2x more time. Computational costs may scale differently:

  • Memory allocation overhead
  • Cache efficiency changes
  • I/O bottlenecks at different scales

Realistic Estimates (With Uncertainty)

Based on measured data, rough estimates for common use cases:

Configuration Estimated Time Confidence
500 frames, 25 cells, 256×256 ~8 minutes Low
1000 frames, 49 cells, 256×256 ~15 minutes Low
5000 frames, 49 cells, 512×512 ~75 minutes Very Low

These are extrapolations. Actual performance may differ significantly.

What Limits Performance

Current implementation bottlenecks (not profiled, estimated):

  1. Gaussian rendering - O(frames × cells × pixels²) computation
  2. Optical effects - PSF convolution per frame, shot noise generation
  3. File I/O - Writing MP4 and TIF files
  4. Memory allocation - NumPy array operations

Potential Optimizations

These have NOT been implemented or tested:

  • Vectorization improvements - Further optimize rendering loops
  • GPU acceleration - Use CuPy for Gaussian rendering
  • Parallel processing - Multiprocessing for frame generation
  • Caching - Reuse PSF kernels, pre-allocate arrays
  • Streaming - Write output files incrementally

Running Benchmarks

Run benchmark tests yourself:

pixi run pytest tests/test_benchmark.py -v -s

This will show timing on your machine, which is more relevant than these results.

Recommendation for Researchers

  • Start small: Test with 100 frames, 9 cells, 128×128
  • Measure your case: Run benchmarks on your hardware
  • Don't extrapolate: Don't assume linear scaling
  • Plan accordingly: Allow buffer time for processing