Skip to content

add cache vs origin benchmark harness#17

Merged
SISUinSea merged 1 commit into
mainfrom
chore/demo-bench-ci-readme
Mar 18, 2026
Merged

add cache vs origin benchmark harness#17
SISUinSea merged 1 commit into
mainfrom
chore/demo-bench-ci-readme

Conversation

@SISUinSea
Copy link
Copy Markdown
Collaborator

What changed

  • Replaced the old simulated benchmark helper with a self-contained cache-vs-origin benchmark harness under tests/benchmark/*.
  • Added an in-process dummy origin ASGI app that simulates a slow upstream fetch with fixed latency and deterministic payloads.
  • Added a Mini Redis HTTP cache benchmark path that uses the real /v1/keys/* contract through app.main.create_app().
  • Updated the benchmark CLI script to compare without_cache and with_redis_cache, including total time, average latency, origin fetch count, cache hits/misses, speedup, and improvement percentage.
  • Rewrote benchmark tests to cover warmup behavior, cache miss-to-hit flow, payload consistency, state reset between runs, and summary formatting.

Why

  • The previous benchmark only simulated a local cache function and did not exercise Mini Redis itself.
  • For the demo, we need a short and repeatable benchmark that shows the relative difference between calling a slow origin directly and serving the same hot key through Mini Redis as a cache.
  • This keeps protocol and store semantics unchanged while giving presenters a concrete benchmark flow that matches the branch scope.

Tests

  • automated:
    • ./.venv/bin/python -m pytest tests/benchmark -q
    • ./.venv/bin/python -m ruff check .
    • ./.venv/bin/python -m ruff format --check .
    • ./.venv/bin/python -m mypy app
    • ./.venv/bin/python -m pytest
  • manual smoke:
    • ./.venv/bin/python tests/benchmark/run_benchmark_demo.py --iterations 20 --warmup 1 --origin-latency-ms 20

Docs updated

  • None in this slice.
  • Shared docs, README, CI, and smoke artifacts were intentionally left unchanged because this step was limited to tests/benchmark/*.

Known limitations

  • This benchmark is HTTP-only and does not cover RESP.
  • The dummy origin runs in-process through ASGI transport, so this is a demo-oriented relative comparison rather than a full network benchmark.
  • README, CI workflow, and presenter smoke flow updates are still pending for later work in chore/demo-bench-ci-readme.

@SISUinSea SISUinSea merged commit 10c435e into main Mar 18, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant