feat: add E2E perf tests for mlperf-endpoints

[viraatc]

Drive the cyclopts inference-endpoint app in-process against the existing MaxThroughputServer and VariableResponseServer stubs. Two families, both marked @pytest.mark.performance (CI-skipped — run on demand):

• Roofline — measures peak QPS for max_throughput, concurrency sweep (1k / 4k / 16k), and binary-searches the largest 10 k-multiple target_qps Poisson sustains. Reports numbers rather than asserting on them.

• Low-QPS correctness — 5 QPS Poisson against the realistic stub for ~20 s; asserts zero failed requests. Guards keep-alive / idle-pool / slow-response regressions that only surface when connections sit idle longer than TCP_KEEPIDLE.

A conftest.py captures each parameterized case via a record_result fixture and renders a unified summary table with host / CPU / core info at end of session, so cross-machine roofline runs are easy to compare.

WARNING: full run takes ~8–10 min wall.

How to run

uv run pytest -vs -m performance --no-cov tests/performance/commands/test_e2e_perf.py

Example output — local dev box

=========================== E2E Performance Summary ============================
Host:  nova-2    Arch: x86_64    Cores: 48
CPU:   AMD Ryzen Threadripper PRO 7965WX 24-Cores

Test                       Stream  QPS        Total       Elapsed   Failed
-------------------------  ------  ---------  ----------  --------  ------
max_throughput (2M burst)  off        42,929   2,000,000    46.59s     0
concurrency c=1,000        off        51,433     617,330    11.99s     0
concurrency c=4,000        off        51,819     622,182    11.98s     0
concurrency c=16,000       off        50,160     604,654    11.80s     0
poisson max_sustained      off        30,000  —           —            0
max_throughput (2M burst)  on         32,025   2,000,000    62.45s     0
concurrency c=1,000        on         30,337     364,982    12.00s     0
concurrency c=4,000        on         31,348     380,157    12.00s     0
concurrency c=16,000       on         31,495     393,876    12.00s     0
poisson max_sustained      on         30,000  —           —            0
low_qps target=5           off          4.99         200    40.07s     0
low_qps target=5           on           3.97         200    50.39s     0
========================= 12 passed, 2 warnings in 495.47s (0:08:15) =========================

What does this PR do?

Adds the smallest set of E2E parameterized perf tests that exercise all three online load patterns + low-QPS keep-alive guard, behind the existing performance marker so CI is unaffected.

Type of change

• Bug fix
• New feature
• Documentation update
• Refactor/cleanup

Related issues

Testing

• Tests added/updated
• All 12 tests pass locally (~8 min)
• Manual testing completed

Checklist

• Code follows project style
• Pre-commit hooks pass
• Documentation updated (if needed)

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Code Review

This pull request introduces end-to-end performance and correctness tests for the benchmark CLI, including roofline tests, low-QPS correctness tests, and a pytest terminal summary reporter. The review feedback identifies several critical improvements: fixing a logic bug in the Poisson binary search where the lowest target might be skipped, using os.cpu_count() for cross-platform core detection instead of parsing /proc/cpuinfo, defensively handling type conversion errors in the summary formatter to prevent pytest crashes, and adding headroom to --num-samples in the Poisson load pattern test to prevent premature termination.