[TransferEngine] Add instant bandwidth reporting to tebench

[alogfans]

Description

This PR adds instant bandwidth reporting to tebench.

Changes include:

• Add --request_interval_us to optionally pace issued transfer batches.
• Compute per-transfer instant bandwidth in GB/s.
• Add Avg Inst GB/s to the regular benchmark output table.
• Derive Avg Lat (us) from Avg Inst GB/s, keeping latency and instant bandwidth on the same measurement basis.

The pacing implementation uses std::this_thread::yield() for busy-waiting and does not introduce mutexes in the request pacing path.

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Test commands:

git diff --check
cmake --build build --target tebench

Test results:

• Unit tests pass
• Integration tests pass (if applicable)
• Manual testing done (build verification)

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh
• I have run pre-commit run --all-files and all hooks pass
• I have updated the documentation (if applicable)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed an RFC issue

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)

Codex was used to help implement and refine the benchmark reporting changes and prepare this PR description.

[Copilot]

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds instant per-transfer bandwidth reporting to tebench, including a new pacing flag and updated benchmark output so latency can be derived from the same measurement basis as instant bandwidth.

Changes:

• Added --request_interval_us flag and plumbing in XferBenchConfig to pace transfer batch issuance.
• Collected per-transfer “instant bandwidth” and added Avg Inst GB/s to the benchmark table output.
• Derived Avg Lat (us) from Avg Inst GB/s (instead of total-duration-based latency).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
mooncake-transfer-engine/benchmark/utils.h	Adds config field for request pacing and a new stats channel for instant bandwidth.
mooncake-transfer-engine/benchmark/utils.cpp	Adds the new flag, wires it into config, and extends output table + latency derivation.
mooncake-transfer-engine/benchmark/main.cpp	Implements per-transfer instant bandwidth sampling and request pacing via busy-wait/yield.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[staryxchen]

Avg Lat (us) is now near-duplicate of Avg Tx (us) and silently changes semantics

Avg Tx (us) is already stats.transfer_duration.avg() — the arithmetic mean of per-transfer duration. The new Avg Lat (us) (when avg_instant_gbps > 0) simplifies to 1 / mean(1/duration_i) — the harmonic mean of the same per-transfer durations. By Jensen's inequality (1/x is convex), harmonic <= arithmetic, with equality only when all durations are identical. In my local run (1MB, 2 threads, no pacing) the two columns read 1539.9 vs 1540.2 us — within noise, so the new column adds no information beyond Avg Tx (us).

Beyond redundancy, this silently alters the semantics of Avg Lat (us):

• Old: total_duration * num_threads / num_ops — wall-clock per-op, includes pacing / scheduling gaps between transfers.
• New: 1 / mean(1/duration_i) — per-transfer duration only, excludes gaps.

When --request_interval_us is set, the old metric would have reported transfer time + pacing gap, while the new one reports only transfer time. Existing users who rely on Avg Lat (us) as a wall-clock per-op figure will be misled, and the PR description doesn't flag this change.

Suggestion: Since Avg Tx (us) already covers per-transfer latency, the minimal change is to keep Avg Lat (us) at its old wall-clock semantics and let Avg Inst GB/s be the sole new column. Alternatively, if per-transfer latency is the intended meaning, drop Avg Lat (us) and relabel Avg Tx (us) — but that's a bigger break.