feat(llama3.1-8b): expose vLLM engine params as CLI arguments

[ssaketh-ch]

Summary

Expose vLLM engine memory and scheduling parameters as CLI arguments in the
Llama 3.1-8B benchmark harness, removing the need to edit source code to tune them.

Problem

When running the vLLM-backed SUT, parameters like gpu_memory_utilization,
max_num_seqs, and max_num_batched_tokens are hardcoded in load_model().
This causes two practical problems:

1. Out-of-memory (OOM) crashes -- on GPUs with less VRAM, or when running
   alongside other processes, there is no way to reduce memory usage without
   modifying source code.
2. No visibility into tunable knobs -- users hitting performance or memory
   issues have no obvious way to know which parameters exist or what values to
   try. The only option is to read the source, edit it, and re-run.

This is especially painful during bring-up on new hardware where the right
memory configuration is not known in advance.

Solution

Add 7 CLI flags to main.py and thread them through to the SUT and
SUTServer constructors and load_model() calls:

• --gpu-memory-utilization -- reduce if hitting OOM (default: 0.90)
• --max-num-batched-tokens -- cap total tokens scheduled per step
• --max-num-seqs -- limit concurrent request slots
• --block-size -- KV cache paging granularity
• --enforce-eager / --no-enforce-eager
• --enable-chunked-prefill / --no-enable-chunked-prefill
• --max-model-len -- cap the KV cache context window

Backward Compatibility

All flags default to vLLM's own defaults. Existing run scripts and
user.conf setups are completely unaffected.

Example

Reducing memory pressure on a smaller GPU:

python main.py --vllm --scenario Offline \
  --gpu-memory-utilization 0.80 \
  --max-num-seqs 128

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[hanyunfan]

WG: Assigned to Thomas to review it.

[attafosu]

Please address the comment on the prefix-caching and should be gtg

[attafosu]

Prefix caching should strictly be off (this is per the rules of the benchmark)

[ssaketh-ch]

Hi,
I opened two new commits to remove it. Please take a look.

[attafosu]

I'd remove this from the cli to avoid any confusion - the rules for the benchmark does not allow for prefix-caching.

[ssaketh-ch]

Hi,
I opened two new commits to remove it. Please take a look.

[attafosu]

Please sign the cla and we'll merge next week

[ssaketh-ch]

Hi,
I have signed the CLA, Let me know if there are further steps I need to take.
Sorry for the delay. Processing took longer than I expected for some reason.