Skip to content

Commit 82decf1

Browse files
committed
fix(examples): vllm example uses python3 + moves port into yaml
Found during dev B200 verification (Qwen2.5-0.5B variant of this shape): - vllm/vllm-openai image has python3 only, no `python` symlink — the K8s container fails to start with `exec: "python": executable file not found in $PATH` because the platform's `command` field overrides the image's ENTRYPOINT entirely. - The platform helm path passes numeric arg tokens to the Rollout spec as integers, and the K8s API server rejects them (`args[3] ... must be of type string: "integer"`). Moving `port: 8000` into the YAML config keeps every CLI token a non-numeric string while still letting vLLM pick up the port — and matches the "all config in one file" intent of --config. Verified end-to-end on dev (cluster c-01-c-11-centml-org, hw x1-large-b200): created deployment via SDK → rollout HEALTHY at t+76s → POST /v1/chat/completions returned HTTP 200 with a real Qwen completion. Signed-off-by: Honglin Cao <hocao@nvidia.com>
1 parent 2095ab5 commit 82decf1

2 files changed

Lines changed: 2 additions & 1 deletion

File tree

examples/sdk/create_inference_vllm.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ def main():
2727
healthcheck="/health",
2828
concurrency=10,
2929
env_vars={"HF_TOKEN": "<your-hf-token>"},
30-
command="python -m vllm.entrypoints.openai.api_server --port 8000 --config /etc/vllm/vllm_config.yaml",
30+
command="python3 -m vllm.entrypoints.openai.api_server --config /etc/vllm/vllm_config.yaml",
3131
config_file=load_config_file_mount(path="./vllm_config.yaml", mount_path="/etc/vllm"),
3232
)
3333
response = cclient.create_inference(request)

examples/sdk/vllm_config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
port: 8000
12
model: meta-llama/Llama-3.1-8B-Instruct
23
tokenizer: meta-llama/Llama-3.1-8B-Instruct
34
runner: generate

0 commit comments

Comments
 (0)