Skip to content

vLLM container crashloops on llm-d-cuda:v0.7.0 (no command set, NVIDIA wrapper entrypoint) #123

Description

@dcmcand

Found while testing #117 live.

A served LLMModel's vllm pod crashloops immediately:

/opt/nvidia/nvidia_entrypoint.sh: line 67: exec: --: invalid option

operator/internal/controller/reconcilers/modelservice.go sets the vllm container's Args (the vLLM flags: --model, --served-model-name, ...) but no Command. ghcr.io/llm-d/llm-d-cuda:v0.7.0 has Entrypoint: ["/opt/nvidia/nvidia_entrypoint.sh"] and no default CMD, so the wrapper exec's the flags as if they were the command and exits.

Impact: served models never start on the default serving image.

Fix: set the container Command to the standard vLLM OpenAI-server entrypoint (python3 -m vllm.entrypoints.openai.api_server); the image's venv python is on PATH. Done on the #117 branch (commit c512e37) with a test asserting the command is set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Start date

    None yet

    Target date

    None yet

    Size

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions