Add nvidia.com/gpu toleration to GPU serving workloads

[dcmcand]

Background

nebari-dev/nebari-infrastructure-core#370 makes NIC automatically taint AWS GPU node groups with nvidia.com/gpu=true:NoSchedule, so only pods that tolerate it land on GPU nodes.

Nothing injects the toleration for us on EKS:

• The NVIDIA GPU Operator only tolerates this taint on its own operands, not on serving workloads.
• The ExtendedResourceToleration admission controller is not available on EKS, so GPU-requesting pods do not get an automatic toleration. AWS's GPU guide hand-writes the toleration on every example.

Once #370 lands, llm-d serving pods that request GPUs will stop scheduling onto GPU nodes unless they carry the toleration.

Request

Add a toleration for nvidia.com/gpu to the GPU-bearing serving workloads this pack deploys (model servers / llm-d components that request nvidia.com/gpu). Preferably apply it only to workloads that actually request a GPU. The toleration:

tolerations:
- key: "nvidia.com/gpu"
  operator: "Exists"
  effect: "NoSchedule"

operator: Exists matches any taint value, so it works with NIC's value: "true" and any other.

Acceptance criteria

• A served model that requests a GPU schedules onto a node tainted nvidia.com/gpu=true:NoSchedule and serves requests.
• Workloads that do not request a GPU are unchanged.

Blocks

This blocks nebari-dev/nebari-infrastructure-core#370. The taint should not roll out until GPU serving workloads tolerate it.