English (polished)
Symptom (Volcano + vGPU)
With Volcano for vGPU scheduling, when many inference jobs are submitted automatically at scale, many PodGroups stay Pending / Inqueue for a long time, while only one or two are Running.
Typical explanation
GPU, CPU, or queue quota already in use prevents the whole gang from being satisfied at once. Pods that have already started wait idle for their peers; pods that have not started cannot join because of the gang constraint—so the workload stalls as a group, which feels like a resource deadlock.
Contrast (HAMI vGPU)
With HAMI vGPU, even if some pods are Pending, the backlog drains gradually in batches, without the same prolonged whole-group stall.
Takeaway (subjective)
In this kind of workload, gang scheduling feels more prone to getting stuck than the HAMI path.
中文:
现象（Volcano + vGPU）
在 Volcano 上做 vGPU 调度时，若自动、大批量提交推理任务，容易出现大量 PodGroup 长期处于 Pending / Inqueue，同时 Running 的只有 1～2 个。
常见解释
已占用的 GPU、CPU 或队列额度等资源，使「整组任务」无法一次性满足；已调起的 Pod 在等组内同伴，未调起的 Pod 又因组约束进不来，彼此牵制，整体推进困难，观感上接近资源层面的死锁。
对比（HAMI vGPU）
换成 HAMI vGPU 后，即便仍有 Pending，队列也会按批次逐步消化，不会像上面那样长时间整组卡死。
结论（主观感受）
相较之下，gang 式组调度在大量自动推理场景下，更容易出现「整组卡住、难以自动消化」的印象。
English (polished)
Symptom (Volcano + vGPU)
With Volcano for vGPU scheduling, when many inference jobs are submitted automatically at scale, many PodGroups stay Pending / Inqueue for a long time, while only one or two are Running.
Typical explanation
GPU, CPU, or queue quota already in use prevents the whole gang from being satisfied at once. Pods that have already started wait idle for their peers; pods that have not started cannot join because of the gang constraint—so the workload stalls as a group, which feels like a resource deadlock.
Contrast (HAMI vGPU)
With HAMI vGPU, even if some pods are Pending, the backlog drains gradually in batches, without the same prolonged whole-group stall.
Takeaway (subjective)
In this kind of workload, gang scheduling feels more prone to getting stuck than the HAMI path.
中文:
现象（Volcano + vGPU）
在 Volcano 上做 vGPU 调度时，若自动、大批量提交推理任务，容易出现大量 PodGroup 长期处于 Pending / Inqueue，同时 Running 的只有 1～2 个。
常见解释
已占用的 GPU、CPU 或队列额度等资源，使「整组任务」无法一次性满足；已调起的 Pod 在等组内同伴，未调起的 Pod 又因组约束进不来，彼此牵制，整体推进困难，观感上接近资源层面的死锁。
对比（HAMI vGPU）
换成 HAMI vGPU 后，即便仍有 Pending，队列也会按批次逐步消化，不会像上面那样长时间整组卡死。
结论（主观感受）
相较之下，gang 式组调度在大量自动推理场景下，更容易出现「整组卡住、难以自动消化」的印象。