Issue running on host with cuda 12.8

[okyspace]

Hi, I was trying fastsafetensors on vllm product stack helm chart.
I encountered error like provided PTX was compiled with an unsupportd toolchain.

From what I read, fastsafetensors 0.3 was tested on PyTorch 2.1, which supports up to cuda 12.1?

Is this the possible cause? And how can I overcome this issue?

[takeshi-yoshimura]

Did you try default safetensors loader? Please also check the version of nvidia driver in your system.

[takeshi-yoshimura]

please also check https://docs.vllm.ai/en/latest/usage/troubleshooting/#cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain