💾 CPU model
AMD EPYC 9654
🎮 GPU model
NVIDIA RTX PRO 6000
📊 Performance data
──────────────────────────────────────────────────────────────────────
[Current Generation]: 335 for TPS | Time: 5.28s | Speed: 63.43 tokens/s
[Total Statistics]: 2368 tokens | Average Speed: 59.65 tokens/s
[Current VRAM Usage]: 5.54GB/94.97GB (5.8%) | Cache: 5.79GB
──────────────────────────────────────────────────────────────────────
🛠️ System information [optional]
Platform: Linux-5.15.0-105-generic-x86_64-with-glibc2.35
Python Version: 3.11.8
PyTorch Version: 2.12.1+cu130
🖥 Screenshot [optional]
💾 CPU model
AMD EPYC 9654
🎮 GPU model
NVIDIA RTX PRO 6000
📊 Performance data
──────────────────────────────────────────────────────────────────────
[Current Generation]: 335 for TPS | Time: 5.28s | Speed: 63.43 tokens/s
[Total Statistics]: 2368 tokens | Average Speed: 59.65 tokens/s
[Current VRAM Usage]: 5.54GB/94.97GB (5.8%) | Cache: 5.79GB
──────────────────────────────────────────────────────────────────────
🛠️ System information [optional]
Platform: Linux-5.15.0-105-generic-x86_64-with-glibc2.35
Python Version: 3.11.8
PyTorch Version: 2.12.1+cu130
🖥 Screenshot [optional]