Training Resources Requirements

[ChongWang1024]

Hi!

Thanks for sharing such an interesting paper and the corresponding codes.

May I ask about the training resources and training time needed for both the pre-training and fine-tuning stages?

Thanks in advance!

Best regards

[CserDu]

@ChongWang1024 Hello, thanks for your interest. Pre-training (primarily visual pre-training) on 8 NVIDIA A100-40G GPUs takes about 1 day, and fine-tuning takes about 2 days.

[tuyunbin]

Hi, can I use 4090 GPUs for fine-tuning?

[CserDu]

@tuyunbin This might be difficult since the 4090 GPUs do not support bf16, but it's pretty good for inference.