This repository was archived by the owner on Oct 31, 2023. It is now read-only.
This repository was archived by the owner on Oct 31, 2023. It is now read-only.
larger batch size with linear scale does not work #35
Hi, I tried to enlarge the batch size to 512 * 8 = 4906 with lr 0.03 * 4096 / 256 = 0.48 for mocov2 and 0.05 * 4096 / 256 = 0.8 for simsiam, however, after pre-training the model perform much worse on multiple downstream tasks, and both simsiam and mocov2 failed to convergence on linear evaluation. Could you give me some advice?