Hello, thanks for your great work! I want to re-train this work in whole BEAT2 25 speakers, and found that the description of training setting of LLM in paper is different with README. In paper, it said the batch sizes per GPU are 32, 20, and 12, respectively. The gradient accumulation steps are set to 4, 6, and 10, , but in README, it said num_gpus × batch_size × gradient_accumulation_steps ≈ 256, and in three stages' config.yaml, default batch size is both 128 for 2 GPUs, and gradient_accumulation_steps is also same as default value 1.
So may I ask which setting is actually yours, which make it possible for me to making a fair compare, thanks a lot.
Hello, thanks for your great work! I want to re-train this work in whole BEAT2 25 speakers, and found that the description of training setting of LLM in paper is different with README. In paper, it said
the batch sizes per GPU are 32, 20, and 12, respectively. The gradient accumulation steps are set to 4, 6, and 10,, but in README, it saidnum_gpus × batch_size × gradient_accumulation_steps ≈ 256, and in three stages' config.yaml, defaultbatch sizeis both 128 for 2 GPUs, andgradient_accumulation_stepsis also same as default value1.So may I ask which setting is actually yours, which make it possible for me to making a fair compare, thanks a lot.