Qwen3-30B-A3B的训练脚本

[stefanxinhong]

请问是否有计划支持 Qwen3-30B-A3B的训练 @bingyang-lei

[bingyang-lei]

Thanks for the question.

In fact, we already support training for Qwen3‑30B‑A3B. As long as the model follows the DFlash architecture, you can simply pass a different draft model path — the same pipeline works out of the box.

The draft model for 30B‑A3B and the one for 4B are essentially the same in terms of architecture and training logic; the only difference is the number of layers. Of course, to accommodate a larger target model, you may need to adjust some training setting — for example, setting actor.fsdp_config.model_dtype=bf16 to reduce memory usage. Feel free to let your LLM optimize the script accordingly.

Feel free to try it out!

[stefanxinhong]

@bingyang-lei 是否可以提供一个调通的训练配置脚本，感谢

[bingyang-lei]

We have now provided a training script suitable for 30-A3B. However, since GPU configurations vary significantly across researchers, this script may not be optimal for your specific research setup. Please adjust according to your actual resources.