Skip to content

[no-merge] add fc yaml#4710

Open
Xing-lil wants to merge 3 commits into
PaddlePaddle:developfrom
Xing-lil:gbzz_fc_yaml
Open

[no-merge] add fc yaml#4710
Xing-lil wants to merge 3 commits into
PaddlePaddle:developfrom
Xing-lil:gbzz_fc_yaml

Conversation

@Xing-lil

Copy link
Copy Markdown
Collaborator

Before submitting

  • Lint code. If there are lint issues, please format the code first.
# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py
  • Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

Others

PR changes

Others

Description

add fc yaml

@Xing-lil Xing-lil changed the title [FC] add fc yaml gbzz [FC] add fc yaml Jun 25, 2026
@Xing-lil Xing-lil closed this Jun 25, 2026
@Xing-lil Xing-lil reopened this Jun 26, 2026
@Xing-lil Xing-lil changed the title [FC] add fc yaml [no-merge] add fc yaml Jun 26, 2026

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已完成首轮 review。当前发现会阻塞这些新增 FT 配置按节点数运行的并行度问题,细节和可应用修改已放在行内评论里。

P3 优先级:P3 非行级:PR 标题和描述无法挂到具体 diff 行。当前标题带有 [no-merge],描述也只有 add fc yaml,如果准备继续合入,建议去掉 no-merge 标记,并在描述中补充这些配置的目标卡数/节点数、启动方式和已验证结果。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment on lines +45 to +48
sharding_parallel_size: 4
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这个 2nodes 配置按同目录 FT 配置的 2 节点/8 卡约定运行时会在训练参数初始化阶段直接失败。TrainingArguments._post_init_parallel_degree() 会计算 data_parallel_size = world_size / (sharding_parallel_size * tensor_model_parallel_size * sep_parallel_size * pipeline_model_parallel_size);16 卡下这里是 16/(4*1*1*2)=2,同时 expert_model_parallel_size=4 又得到 moe_sharding_parallel_size=16/(2*4)=2,随后代码明确拒绝 moe_sharding_parallel_size > 1 && data_parallel_size > 1 以及 data_parallel_size > 1 && expert_model_parallel_size > 1 的组合,训练不会启动。若这个文件仍是 2 节点 8 卡场景,请保持 sharding/EP 度数覆盖完整每个 PP stage;若目标其实是 2 节点 4 卡,也需要在文件名或注释里标清并配套启动脚本固定 world_size=8

Suggested change
sharding_parallel_size: 4
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 4
sharding_parallel_size: 8
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 8

Comment on lines +45 to +48
sharding_parallel_size: 4
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这个 3nodes 配置也会在常规 3 节点/8 卡的 FT 运行里被并行度校验拒绝。24 卡下当前值会得到 data_parallel_size = 24/(4*1*1*3)=2,同时 moe_sharding_parallel_size = 24/(3*4)=2TrainingArguments._post_init_parallel_degree() 不支持 EP 和数据并行同时大于 1,也不支持 moe_sharding_parallel_size > 1 时再有 data_parallel_size > 1,因此训练初始化会抛错。若该文件表示 3 节点 8 卡,请改回覆盖完整 PP stage 的 sharding/EP 度数;若实际只打算 4 卡/节点运行,请把这个 rank 约束写进配置说明/文件名,避免按 3nodes 直接使用时失败。

Suggested change
sharding_parallel_size: 4
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 4
sharding_parallel_size: 8
pp_delay_scale_loss: true
use_expert_parallel: true
expert_model_parallel_size: 8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants