Skip to content

Add Seed-OSS model support#4681

Open
Minestar6 wants to merge 2 commits into
PaddlePaddle:developfrom
Minestar6:feature/add_seed_oss
Open

Add Seed-OSS model support#4681
Minestar6 wants to merge 2 commits into
PaddlePaddle:developfrom
Minestar6:feature/add_seed_oss

Conversation

@Minestar6

@Minestar6 Minestar6 commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Add seed_oss

添加了seed_oss模型的PaddlePaddle 实现,包含模型代码、配置文件、单元测试。

对齐验证

精度对齐

前 20 个 token 的 logits 对齐结果如下:

  • mean diff: 0.000000e+00
  • max diff: 0.000000e+00

训练loss对齐

训练数据

  • GSM8K SFT 数据
  • 训练步数:300 steps

训练结果

3_seed_oss_diff 3_seed_oss

@paddle-bot

paddle-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

Thanks for your contribution!

@Minestar6
Minestar6 force-pushed the feature/add_seed_oss branch from 836ff73 to 58ecbc4 Compare July 19, 2026 06:45

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已完成首轮审查,发现几处需要先修复的功能性问题,具体见行内评论。

  • P3 优先级:P3 非行级:PR 标题目前是 first,和实际变更内容不匹配。建议改成 Add Seed-OSS model support 或中文等价标题,方便后续检索和 release note 汇总。
    处理要求:请针对该评论进行回复(同意并已修改请回复 Done,不同意请说明理由)。
Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment thread paddleformers/transformers/seed_oss/modeling.py Outdated
Comment thread paddleformers/transformers/seed_oss/configuration.py Outdated
Comment thread paddleformers/datasets/template/template.py Outdated
@Minestar6 Minestar6 changed the title first Add Seed-OSS model support Jul 19, 2026
@Minestar6
Minestar6 force-pushed the feature/add_seed_oss branch from 58ecbc4 to 69f1e4b Compare July 20, 2026 15:45

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交,之前关于默认配置、模板注册和 sequence parallel 形状路径的功能性问题看起来已经处理。

当前仍有一个 lint 阻塞项,另外 README 里还有一处说明需要和实际模板注册对齐,细节已放在行内评论。请先按行内意见修复/回复后再继续复查。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment thread README.md
<tr>
<td>Seed-OSS</td>
<td>ByteDance-Seed/Seed-OSS-36B-Base、ByteDance-Seed/Seed-OSS-36B-Instruct</td>
<td>seed_oss</td>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 优先级:P3

这里把 Seed-OSS 的 Chat Template 写成了 seed_oss,但当前分支里已经没有对应的模板注册了,README 会误导使用者以为还能直接套用这个模板。建议把这里改成 -,或者直接移除这列对应的模板值。

处理要求:请针对该评论进行回复(同意并已修改请回复 Done,不同意请说明理由)。

Suggested change
<td>seed_oss</td>
<td>-</td>

@@ -0,0 +1 @@

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这个新文件现在只有一行空白,CI 里的 flake8 已经明确报了 W391 blank line at end of file,会直接把 lint 卡住。请删掉这行空白,并把 paddleformers/transformers/seed_oss/modeling.pytests/transformers/seed_oss/test_modeling.py 按 Black 的输出一起整理后再提交新的 commit。

处理要求:请针对该评论修复并提交新的 commit。

Suggested change

@Paddle-CI-Bot

Copy link
Copy Markdown

PaddleFormers Log Analysis

Run #29758932487 · Attempt 1

日志分析报告

流水线名称 问题标签 修复建议 日志片段
Unittest GPU CI 模型API Bug(AutoModel注册缺失 + Dropout p=None) seed_oss/modeling.py 中将 SeedOssForCausalLM 注册到 AutoModel,并修复 SeedOssForTokenClassification.__init__nn.Dropout 初始化时未传 p 值(传了 None 报错代码

失败的测试case:

tests/transformers/seed_oss/test_modeling.py::SeedOssModelTest::test_auto_model_for_causal_lm_from_config
tests/transformers/seed_oss/test_modeling.py::SeedOssModelTest::test_seed_oss_token_classification_model

根本原因分析:

本PR新增了 seed_oss 模型,但存在两处代码缺陷:

  1. test_auto_model_for_causal_lm_from_configAutoModelForCausalLM.from_config(config) 返回的是 SeedOssModel 而非 SeedOssForCausalLM,说明 SeedOssForCausalLM 未正确注册到 AutoModel 映射表(MODEL_FOR_CAUSAL_LM_MAPPING@register_auto_class)。

  2. test_seed_oss_token_classification_modelSeedOssForTokenClassification.forward 调用 self.dropout(outputs[0]) 时,paddle.nn.Dropoutp 参数为 Nonep = None 进入 F.dropout),触发 TypeError: p argument should be a number or Variable,根因是 SeedOssForTokenClassification.__init__nn.Dropout(classifier_dropout)classifier_dropout 取值路径返回了 None(通常是 config.classifier_dropout or config.hidden_dropout_prob 未设默认值)。

修复建议:

  1. paddleformers/transformers/seed_oss/modeling.py 中,为 SeedOssForCausalLM 添加 AutoModel 注册,例如:

    # 在类定义上方加装饰器,或在 __init__.py 中注册
    AutoModelForCausalLM.register(SeedOssConfig, SeedOssForCausalLM)

    或确认 configuration.pymodel_type 与全局 MODEL_FOR_CAUSAL_LM_MAPPING 的 key 一致。

  2. SeedOssForTokenClassification.__init__ 中,修复 classifier_dropout 取值逻辑,确保传入 nn.Dropout 的值不为 None

    classifier_dropout = (
        config.classifier_dropout
        if config.classifier_dropout is not None
        else config.hidden_dropout_prob
    )
    self.dropout = nn.Dropout(classifier_dropout)

🔍 准确性记录:请点击评论底部 😊 图标,选择 👍(准确)或 👎(有误),将自动记录到 CI 监控系统

🔄 每次 Re-run 后自动更新

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants