Skip to content

_causal_mask写反了,训练时前面token可以看到后面的token。 #1

Description

@LIghtJUNction

LCM 当前 _causal_mask 方向写反了,训练时前面位置可以看到未来 token;这能解释仓库报告里训练 PPL 低到 1.x、实际生成仍是乱码

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions