Unpin liger-kernel 0.8.1 - #6761
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 545bf8f0de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
albertvillanova
left a comment
There was a problem hiding this comment.
Thanks, @qgallouedec.
I agree with your EDIT: this shouldn't be merged, and I'd go further and close it.
Some context on why the constraint has its current shape: the !=0.8.1 in #6518 was chosen over <0.8.1 precisely for this situation. It excludes exactly the one broken release while leaving every future release allowed, so no TRL-side change is needed for 0.8.2 to be picked up; the current pin already permits it. That makes this PR unnecessary rather than merely early.
Three additional reasons beyond the GPU-arch point:
- The end state in this diff is
>=0.8.0, which permanently re-permits 0.8.1. Even once 0.8.2 exists, a bare>=0.8.0lets a resolver land on the broken release (old lockfiles,--no-depsinstalls, constrained resolutions). The right target is>=0.8.2, as your EDIT says. - Timing makes it worse than it looks. Because 0.8.1 is currently the newest release, unpinning today doesn't just allow the broken version: every fresh
pip install trl[liger]would resolve to exactly it on pre-Ampere hardware (#6516). - The
filterwarningsremoval is coupled to the same bump. With!=0.8.1in force, CI resolves 0.8.0, which predates #1274, so dropping the ignore entry brings theIndexPutBackward0noise back. The existing comment already records this (Remove once: liger-kernel >= 0.8.2 is required).
And as you note, PEP 508 markers can't express "0.8.1 is fine iff compute capability >= 8.0", so there's no way to keep CI on L40S free of the constraint while protecting published trl[liger] users. A runtime guard in TRL would just duplicate the upstream fix, which I don't think is worth it.
So in conclusion, I would close this PR. Thanks for catching this before it went in: the GPU CI move to L40S (#6741) is genuinely the right change, but it just doesn't unblock this one.
|
For the record, the follow-up work is tracked in two issues, so nothing is lost when this PR is closed:
The two are coupled and should land in the same PR; #6767 carries the full checklist. |
Requires #6741 (GPU CI on L40S).
0.8.1 was excluded because it needs bf16 gemm (compute capability >= 8.0) and our T4 runner is sm_75 see #6516. The L40S runner is sm_89, so the constraint is gone.
liger-kernel>=0.8.0,!=0.8.1→liger-kernel>=0.8.0(liger+devextras)IndexPutBackward0warning filter, fixed upstream in 0.8.1 (Avoid torch.compile graph break in SAPO loss linkedin/Liger-Kernel#1274)EDIT: after second thought, we shouldn't merge this now:
trl[liger]is published, so unpinning breaks users on pre-Ampere GPUs (#6516) even though our CI moved to an L40S. PEP 508 markers can't scope a dependency by GPU arch. The upstream guard (linkedin/Liger-Kernel#1316) merged ~24h after 0.8.1 shipped, so it lands in 0.8.2 (not to be confused with #1274 (SAPO fix), which is in 0.8.1.)When 0.8.2 ships: bump both extras to
liger-kernel>=0.8.2, drop the SAPOIndexPutBackward0filterwarnings entry.Note
Low Risk
Packaging and test-warning cleanup only; no runtime training logic changes.
Overview
Allows
liger-kernel0.8.1 again by changingliger-kernel>=0.8.0,!=0.8.1toliger-kernel>=0.8.0in theligeranddevoptional dependencies. That exclusion existed because 0.8.1 needs bf16 GEMM (compute capability ≥ 8.0), which the old T4 CI runner could not satisfy; GPU CI on L40S removes that constraint.Removes the pytest
filterwarningsentry that ignored Liger SAPOIndexPutBackward0undertorch.compile, since that behavior was fixed upstream in 0.8.1.Reviewed by Cursor Bugbot for commit 545bf8f. Bugbot is set up for automated code reviews on this repo. Configure here.