Skip to content

Add TRL#1

Open
arsalanu wants to merge 3 commits intomainfrom
add-trl
Open

Add TRL#1
arsalanu wants to merge 3 commits intomainfrom
add-trl

Conversation

@arsalanu
Copy link
Owner

TODO:

  • Add TRL implementation for RL
  • TRL currently breaks torchforge dependencies, lots of conflicting packages - mainly vLLM
  • Need to alternate RL environment between either building for TRL OR building for torchforge
  • Handle dependencies as extensions to core dependencies (torchtune is more mature)
  • For TRL: need to handle flash-attn install for cp12/x86/torch2.9 wheel.
  • Everything needs to be built with torch 2.9.0, need to fix versions

Small extra:

  • Also moves qwen3 handling for torchforge into its own file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant