Skip to content

Add Support for Gemma 2 Models#167

Open
shrijayan wants to merge 3 commits intoMcGill-NLP:mainfrom
shrijayan:main
Open

Add Support for Gemma 2 Models#167
shrijayan wants to merge 3 commits intoMcGill-NLP:mainfrom
shrijayan:main

Conversation

@shrijayan
Copy link
Copy Markdown

This pull request introduces support for Gemma 2 models in the llm2vec repository by implementing the following enhancements:

Key Changes:

  1. Updated Transformers Version Constraint:

    • Modified dependencies to allow transformers==4.45.2, ensuring compatibility with the latest Gemma 2 configurations.
  2. Added Support for Gemma2Config:

    • Integrated Gemma2Config to enable seamless usage of Gemma 2 models within the llm2vec framework.
    • Ensured backward compatibility with existing model configurations.
  3. Enhanced GemmaBiModel and GemmaBiForMNTP:

    • Updated GemmaBiModel and GemmaBiForMNTP to handle Gemma 2 architecture properly.
    • Refactored encoding logic to leverage new features from Gemma 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant