Idefics3#152
Merged
Merged
Conversation
- Introduced a new module `idefics3` in the models directory, which includes a new `processing.rs` file. - Implemented the `Idefics3ImageProcessor` struct with methods for image resizing, splitting, rescaling, and normalization. - Updated the main models module to include the new `idefics3` module alongside existing modules. - Enhanced image processing capabilities to support various preprocessing tasks for vision models.
- Introduced a new `colsmol_ort` module for ColSmol embedding, including the `OrtColSmolEmbedder` struct with methods for embedding images, files, and queries. - Added a new example in `colsmol.rs` demonstrating how to use the ColSmol embedder with command-line argument parsing. - Updated the `mod.rs` file to include the new `colsmol_ort` module under the `embeddings/local` directory. - Enhanced the `Idefics3ImageProcessor` with additional preprocessing capabilities for better integration with the ColSmol model. - Implemented tests for the new embedding functionality to ensure reliability and correctness.
- Updated `colpali.py` to enable loading of the Colpali model directly, while commenting out the ONNX model loading for clarity. - Enhanced the `ColpaliModel` in `colpali.rs` to support both Colpali and ColSmol models based on the model ID provided. - Introduced a new `ColSmolEmbedder` in `colsmol.rs` for embedding images and files, with improved preprocessing capabilities. - Updated the `colsmol` module in `mod.rs` to include the new `ColSmolEmbedder`. - Refactored example usage in `colsmol.rs` to demonstrate the new embedding functionality with command-line argument parsing. - Added tests for the new embedding features to ensure reliability and correctness.
sonam-pankaj95
approved these changes
Jun 25, 2025
Collaborator
|
New range of modern models, here we come. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.