Skip to content

Commit 2dae477

Browse files
authored
feat: Generic Multimodal Chat Handler (abetlen#2256)
1 parent 9f6efb0 commit 2dae477

4 files changed

Lines changed: 518 additions & 49 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
- feat: Generic Multimodal Chat Handler by @abetlen in #2256
1011
- feat: update llama.cpp to ggml-org/llama.cpp@e3ba22d6c
1112
- feat(ci): add ROCm wheel builds by @abetlen in #2252
1213
- feat(ci): add Vulkan wheel builds by @abetlen in #2251

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -536,6 +536,7 @@ Below are the supported multi-modal models and their respective chat handlers (P
536536
| [minicpm-v-2.6](https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf) | `MiniCPMv26ChatHandler` | `minicpm-v-2.6` |
537537
| [qwen2.5-vl](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-GGUF) | `Qwen25VLChatHandler` | `qwen2.5-vl` |
538538
| [gemma-4](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF) | `Gemma4ChatHandler` | `gemma4` |
539+
| GGUF models with an mtmd projector and embedded chat template | `MTMDChatHandler` | `mtmd` |
539540

540541
Then you'll need to use a custom chat handler to load the clip model and process the chat messages and images.
541542

0 commit comments

Comments
 (0)