Skip to content

Bump to transformers v5 and fix ColPali processor compatibility#389

Open
ManuelFay wants to merge 17 commits intoilluin-tech:mainfrom
ManuelFay:codex/update-package-version-to-transformers-5
Open

Bump to transformers v5 and fix ColPali processor compatibility#389
ManuelFay wants to merge 17 commits intoilluin-tech:mainfrom
ManuelFay:codex/update-package-version-to-transformers-5

Conversation

@ManuelFay
Copy link
Collaborator

@ManuelFay ManuelFay commented Feb 13, 2026

  • Add regression tests for ColPaliProcessor to validate Transformers v5 modality registration and fallback loading behavior when a processor bundle is incomplete.

Changed

  • Bump runtime compatibility to transformers>=5.0.0,<6.0.0, peft>=0.18.0,<0.19.0, and accelerate>=1.1.0,<2.0.0.
  • Update supported Python versions to >=3.10,<3.15 and align CI workflows to Python 3.10–3.14.
  • Update all affected processor subclasses (Qwen2/Qwen2.5/Qwen3, Gemma3, Idefics3, ModernVBert, Qwen2.5 Omni) to explicit __init__ modality signatures required by Transformers v5 ProcessorMixin.

Fixed

  • Fix ColPali/PaliGemma model loading under Transformers v5 by adapting wrapper internals to new module layout and tied-weights expectations.
  • Fix ColPali processor loading for checkpoints without a complete processor bundle by explicitly falling back to AutoImageProcessor + AutoTokenizer.
  • Fix ColPali collator image token id lookup to use convert_tokens_to_ids, compatible with Transformers v5 tokenizer backend changes.
  • Fix test collection on Python 3.14 by making tests an explicit package (tests/__init__.py).
  • Fix CI formatting failure by applying ruff format to updated ColPali processing tests.
  • Fix ColQwen2 and ColQwen2.5 initialization across Transformers versions by resolving hidden size from either config.hidden_size or config.text_config.hidden_size.
  • Call post_init() in ColIdefics3 and ColModernVBert to align model initialization with Transformers v5 expectations.
  • Improve VisualRetrieverCollator image token id resolution by preferring processor-level image_token_id when available.
  • Fix ColQwen2 and ColQwen2.5 LoRA checkpoint key remapping for custom_text_proj (base_model.model.* -> model keys) to avoid missing/unexpected adapter keys at load time.
  • Fix ColPali LoRA adapter key remapping for custom_text_proj (base_model.model.* -> model keys) and ignore expected missing model.lm_head.weight during load.
  • Fix ColModernVBert LoRA adapter key remapping for custom_text_proj (base_model.model.* -> model keys) to avoid missing/unexpected adapter keys at load time.

@ManuelFay
Copy link
Collaborator Author

ManuelFay commented Feb 13, 2026

  • I tested end to end on colpali, colqwené, colqwen2.5, colmodernvbert, colqwenomni;

not tested colqwen3 and colgemma because no model weights are public - which is annoying

@ManuelFay
Copy link
Collaborator Author

ManuelFay commented Feb 13, 2026

Note: The slow tests show some regressions on ColPali - not exactly the same behavior.
Running the slow tests with the old package version shows this was already the case - and that the values are very similar with before this PR. If there was a regression, it happened before so should be mergeable !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant