Support for other granite-4 models

[HelloKS]

Hello, thanks for the easy method to extend model capability.

I tried with IBM Granite 4.0-h-small, but it seems doesn't like its structure. (No attn layers but mamba)
Do you think support for other variants are viable?

[p-e-w]

See #14, apparently @pszemraj has solved this already and will send a PR.

[pszemraj]

Sorry for the delay! Work has been busier than anticipated this week so it's going to be a weekend project

(sort of my fault, I didn't fork this repo like I should have with contributing in mind)

I've been able to get:

• granite-4.0 hybrid/moe - this is "tiny" which is the 7b
• lfm2-vl - which required support for convolution layers and smart loading the vision models
  • note the vision part is not abliterated itself, the decoder "inside" the VLM is.
  • The 100 test prompts don't include images to try so that part will have to be hand validated, but based on the text refusal post-abliteration (~4/100 iirc, on model card) it worked there

[pszemraj]

first pass done! see #43 ✨