Skip to content

Add support for Gemma3 VLM architecture#1601

Merged
xenova merged 9 commits intomainfrom
add-gemma3
Mar 23, 2026
Merged

Add support for Gemma3 VLM architecture#1601
xenova merged 9 commits intomainfrom
add-gemma3

Conversation

@xenova
Copy link
Collaborator

@xenova xenova commented Mar 23, 2026

Closes #1438

Example

import {
  AutoProcessor,
  AutoModelForImageTextToText,
  load_image,
  TextStreamer,
} from "@huggingface/transformers";

// Load model and processor
const model_id = "onnx-community/gemma-3-4b-it-ONNX";
const processor = await AutoProcessor.from_pretrained(model_id);
const model = await AutoModelForImageTextToText.from_pretrained(model_id, {
  device: "webgpu",
  dtype: "q4",
});

// Define the list of messages
const messages = [
  {
    role: "user",
    content: [
      { type: "image" },
      { type: "text", text: "Describe this image." },
    ],
  },
];

// Prepare inputs
const prompt = processor.apply_chat_template(messages, { add_generation_prompt: true });
const url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg";
const image = await load_image(url);
const inputs = await processor(prompt, image, { add_special_tokens: false });

const outputs = await model.generate({
  ...inputs,
  max_new_tokens: 2048,
  streamer: new TextStreamer(processor.tokenizer, {
    skip_prompt: true,
    // callback_function: (text) => { /* Do something with the streamed output */ },
  }),
});

// Decode output
const decoded = processor.batch_decode(
  outputs.slice(null, [inputs.input_ids.dims.at(-1), null]),
  { skip_special_tokens: true },
);
console.log(decoded[0]);

@xenova xenova changed the title Add support for Gemma3 Add support for Gemma3 VLM architecture Mar 23, 2026
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@xenova xenova merged commit a97b51b into main Mar 23, 2026
4 checks passed
@xenova xenova deleted the add-gemma3 branch March 23, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Gemma3 (VLM)

2 participants