Skip to content

Add support for qwen3_5_text <-> Qwen3_5ForCausalLM#1602

Merged
xenova merged 1 commit intomainfrom
add-Qwen3_5ForCausalLM
Mar 24, 2026
Merged

Add support for qwen3_5_text <-> Qwen3_5ForCausalLM#1602
xenova merged 1 commit intomainfrom
add-Qwen3_5ForCausalLM

Conversation

@xenova
Copy link
Collaborator

@xenova xenova commented Mar 24, 2026

So you can load and run qwen3_5 models that only have the text component. e.g.,

import { pipeline, TextStreamer } from "@huggingface/transformers";

// Create a text generation pipeline
const generator = await pipeline(
  "text-generation",
  "onnx-community/Qwen3.5-0.8B-Text-ONNX",
  { device: "webgpu", dtype: "q4f16" },
);

// Define the list of messages
const messages = [
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is the capital of France?" },
];

// Generate a response
const output = await generator(messages, {
  max_new_tokens: 512,
  do_sample: false,
  streamer: new TextStreamer(generator.tokenizer, {
    skip_prompt: true,
    skip_special_tokens: true,
  }),
});
console.log(output[0].generated_text.at(-1).content);

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@xenova xenova merged commit 99a8eeb into main Mar 24, 2026
4 checks passed
@xenova xenova deleted the add-Qwen3_5ForCausalLM branch March 24, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants