🎬 VACE-WAN: Open-source AI Video Creation Platform

VACE-WAN is a powerful, fully open-source AI video creation tool that brings together cutting-edge generative AI models to enable one-click cinematic creation. With just a single theme input, users can generate scripts, narration, scene images, and videos — all seamlessly integrated into a cohesive short film.

💡 Multilingual support included! Create in English or Chinese. 🚀 Powered by open-source models. No subscription. No hidden cost.

---

🌟 Features

• Text-to-Script: Auto-generate creative screenplays with [deepseek-r1-distill-llama-70b]
• Text-to-Image: Generate vivid scene visuals with [Stable Diffusion 2.1]
• Text-to-Speech: Narrate your scenes using [gTTS] (Google Text-to-Speech)
• Image-to-Video: Animate scenes using [Stable Video Diffusion (Img2Vid-XT)]
• Gradio Interface: Intuitive UI for customization or one-click generation
• Full Pipeline: From concept to video — script → image → voice → video

---

📷 Demo Screenshot

---

🛠️ Installation

Ensure your environment includes at least 20MiB GPU memory.

%pip install -U diffusers
%pip install -q gradio torch torchvision groq ffmpeg-python gTTS av
%pip install -U transformers accelerate sentencepiece protobuf opencv-python
%pip install imageio-ffmpeg

🚀 Usage

Run the notebook directly in Jupyter or Colab:

jupyter notebook "VACE-WAN - AI Video Generator.ipynb"

Or deploy it via Gradio!

Basic Workflow:

Enter a theme → e.g., "A robot discovers a beach paradise"

Generate Script → Automatically written screenplay

Generate Voice → Narration from the script

Generate Scene Images → AI-generated cinematic visuals

Render Video → Combine everything into a stunning short film

📚 Models Used

Task	Model
Script Generation	deepseek-r1-distill-llama-70b via Groq API
Image Generation	stabilityai/stable-diffusion-2-1
Speech Generation	gTTS (Google Text-to-Speech)
Video Generation	stabilityai/stable-video-diffusion-img2vid-xt

🎯 Motivation

Inspired by Google I/O 2025 and the revolutionary demo of Veo3 + Flow, this project was born to democratize video generation for everyone. While commercial tools remain powerful but costly and restricted, VACE-WAN aims to be:

💸 Free and open-source

🌍 Language-friendly (supports Chinese & English)

🛠️ Accessible for developers and creators

🧠 Educational for learning generative AI pipelines

⚙️ Hardware Requirements

Recommended: 20MiB+ GPU memory for img2vid

For better stability, consider running locally or with Colab Pro

📂 Project Structure

VACE-WAN/
├── VACE-WAN - 開源 AI 影片創作工具.ipynb   # Main notebook
├── README.md                             # You're reading it!
└── assets/                               # (Optional) Screenshots or outputs

🙌 Contributing

Feel free to fork, submit issues, or contribute improvements! Let’s build the future of open-source AI filmmaking together.