This repository contains code for generating SVGs using diffusion models. The pipeline first generates images with a fine-tune of Stable Diffusion 3.5 Medium that has been trained with DRaFT to produce high-quality, SVG-style outputs. The images are then heuristically converted to rough SVGs, which are further refined using diffvg, a differentiable rasterizer.
This method was used to win 4th place in the Drawing with LLMs kaggle competition. It was the only entrant in the top 5 that didn't "attack" the evaluation metrics: 1st and 2nd place figured out how to hide text in their SVGs while 3rd and 5th place found ways to directly optimize for the aesthetic score. You can read more about the kaggle competition entry in this discussion post.
To train a LoRA with DRaFT, run draft.py. The dataset of prompts used for training is available
here. A Colab notebook for fine-tuning is available here (use an A100 runtime). The inference code is available on kaggle through this package.
Note: in addition to the packages in requirements.txt, you'll need to install diffvg (https://github.com/BachiLi/diffvg) separately.