How to use local data for fine tuning of stable diffusion using dreambooth ?

[HripsimeS]

Hello @soheil-mp

I am trying to test your notebook related to fine tuning of stable diffusion using dreambooth.
https://github.com/soheil-mp/GameAssetLab/blob/master/fine-tuning/stable%20diffusion%20using%20dreambooth/stable-diffusion-fine-turning-on-gaming-assets.ipynb

I was wondering where is the part you can add a local path to the dataset to do fine tuning? For example if I want to use your game characters images with their dataset.csv https://github.com/soheil-mp/GameAssetLab/tree/master/data

How to modify "Fine-tuning Stable Diffusion using Dreambooth" part in the notebook to be able to run it in jupyter notebook and with local data?

Thanks in advance!

[soheil-mp]

Hello @HripsimeS, For the path, you need to change this line inside stable-diffusion-fine-turning-on-gaming-assets.ipynb (which is the path for the image folder):

    --image-path "/kaggle/input/gaming-asset-for-text-to-image-models/images"/ 

[HripsimeS]

@soheil-mp thanks for your quick reply! I am getting this error when I run the Jupyter notebook.

[soheil-mp]

@HripsimeS The error is likely because autotrain isn’t installed or recognized in your environment. Try running this first:

pip install autotrain

[HripsimeS]

@soheil-mp Thanks for your reply. I did instllation following the requirements.txt file pip install autotrain-advanced
But now I get a different type of error, it seems that dreambooth command is not recognized by the autotrain CLI. Probably because the version of autotrain-advanced I have installed does not support DreamBooth training or the correct module for DreamBooth is missing. Any ideas how this can be fixed? Thanks in advance!

I confirm that "Generate Gaming Characters Using the Fine-Tuned Model" part is working well by using trained model that is in the weights folder pytorch_lora_weights.safetensors. But still not possible to do fine-tuning stable diffusion using Dreambooth due to error I get for autotrain.

[soheil-mp]

Dear @HripsimeS, it’s possible that the issue could be related to the version of the autotrain library.

I suggest you to read this blog, it's very informative :)
https://www.datacamp.com/tutorial/fine-tuning-stable-diffusion-xl-with-dreambooth-and-lora

[HripsimeS]

@soheil-mp thanks for your reply! Indeed the issue is the current version of autotrain-advanced which is now 0.8.36. Dreambooth training was removed from this newest version and we need to install much older version to be able to use dreambooth via autotrain.
Can you please check which version of autotrain-advanced did you use for this notebook, this will be the solution. Thanks :)