Some questions

[scarbain]

Hi,

Thanks for sharing the code.

Have you run a finetuning of Ella on SD1.5 ?

Also, shouldn't it be trained on only one timestep instead of a full generation ? And in the paper they mentioned a weight decay of 0.01

And perhaps using a training script from diffusers as the base could be better, it would allow using xformers, different dtypes, batch size & grad acc, adam decay, ...

I'm currently running a finetune of the existing weights for SD1.5 as a test (LR 1e-5, xformers + fp16 for the pipeline and fp16 for the T5encoder), I'll let it run for a few hours.

[DataCTE]

Thank you for raising this issue. We understand your concern and appreciate you bringing it to our attention.

Our primary goal was to release a working base implementation as quickly as possible to ensure transparency with the public regarding our progress and methods. We acknowledge that the current implementation may not be optimal and is merely the first step in the process.

We want to assure you that we are actively working on improving the implementation. Currently, we have an employee thoroughly reviewing the relevant research paper and focusing on developing a proper SDXL implementation. Once that is complete, we will circle back to refine the values and pointers specifically for high-quality outcomes in the SD1.5 training.

Please bear with us as we continue to iterate and enhance the codebase. We remain committed to transparency and will keep the community updated on our progress.

If you have any further questions or concerns, please don't hesitate to reach out. We value your feedback and contributions to making this project the best it can be.

[scarbain]

What a corporate response !

Thanks, and just to let you know, it seems the authors of Ella are planning to release the training/finetuning code : TencentQQGYLab/ELLA#27