feat: add Flux.2 VAE proj matrix for previews#1017
Conversation
|
Not working, let me look into it. I think the latents are patchified by 2x2 patches or something Edit: fixed |
|
Perhaps placing the logic in preview_latent_video could make the code cleaner. |
|
@leejet is it good now or should I actually move the calls to |
|
I think the best approach is to move unpatchify_latents logic into preview_latent_video, which would eliminate the need to call repatchify_latents. |
I will look into that soon. Maybe there's a way to avoid allcoating a temp buffer too by doing so, that would be great |
| uint32_t dim = latents->ne[ggml_n_dims(latents) - 1]; | ||
|
|
||
| if (preview_mode == PREVIEW_PROJ) { | ||
| const float(*latent_rgb_proj)[channel] = nullptr; |
There was a problem hiding this comment.
Not really related to the PR, but: maybe move the proj matrix selection to a virtual VAE object member, to avoid the decision tree here?
There was a problem hiding this comment.
Indeed, that would probably be cleaner. Maybe in a follow-up PR.
|
Thank you for your contribution. |
Seems like the proj is harder to train for this new VAE, reminds me of SDXL.