ip adapter and controlnet pipelines

[loboere]

This has a lot of potential if used together with ip adapter and controlnet
However, the current pipeline is very old and does not support the new features.
can you please update and add pipelines for ip adapter and controlnet

[CesarERamosMedina]

With the way the latents are calculated in diff diff, I am unsure whether this would work out of the box for controlnet and ip adapter pipelines. My early experiments just output noise, even after a large number of steps.

[exx8]

Hi,
Sorry for the delay and thanks for your interest in diff diff.
I actually tested now the paper with ControlNet and it works!
I haven't decided what is my support plan for this repo yet.
As fields progress rapidly, I believe we need a lasting solution, especially considering that more papers expected to work with diff diff are expected to be released.
I will try to speak with https://github.com/huggingface/diffusers 's maintainers.

[berkanz]

Hi, do you plan to share the code with controlnet implementation? I experienced also a rough noise problem similar to @CesarERamosMedina, especially with white map images.

[exx8]

I'm considering sharing my original, non-optimized implementation with the public and asking for assistance from the Diffusers team. They've done an amazing job, particularly with the introduction of diff diff to SD3 and HunyuanDiT. I plan to email them to see if they're interested. Alternatively, you could open an issue in the Diffusers repository, and I'll try to help as well.
https://github.com/huggingface/diffusers

Thank you for your interest in diff-diff!