SD1.5 - pipeline_controlnet_img2img and pipeline_controlnet_inpaint are mixing variables "image" and "control_image"

[buddyforgothisemail]

Describe the bug

"image" is used in pipeline_controlnet for a list of controlnet images.
"control_image" is used in pipeline_controlnet_img2img and pipeline_controlnet_inpaint for a list of controlnet images.
img2img and inpaint are missing "control_image" in check_inputs function causing an error

The img2img and inpaint check_inputs function is checking against "image" twice instead of "image" and "control_image".
this section from both files is for the "control_image" list but is checking against "image"

        if not isinstance(image, list):
            raise TypeError("For multiple controlnets: `image` must be type `list`")

        # When `image` is a nested list:
        # (e.g. [[canny_image_1, pose_image_1], [canny_image_2, pose_image_2]])
        elif any(isinstance(i, list) for i in image):
            raise ValueError("A single batch of multiple conditionings are supported at the moment.")
        elif len(image) != len(self.controlnet.nets):
            raise ValueError(
                f"For multiple controlnets: `image` must have the same length as the number of controlnets, but got {len(image)} images and {len(self.controlnet.nets)} ControlNets."
            )

        for image_ in image:
            self.check_image(image_, prompt, prompt_embeds)

quick fix - add "control_image" to def check_inputs and self.check_inputs and replace the above with the following

        if not isinstance(control_image, list):
            raise TypeError("For multiple controlnets: `control_image` must be type `list`")

        # When `control_image` is a nested list:
        # (e.g. [[canny_image_1, pose_image_1], [canny_image_2, pose_image_2]])
        elif any(isinstance(i, list) for i in control_image):
            raise ValueError("A single batch of multiple conditionings are supported at the moment.")
        elif len(control_image) != len(self.controlnet.nets):
            raise ValueError(
                f"For multiple controlnets: `control_image` must have the same length as the number of controlnets, but got {len(control_image)} images and {len(self.controlnet.nets)} ControlNets."
            )

        for image_ in control_image:
            self.check_image(image_, prompt, prompt_embeds)

@yiyixuxu @asomoza @DN6 @sayakpaul

Reproduction

no reproducible code

Logs

System Info

• 🤗 Diffusers version: 0.37.1
• Platform: Windows-10-10.0.19045-SP0
• Running on Google Colab?: No
• Python version: 3.10.6
• PyTorch version (GPU?): 2.12.1+cpu (False)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 1.9.1
• Transformers version: 5.5.0
• Accelerate version: 1.12.0
• PEFT version: 0.18.1
• Bitsandbytes version: not installed
• Safetensors version: 0.8.0

Who can help?

No response

[buddyforgothisemail]

Note - I think the problem started because the img2img_controlnet and inpaint_controlnet were most likely copied from the base pipeline_controlnet.py where "image" worked because it was not already used, but in img2img and inpaint pipeline, "image" represents the users RGB image and not a list of canny, pose, depth images.
The issue wont happen with a single image and single controlnet if everything is passed properly. It will happen once there are mutiple controlnets and multiple "control_images".