[Feature] Better Inpainting & Outpainting

[iwr-redmond]

InvokeAI made two design choices that have improved inpainting on that platform, both of which could be considered for inclusion in this package:

• Full size inpainting: calculate the size of a bounding box around the inpainting area, copy and upscale just this area to full model resolution, do the inpainting, and then downscale and paste the result to the original area size
• PatchMatch algorithm for outpainting: use the PatchMatch C++ library to fill the blank area where outpainting is about to occur prior to starting the img2img process

Both of these features could be considered for inclusion in this project, and would make this GUI much more useful than many alternatives for refining generated images.

[fszontagh]

I've thought through your ideas.

The current inpainting/outpainting implementation in the GUI allows outpainting by simply resizing the input image. You can adjust the canvas size independently in all four directions, and the newly added areas will be filled using diffusion. (Currently, there are no size limitations, so, for example, you can extend the image width by an additional 1024 pixels. However, a high-end GPU is required for the computations, and the extended pixel dimensions must be divisible at least by 8.)

Glitches may occur if the model struggles with inpainting. If you notice visible edges between the original image and the outpainted area, you can draw lines (e.g., paint over the edge) to create a smoother transition and reduce artifacts. In my tests, I only encountered these "edge glitches" when certain settings, such as the CFG value, were too extreme.

Since the model has access to the entire image (except in tiled diffusion), these issues may be less significant, depending on how well the model handles outpainting.

Currently, i'm focusing on adding more quality-of-life features to the GUI and implementing some missing functionalities from the library, such as text-to-video, PhotoMaker, and Canny preprocessing.

[iwr-redmond]

Currently, i'm focusing on adding more quality-of-life features to the GUI and implementing some missing functionalities from the library, such as text-to-video, PhotoMaker, and Canny preprocessing.

That''s a great place to start.