Implementation:AUTOMATIC1111 Stable diffusion webui Soft Inpainting

Knowledge Sources	AUTOMATIC1111_Stable_diffusion_webui
Domains	Inpainting, Image Blending, Latent Space
Last Updated	2025-05-15 00:00 GMT

Overview

Implements soft inpainting that seamlessly blends original image content with inpainted content according to mask opacity, using magnitude-aware latent interpolation during denoising and adaptive pixel-level compositing afterward.

Description

This module provides the Script class (an always-on img2img script) and supporting functions for soft inpainting. The core algorithm operates in multiple pipeline stages:

Latent blending (on_mask_blend): Replaces standard hard mask blending with latent_blend, which linearly interpolates latent vectors while separately interpolating their magnitudes using a configurable inpaint_detail_preservation exponent. This preserves contrast that would otherwise be lost with naive linear interpolation. The mask is dynamically scaled by get_modified_nmask based on the current noise level (sigma) and schedule bias parameters.

Adaptive compositing (post_sample): After sampling, generates pixel-level compositing masks by computing L2 distances between original and processed latents, filtering them with weighted_histogram_filter (a generalized convolution supporting weighted percentile-based averaging), and applying sigmoid-like transitions via smootherstep. These masks control how original and inpainted pixels blend in the final image.

Settings: Six configurable parameters are exposed through the UI: Schedule bias (mask_blend_power), Preservation strength (mask_blend_scale), Transition contrast boost (inpaint_detail_preservation), Mask influence (composite_mask_influence), Difference threshold (composite_difference_threshold), and Difference contrast (composite_difference_contrast).

Usage

Enable "Soft inpainting" in the img2img Inpaint settings accordion. This works best with high Mask blur values. Adjust the six parameters to control the balance between original content preservation and inpainted content integration.

Code Reference

Source Location

• Repository: AUTOMATIC1111_Stable_diffusion_webui
• File: extensions-builtin/soft-inpainting/scripts/soft_inpainting.py
• Lines: 1-760

Signature

class SoftInpaintingSettings:
    def __init__(self, mask_blend_power, mask_blend_scale,
                 inpaint_detail_preservation, composite_mask_influence,
                 composite_difference_threshold, composite_difference_contrast): ...

class Script(scripts.Script):
    def title(self): ...
    def show(self, is_img2img): ...
    def ui(self, is_img2img): ...
    def process(self, p, enabled, power, scale, detail_preservation,
                mask_inf, dif_thresh, dif_contr): ...
    def on_mask_blend(self, p, mba, enabled, power, scale,
                      detail_preservation, mask_inf, dif_thresh, dif_contr): ...
    def post_sample(self, p, ps, enabled, power, scale,
                    detail_preservation, mask_inf, dif_thresh, dif_contr): ...
    def postprocess_maskoverlay(self, p, ppmo, enabled, power, scale,
                                detail_preservation, mask_inf, dif_thresh, dif_contr): ...

def latent_blend(settings, a, b, t): ...
def get_modified_nmask(settings, nmask, sigma): ...
def apply_adaptive_masks(settings, nmask, latent_orig, latent_processed,
                         overlay_images, width, height, paste_to): ...
def weighted_histogram_filter(img, kernel, kernel_center,
                              percentile_min=0.0, percentile_max=1.0, min_width=1.0): ...

Import

# This script is auto-discovered by the WebUI script system from:
# extensions-builtin/soft-inpainting/scripts/soft_inpainting.py
# Internal functions can be imported as:
from extensions-builtin.soft_inpainting.scripts.soft_inpainting import (
    latent_blend,
    get_modified_nmask,
    SoftInpaintingSettings
)

I/O Contract

Inputs

Name	Type	Required	Description
enabled	bool	Yes	Whether soft inpainting is enabled
mask_blend_power	float	Yes	Schedule bias controlling when preservation occurs during denoising (0-8, default: 1)
mask_blend_scale	float	Yes	Preservation strength biasing original vs. inpainted content (0-8, default: 0.5)
inpaint_detail_preservation	float	Yes	Contrast boost for interpolated latent magnitudes (1-32, default: 4)
composite_mask_influence	float	Yes	How strongly the original mask biases the difference threshold (0-1, default: 0)
composite_difference_threshold	float	Yes	Distance at which original pixels reach 50% opacity (0-8, default: 0.5)
composite_difference_contrast	float	Yes	Sharpness of transition between blended and unblended regions (0-8, default: 2)

Outputs

Name	Type	Description
blended_latent	torch.Tensor	Modified latent tensor with soft mask blending applied during denoising
masks_for_overlay	list[PIL.Image]	Generated compositing masks for final pixel-level blending
overlay_images	list[PIL.Image]	Modified overlay images with alpha masks applied

Usage Examples

# Soft inpainting is typically used via the WebUI interface.
# Enable it in the img2img tab under "Soft inpainting" accordion.

# Programmatic usage via the API:
import requests
response = requests.post("http://localhost:7860/sdapi/v1/img2img", json={
    "init_images": [base64_image],
    "mask": base64_mask,
    "prompt": "a beautiful landscape",
    "denoising_strength": 0.75,
    "alwayson_scripts": {
        "Soft Inpainting": {
            "args": [True, 1.0, 0.5, 4.0, 0.0, 0.5, 2.0]
        }
    }
})

Related Pages

• Principle:AUTOMATIC1111_Stable_diffusion_webui_Inpainting_Pipeline