Implementation:AUTOMATIC1111 Stable diffusion webui Weighted sum and add difference

Knowledge Sources	stable-diffusion-webui
Domains	Model Merging, Weight Interpolation, Tensor Operations
Last Updated	2026-02-08 00:00 GMT

Overview

Concrete tool for performing element-wise weight interpolation between model tensors provided by stable-diffusion-webui.

Description

The weighted_sum and add_difference functions implement the two core interpolation methods for model merging. They are defined as closures within run_modelmerger and dispatched via the theta_funcs dictionary based on the user-selected interpolation method. A helper function get_difference computes the raw difference between two tensors for the "Add difference" method.

The theta_funcs dictionary maps interpolation method names to a tuple of three elements: a filename generator function, an optional first-pass function (theta_func1), and an optional second-pass function (theta_func2). For "Weighted sum", theta_func1 is None (no first pass needed) and theta_func2 is weighted_sum. For "Add difference", theta_func1 is get_difference (computes B - C in a first pass) and theta_func2 is add_difference (adds the scaled difference to A in a second pass).

Usage

These functions are called internally by run_modelmerger during the checkpoint merging process. They are not typically imported or called directly but can be understood as the mathematical core of the merging pipeline.

Code Reference

Source Location

• Repository: stable-diffusion-webui
• File: modules/extras.py
• Lines: L96-128

Signature

def weighted_sum(theta0, theta1, alpha):
    return ((1 - alpha) * theta0) + (alpha * theta1)

def get_difference(theta1, theta2):
    return theta1 - theta2

def add_difference(theta0, theta1_2_diff, alpha):
    return theta0 + (alpha * theta1_2_diff)

theta_funcs = {
    "Weighted sum": (filename_weighted_sum, None, weighted_sum),
    "Add difference": (filename_add_difference, get_difference, add_difference),
    "No interpolation": (filename_nothing, None, None),
}

Import

# These functions are closures defined within run_modelmerger and are not
# independently importable. They are dispatched via the theta_funcs dict.
# To use the merge pipeline:
from modules.extras import run_modelmerger

I/O Contract

Inputs

weighted_sum:

Name	Type	Required	Description
theta0	torch.Tensor	Yes	Weight tensor from primary model A
theta1	torch.Tensor	Yes	Weight tensor from secondary model B
alpha	float	Yes	Interpolation factor, typically in [0, 1]. At 0 returns theta0; at 1 returns theta1.

get_difference:

Name	Type	Required	Description
theta1	torch.Tensor	Yes	Weight tensor from secondary model B
theta2	torch.Tensor	Yes	Weight tensor from tertiary model C

add_difference:

Name	Type	Required	Description
theta0	torch.Tensor	Yes	Weight tensor from primary model A
theta1_2_diff	torch.Tensor	Yes	Pre-computed difference tensor (B - C)
alpha	float	Yes	Scaling factor for the difference vector

Outputs

weighted_sum:

Name	Type	Description
return	torch.Tensor	Interpolated tensor: ((1 - alpha) * theta0) + (alpha * theta1)

get_difference:

Name	Type	Description
return	torch.Tensor	Element-wise difference: theta1 - theta2

add_difference:

Name	Type	Description
return	torch.Tensor	Result tensor: theta0 + (alpha * theta1_2_diff)

Dispatch Mechanism

The theta_funcs dictionary controls the two-pass merge strategy:

Method	theta_func1 (Pass 1: B,C)	theta_func2 (Pass 2: A, result)
"Weighted sum"	None (no first pass)	weighted_sum(A, B, alpha)
"Add difference"	get_difference(B, C)	add_difference(A, B-C, alpha)
"No interpolation"	None	None (copy A only)

The two-pass design separates the potentially expensive B-C difference computation (which can be released from memory after pass 1) from the A + alpha*(B-C) combination in pass 2.

Usage Examples

Basic Usage

import torch

# Weighted sum: blend two tensors at 30% model B
theta_a = torch.randn(512, 512)
theta_b = torch.randn(512, 512)
alpha = 0.3
result = ((1 - alpha) * theta_a) + (alpha * theta_b)

# Add difference: transfer B's specialization over C to A
theta_a = torch.randn(512, 512)
theta_b = torch.randn(512, 512)  # specialized model
theta_c = torch.randn(512, 512)  # base model
alpha = 1.0
diff = theta_b - theta_c
result = theta_a + (alpha * diff)

Within the Merge Pipeline

# The dispatch is handled internally:
filename_generator, theta_func1, theta_func2 = theta_funcs[interp_method]

# Pass 1: compute B - C (only for "Add difference")
if theta_func1:
    for key in theta_1.keys():
        if key in theta_2:
            theta_1[key] = theta_func1(theta_1[key], theta_2.get(key))

# Pass 2: merge with A
for key in theta_0.keys():
    if theta_1 and key in theta_1:
        theta_0[key] = theta_func2(theta_0[key], theta_1[key], multiplier)

Related Pages

Implements Principle

• Principle:AUTOMATIC1111_Stable_diffusion_webui_Weight_interpolation_methods