Implementation:AUTOMATIC1111 Stable diffusion webui Hypertile Optimization

Knowledge Sources	AUTOMATIC1111_Stable_diffusion_webui
Domains	Performance Optimization, Attention, Tiling
Last Updated	2025-05-15 00:00 GMT

Overview

Implements the Hypertile optimization that splits self-attention computations in U-Net and VAE models into smaller spatial tiles, reducing the quadratic memory and compute cost of attention layers for high-resolution image generation.

Description

This module provides the core Hypertile algorithm, originally by @tfernd. The main entry point hypertile_hook_model walks the model's named modules, identifies self-attention layers matching known depth-layer name mappings (stored in DEPTH_LAYERS for SD 1.5 and DEPTH_LAYERS_XL for SDXL architectures), and replaces each layer's forward method with a wrapped version produced by self_attn_forward. The wrapper uses einops.rearrange to split input tensors into spatial tiles of a randomly chosen divisor size (seeded for reproducibility via RNG_INSTANCE), runs the original attention forward on the smaller tiles, then reassembles the output. For VAE inputs (4D tensors), tiles are split spatially as (b nh nw) c h w. For U-Net inputs (3D tensors), the spatial dimensions are inferred from the aspect ratio and tiles are split as (b nh nw) (h w) c. Helper functions compute valid tile divisors (get_divisors), find aspect-ratio-matching height/width candidates (find_hw_candidates), and determine the largest power-of-2 tile size for given dimensions (largest_tile_size_available). The HypertileParams dataclass stores per-layer configuration including tile size, swap size, aspect ratio, depth, and enabled state.

Usage

This code is called by the Hypertile script extension (hypertile_script.py) during the image generation pipeline. Users enable it via the "Hypertile" section in the generation settings, specifying maximum tile size, swap size, and maximum depth. The optimization is most effective for images with dimensions that are multiples of 128.

Code Reference

Source Location

• Repository: AUTOMATIC1111_Stable_diffusion_webui
• File: extensions-builtin/hypertile/hypertile.py
• Lines: 1-351

Signature

@dataclass
class HypertileParams:
    depth = 0
    layer_name = ""
    tile_size: int = 0
    swap_size: int = 0
    aspect_ratio: float = 1.0
    forward = None
    enabled = False

def get_divisors(value: int, min_value: int, /, max_options: int = 1) -> list[int]
def random_divisor(value: int, min_value: int, /, max_options: int = 1) -> int
def set_hypertile_seed(seed: int) -> None
def largest_tile_size_available(width: int, height: int) -> int
def iterative_closest_divisors(hw: int, aspect_ratio: float) -> tuple[int, int]
def find_hw_candidates(hw: int, aspect_ratio: float) -> tuple[int, int]
def self_attn_forward(params: HypertileParams, scale_depth=True) -> Callable
def hypertile_hook_model(model: nn.Module, width, height, *, enable=False,
                         tile_size_max=128, swap_size=1, max_depth=3,
                         is_sdxl=False)

Import

from extensions-builtin.hypertile.hypertile import (
    hypertile_hook_model,
    set_hypertile_seed,
    HypertileParams
)

I/O Contract

Inputs

Name	Type	Required	Description
model	nn.Module	Yes	The U-Net or VAE model whose self-attention layers will be hooked
width	int	Yes	Target image width in pixels
height	int	Yes	Target image height in pixels
enable	bool	No	Whether to enable the hypertile optimization (default: False)
tile_size_max	int	No	Maximum tile size in pixels (default: 128)
swap_size	int	No	Number of random divisor options for tile size selection (default: 1, deterministic)
max_depth	int	No	Maximum U-Net depth level to apply tiling (default: 3)
is_sdxl	bool	No	Whether the model is SDXL architecture (default: False)

Outputs

Name	Type	Description
(side effect)	None	Modifies the model in-place by wrapping self-attention forward methods with tiled versions

Usage Examples

from extensions-builtin.hypertile.hypertile import hypertile_hook_model, set_hypertile_seed

# Set the seed for reproducible tile size selection
set_hypertile_seed(42)

# Hook the U-Net model with hypertile optimization
hypertile_hook_model(
    model=unet_model,
    width=1024,
    height=1024,
    enable=True,
    tile_size_max=256,
    swap_size=1,
    max_depth=3,
    is_sdxl=False
)

# Subsequent forward passes through the model will use tiled attention
# The hook persists on the model and can be reconfigured by calling again

Related Pages

• Principle:AUTOMATIC1111_Stable_diffusion_webui_Attention_Optimization