Implementation:Huggingface Diffusers Prepare Control Image

Property	Value
Implementation Name	Prepare Control Image
Type	API Doc
Workflow	ControlNet_Guided_Generation
Related Principle	Huggingface_Diffusers_Conditioning_Image_Preparation
Source File	src/diffusers/pipelines/controlnet/pipeline_controlnet.py
Lines	L796-L824
Status	Active
Implements	Principle:Huggingface_Diffusers_Conditioning_Image_Preparation

API Signature

def prepare_image(
    self,
    image,
    width,
    height,
    batch_size,
    num_images_per_prompt,
    device,
    dtype,
    do_classifier_free_guidance=False,
    guess_mode=False,
) -> torch.Tensor:

Class: StableDiffusionControlNetPipeline

Import:

from diffusers import StableDiffusionControlNetPipeline

Parameters

Parameter	Type	Default	Description
image	PipelineImageInput	required	The ControlNet conditioning image. Accepts PIL.Image.Image, np.ndarray, torch.Tensor, or lists thereof.
width	int	required	Target width for resizing the control image.
height	int	required	Target height for resizing the control image.
batch_size	int	required	Effective batch size (typically batch_size * num_images_per_prompt).
num_images_per_prompt	int	required	Number of images to generate per prompt.
device	torch.device	required	Target device for the output tensor.
dtype	torch.dtype	required	Target dtype for the output tensor (typically the ControlNet model dtype).
do_classifier_free_guidance	bool	False	Whether to duplicate the image for classifier-free guidance.
guess_mode	bool	False	When True, skips image duplication even with CFG enabled.

Return Value

Type	Description
torch.Tensor	Preprocessed control image tensor of shape (batch, channels, height, width). When do_classifier_free_guidance=True and guess_mode=False, the batch dimension is doubled.

I/O Contract

Direction	Format	Shape / Type
Input	PIL Image, numpy array, or torch Tensor	Variable; single image or batch
After preprocess	torch.Tensor (float32)	(N, C, H, W) where H, W match target dimensions
After repeat	torch.Tensor	(batch_size, C, H, W) or (batch_size * num_images_per_prompt, C, H, W)
After device/dtype cast	torch.Tensor	Same shape, on target device with target dtype
After CFG duplication	torch.Tensor	Batch dim doubled if CFG active and not guess mode

Source Code Analysis

def prepare_image(
    self,
    image,
    width,
    height,
    batch_size,
    num_images_per_prompt,
    device,
    dtype,
    do_classifier_free_guidance=False,
    guess_mode=False,
):
    image = self.control_image_processor.preprocess(image, height=height, width=width).to(dtype=torch.float32)
    image_batch_size = image.shape[0]

    if image_batch_size == 1:
        repeat_by = batch_size
    else:
        # image batch size is the same as prompt batch size
        repeat_by = num_images_per_prompt

    image = image.repeat_interleave(repeat_by, dim=0)

    image = image.to(device=device, dtype=dtype)

    if do_classifier_free_guidance and not guess_mode:
        image = torch.cat([image] * 2)

    return image

Source: src/diffusers/pipelines/controlnet/pipeline_controlnet.py, lines 796-824.

Step-by-Step Walkthrough

1. Preprocessing: The control_image_processor (a VaeImageProcessor) handles resizing to the target height and width, normalizing pixel values, and converting to a tensor. The result is cast to float32 for numerical stability.
2. Batch expansion: If a single image is provided (image_batch_size == 1), it is repeated batch_size times. If a batch of images is provided (matching the prompt batch size), each image is repeated num_images_per_prompt times.
3. Device and dtype transfer: The tensor is moved to the target device and cast to the model's dtype (e.g., float16).
4. CFG duplication: When classifier-free guidance is active and guess mode is off, the image tensor is concatenated with itself along dim=0, doubling the batch to serve both unconditional and conditional branches.

Usage Examples

Basic Text-to-Image with Canny ControlNet

import cv2
import numpy as np
import torch
from PIL import Image
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from diffusers.utils import load_image

# Load and prepare the conditioning image
source_image = load_image(
    "https://hf.co/datasets/huggingface/documentation-images/resolve/main/diffusers/input_image_vermeer.png"
)
np_image = np.array(source_image)

# Generate Canny edge map
canny_image = cv2.Canny(np_image, 100, 200)
canny_image = canny_image[:, :, None]
canny_image = np.concatenate([canny_image, canny_image, canny_image], axis=2)
canny_image = Image.fromarray(canny_image)

# Load the pipeline
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16
)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16,
)
pipe.enable_model_cpu_offload()

# Generate -- prepare_image is called internally by __call__
output = pipe(
    "a renaissance painting of a woman",
    image=canny_image,
    num_inference_steps=30,
).images[0]

Directly Calling prepare_image

# Manually prepare the control image (for advanced use cases)
prepared = pipe.prepare_image(
    image=canny_image,
    width=512,
    height=512,
    batch_size=1,
    num_images_per_prompt=1,
    device=pipe._execution_device,
    dtype=controlnet.dtype,
    do_classifier_free_guidance=True,
    guess_mode=False,
)
# prepared.shape: torch.Size([2, 3, 512, 512])  -- doubled for CFG

Notes

• The prepare_image method in StableDiffusionControlNetPipeline corresponds to prepare_control_image in StableDiffusionControlNetImg2ImgPipeline; the logic is identical but the naming differs between pipeline variants.
• The control_image_processor is instantiated with do_convert_rgb=True, ensuring grayscale inputs are converted to 3-channel RGB.
• Input validation is performed separately in check_image() and check_inputs() before this method is called.

Related Pages

• Huggingface_Diffusers_Conditioning_Image_Preparation -- Principle: theory of spatial conditioning signal preparation
• Huggingface_Diffusers_ControlNet_Pipeline_Call -- The pipeline __call__ method that invokes prepare_image
• Huggingface_Diffusers_ControlNetModel_Forward -- The forward pass that consumes the prepared image