Implementation:Microsoft DeepSpeedExamples Stable Diffusion Pipeline

Knowledge Sources	Microsoft_DeepSpeedExamples
Domains	Generative AI, Image Generation
Last Updated	2026-02-07 12:00 GMT

Overview

Custom Stable Diffusion pipeline with selective guidance optimization that eliminates unconditional noise prediction in later denoising steps for faster inference.

Description

StableDiffusionPipeline is a locally modified version of the HuggingFace Diffusers StableDiffusionPipeline that introduces a research-driven optimization for text-to-image generation. The pipeline implements the standard Stable Diffusion architecture comprising a VAE (AutoencoderKL) for encoding/decoding images to/from latent space, a CLIP text encoder for processing text prompts, a UNet for iterative denoising, and a scheduler for controlling the diffusion process.

The key optimization is the opt_percentage parameter, which controls what fraction of denoising steps use full classifier-free guidance versus a simplified single-pass approach. During the final portion of denoising (determined by opt_percentage), the pipeline skips the unconditional noise prediction entirely, effectively halving the UNet forward passes for those steps. This is based on the observation that classifier-free guidance has diminishing returns in later iterations when the image is already well-formed.

The pipeline also includes standard features such as VAE slicing and tiling for memory-efficient processing of large batches or high-resolution images, prompt encoding with optional negative prompts, safety checking, and support for latent-space inputs. It inherits from DiffusionPipeline and is compatible with the full Diffusers ecosystem.

Usage

Use this pipeline when running Stable Diffusion inference with DeepSpeed and when you want to optimize inference latency by reducing unnecessary classifier-free guidance computations in later denoising steps.

Code Reference

Source Location

• Repository: Microsoft_DeepSpeedExamples
• File: inference/huggingface/stable-diffusion/local_pipeline_stable_diffusion.py
• Lines: 1-721

Signature

class StableDiffusionPipeline(DiffusionPipeline):
    def __init__(
        self,
        vae: AutoencoderKL,
        text_encoder: CLIPTextModel,
        tokenizer: CLIPTokenizer,
        unet: UNet2DConditionModel,
        scheduler: KarrasDiffusionSchedulers,
        safety_checker: StableDiffusionSafetyChecker,
        feature_extractor: CLIPFeatureExtractor,
        requires_safety_checker: bool = True,
    ):
        ...

    def __call__(
        self,
        prompt: Union[str, List[str]] = None,
        height: Optional[int] = None,
        width: Optional[int] = None,
        num_inference_steps: int = 50,
        guidance_scale: float = 7.5,
        opt_percentage: int = 0,
        negative_prompt: Optional[Union[str, List[str]]] = None,
        num_images_per_prompt: Optional[int] = 1,
        eta: float = 0.0,
        generator: Optional[Union[torch.Generator, List[torch.Generator]]] = None,
        latents: Optional[torch.FloatTensor] = None,
        prompt_embeds: Optional[torch.FloatTensor] = None,
        negative_prompt_embeds: Optional[torch.FloatTensor] = None,
        output_type: Optional[str] = "pil",
        return_dict: bool = True,
        callback: Optional[Callable] = None,
        callback_steps: int = 1,
        cross_attention_kwargs: Optional[Dict[str, Any]] = None,
    ):
        ...

Import

from local_pipeline_stable_diffusion import StableDiffusionPipeline

I/O Contract

Inputs

Name	Type	Required	Description
prompt	str or List[str]	No	Text prompt(s) guiding image generation (required if prompt_embeds not provided)
height	int	No	Height in pixels of generated image (default: unet sample_size * vae_scale_factor)
width	int	No	Width in pixels of generated image (default: unet sample_size * vae_scale_factor)
num_inference_steps	int	No	Number of denoising steps (default: 50)
guidance_scale	float	No	Classifier-free guidance scale; higher values follow prompt more closely (default: 7.5)
opt_percentage	int	No	Percentage of final steps to skip unconditional guidance for speed optimization (default: 0)
negative_prompt	str or List[str]	No	Prompt(s) for negative guidance
num_images_per_prompt	int	No	Number of images per prompt (default: 1)

Outputs

Name	Type	Description
images	StableDiffusionPipelineOutput or tuple	Generated images as PIL Images or numpy arrays, with optional NSFW flags

Usage Examples

Basic Text to Image Generation

import torch
from local_pipeline_stable_diffusion import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
)
pipe = pipe.to("cuda")

# Standard generation
image = pipe("a photo of an astronaut riding a horse on mars").images[0]

# With optimization: skip guidance for last 30% of steps
image = pipe(
    "a photo of an astronaut riding a horse on mars",
    num_inference_steps=50,
    opt_percentage=30
).images[0]