Implementation:Huggingface Diffusers Prior Image Generation

Metadata

Knowledge Sources	Huggingface Diffusers DreamBooth LoRA Training
Domains	Generative_AI Personalization Data_Generation
Last Updated	2026-02-13 00:00 GMT

Overview

The pattern for generating class-prior images using the frozen pretrained model before DreamBooth fine-tuning begins. This implementation loads a DiffusionPipeline from the pretrained checkpoint, samples images conditioned on the class prompt, and saves them to the class data directory for later use as prior preservation regularization data.

Description

Before the DreamBooth training loop starts, the script checks whether sufficient class-prior images already exist in class_data_dir. If the number of existing images is less than num_class_images, a DiffusionPipeline is loaded from the pretrained model weights, and new images are generated in batches using the class prompt.

The generation process:

1. Check class_images_dir and count existing images.
2. If cur_class_images < num_class_images, load the full pretrained pipeline with DiffusionPipeline.from_pretrained().
3. Create a PromptDataset that repeats the class prompt for the required number of new images.
4. Wrap the prompt dataset in a DataLoader with sample_batch_size.
5. Iterate through batches, calling pipeline(prompt) to generate images.
6. Save each generated image with a content-hash filename to avoid collisions.
7. Delete the pipeline and free GPU memory before proceeding to training.

The precision for prior generation can be configured independently of the training precision via --prior_generation_precision.

Usage

This pattern is invoked automatically when --with_prior_preservation is enabled and the class data directory has fewer images than --num_class_images:

accelerate launch train_dreambooth_lora.py \
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
  --instance_data_dir="./my_subject" \
  --instance_prompt="a photo of sks dog" \
  --class_data_dir="./dog_class_images" \
  --class_prompt="a photo of dog" \
  --with_prior_preservation \
  --num_class_images=200 \
  --sample_batch_size=4 \
  --prior_generation_precision="fp16"

Code Reference

Source Location

• Repository: huggingface/diffusers
• File: examples/dreambooth/train_dreambooth_lora.py (lines 810--853)

Signature

# Prior image generation pattern (not a standalone function; inline in main())

if args.with_prior_preservation:
    class_images_dir = Path(args.class_data_dir)
    if not class_images_dir.exists():
        class_images_dir.mkdir(parents=True)
    cur_class_images = len(list(class_images_dir.iterdir()))

    if cur_class_images < args.num_class_images:
        pipeline = DiffusionPipeline.from_pretrained(
            args.pretrained_model_name_or_path,
            torch_dtype=torch_dtype,
            safety_checker=None,
            revision=args.revision,
            variant=args.variant,
        )
        pipeline.set_progress_bar_config(disable=True)

        num_new_images = args.num_class_images - cur_class_images
        sample_dataset = PromptDataset(args.class_prompt, num_new_images)
        sample_dataloader = torch.utils.data.DataLoader(
            sample_dataset, batch_size=args.sample_batch_size
        )

        for example in tqdm(sample_dataloader, desc="Generating class images"):
            images = pipeline(example["prompt"]).images
            for i, image in enumerate(images):
                hash_image = insecure_hashlib.sha1(image.tobytes()).hexdigest()
                image_filename = class_images_dir / f"{example['index'][i] + cur_class_images}-{hash_image}.jpg"
                image.save(image_filename)

        del pipeline
        free_memory()

Import

from diffusers import DiffusionPipeline
from diffusers.training_utils import free_memory
from huggingface_hub.utils import insecure_hashlib

I/O Contract

Inputs

Input Contract

Name	Type	Description
args.pretrained_model_name_or_path	str	Model identifier or path for the pretrained diffusion pipeline.
args.class_data_dir	str	Directory where class-prior images are stored and generated.
args.class_prompt	str	Generic class prompt used to generate prior images, e.g., "a photo of dog".
args.num_class_images	int	Target number of class-prior images. Generation occurs only if the directory has fewer.
args.sample_batch_size	int	Batch size for image generation (default 4).
args.prior_generation_precision	str	Precision for generation: "fp32", "fp16", or "bf16".

Outputs

Output Contract

Name	Type	Description
Generated images	JPEG files on disk	Class-prior images saved to class_data_dir with content-hash filenames, e.g., 0-a1b2c3d4.jpg.

Usage Examples

Example 1: Generating 200 Class-Prior Dog Images

When the class data directory is empty, the full 200 images are generated before training.

# Equivalent to what happens internally when the script detects
# 0 existing images and num_class_images=200

pipeline = DiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16,
    safety_checker=None,
)
pipeline.to("cuda")

for batch_prompts in dataloader:  # batches of "a photo of dog"
    images = pipeline(batch_prompts).images
    for img in images:
        img.save(class_images_dir / f"{idx}-{hash}.jpg")

del pipeline
free_memory()

Example 2: Incremental Generation

If the directory already contains 150 images and num_class_images=200, only 50 new images are generated.

cur_class_images = 150  # from len(list(class_images_dir.iterdir()))
num_new_images = 200 - 150  # = 50
sample_dataset = PromptDataset("a photo of dog", num_new_images)
# Only 50 images will be generated in batches of sample_batch_size

Related Pages

• Principle:Huggingface_Diffusers_Prior_Preservation