Implementation:Zai org CogVideo Inference Load Lora Weights

Overview

Concrete tool for loading and fusing LoRA adapter weights during CogVideoX inference provided by the diffusers library. This enables customized video generation from fine-tuned models.

Source

inference/cli_demo.py:L128-132

Signature

pipe.load_lora_weights(
    lora_path: str,
    weight_name: str = "pytorch_lora_weights.safetensors",
    adapter_name: str = "test_1"
)
pipe.fuse_lora(
    components: List[str] = ["transformer"],
    lora_scale: float = 1.0
)

Key Parameters

Parameter	Type	Default	Description
lora_path	str	(required)	Path to directory containing LoRA weight files
weight_name	str	"pytorch_lora_weights.safetensors"	Filename of the LoRA weights within the directory
adapter_name	str	"test_1"	Identifier for the loaded adapter (used for multi-adapter management)
components	List[str]	["transformer"]	Model components to apply LoRA fusion to
lora_scale	float	1.0	Scaling factor for the LoRA adaptation strength

Inputs

Loaded pipeline -- A CogVideoXPipeline instance from from_pretrained()
LoRA weight file -- A .safetensors file containing the trained LoRA adapter weights

Outputs

Pipeline with LoRA weights fused into transformer -- The same pipeline instance with adapter weights permanently merged into the transformer model weights

Usage Example

from diffusers import CogVideoXPipeline
import torch

# Load base pipeline
pipe = CogVideoXPipeline.from_pretrained(
    "THUDM/CogVideoX-5b",
    torch_dtype=torch.bfloat16
)

# Load LoRA adapter weights
pipe.load_lora_weights(
    "/path/to/lora/checkpoint",
    weight_name="pytorch_lora_weights.safetensors",
    adapter_name="my_adapter"
)

# Fuse LoRA weights into transformer for zero-overhead inference
pipe.fuse_lora(
    components=["transformer"],
    lora_scale=1.0
)

Import

from diffusers import CogVideoXPipeline  # load_lora_weights is a mixin method

External Dependencies

diffusers -- Provides the pipeline and LoRA loading mixin
peft -- Provides the underlying LoRA implementation and weight manipulation utilities

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment