Implementation:Huggingface Peft DeloraModel

Knowledge Sources	Huggingface_Peft
Domains	Deep_Learning, Parameter_Efficient_Finetuning
Last Updated	2026-02-07 14:00 GMT

Overview

Concrete tool for applying DeLoRA (Decomposed Low-Rank Adaptation) to pretrained transformer models, provided by the Huggingface PEFT library.

Description

DeloraModel is a tuner class that creates a DeLoRA model from a pretrained transformers model. It injects decomposed low-rank adapter layers into target Linear modules, supporting per-module rank and lambda patterns. The class belongs to the LoRA method family with decomposition-based extensions for improved parameter efficiency.

Usage

DeloraModel is typically created internally by calling get_peft_model with a DeloraConfig. It can also be instantiated directly by passing a base model, a DeloraConfig, and an adapter name.

Code Reference

Source Location

• Repository: Huggingface_Peft
• File: src/peft/tuners/delora/model.py
• Lines: 28-106

Signature

class DeloraModel(BaseTuner):
    prefix: str = "delora_"
    # Inherits __init__ from BaseTuner:
    # def __init__(self, model, config, adapter_name):
    #     ...

Import

from peft.tuners.delora import DeloraModel

I/O Contract

Inputs

Name	Type	Required	Description
model	nn.Module	Yes	The pretrained model to adapt
config	DeloraConfig	Yes	Configuration for the DeLoRA adapter (r, delora_lambda, module_dropout, rank_pattern, lambda_pattern)
adapter_name	str	Yes	Name identifier for the adapter, defaults to "default"

Outputs

Name	Type	Description
adapted_model	DeloraModel	Model with DeLoRA adapter layers injected into target Linear modules

Usage Examples

Basic Usage

from peft import get_peft_model, DeloraConfig
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model-name")
config = DeloraConfig(
    r=8,
    target_modules=["q_proj", "v_proj"],
)
model = get_peft_model(model, config)

Related Pages

• Environment:Huggingface_Peft_Python_Core_Dependencies