Implementation:Huggingface Peft HRAModel

Knowledge Sources	Huggingface_Peft
Domains	Deep_Learning, Parameter_Efficient_Finetuning
Last Updated	2026-02-07 14:00 GMT

Overview

Concrete tool for applying Householder Reflection Adaptation (HRA) to pretrained models, provided by the Huggingface PEFT library.

Description

HRAModel is a tuner class that creates a Householder Reflection Adaptation model from a pretrained transformers model. It applies orthogonal transformations via Householder reflections to target Linear and Conv2d layers. The method is described in detail in https://huggingface.co/papers/2405.17484 and supports optional Gram-Schmidt orthogonalization.

Usage

HRAModel is typically created internally by calling get_peft_model with an HRAConfig. It can also be instantiated directly by passing a base model, an HRAConfig, and an adapter name.

Code Reference

Source Location

• Repository: Huggingface_Peft
• File: src/peft/tuners/hra/model.py
• Lines: 24-132

Signature

class HRAModel(BaseTuner):
    prefix: str = "hra_"
    # Inherits __init__ from BaseTuner:
    # def __init__(self, model, config, adapter_name):
    #     ...

Import

from peft.tuners.hra import HRAModel

I/O Contract

Inputs

Name	Type	Required	Description
model	nn.Module	Yes	The pretrained model to adapt
config	HRAConfig	Yes	Configuration for the HRA adapter (r, apply_GS, init_weights)
adapter_name	str	Yes	Name identifier for the adapter, defaults to "default"

Outputs

Name	Type	Description
adapted_model	HRAModel	Model with HRA adapter layers injected into target Linear and Conv2d modules

Usage Examples

Basic Usage

from peft import get_peft_model, HRAConfig
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model-name")
config = HRAConfig(
    r=8,
    target_modules=["k_proj", "q_proj", "v_proj", "out_proj", "fc1", "fc2"],
    init_weights=True,
)
model = get_peft_model(model, config)

Related Pages

• Environment:Huggingface_Peft_Python_Core_Dependencies