Implementation:Huggingface Peft BOFTModel

Knowledge Sources	Huggingface_Peft
Domains	Deep_Learning, Parameter_Efficient_Finetuning
Last Updated	2026-02-07 14:00 GMT

Overview

Concrete tool for applying Butterfly Orthogonal Finetuning (BOFT) to pretrained transformer models, provided by the Huggingface PEFT library.

Description

BOFTModel is a tuner class that creates a BOFT (Butterfly Orthogonal Finetuning) model from a pretrained transformers model. It applies parameter-efficient orthogonal transformations via butterfly factorization to target modules, supporting both Linear and Conv2d layers. The method is described in "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization" (ICLR 2024).

Usage

BOFTModel is typically created internally by calling get_peft_model with a BOFTConfig. It can also be instantiated directly by passing a base model, a BOFTConfig, and an adapter name.

Code Reference

Source Location

• Repository: Huggingface_Peft
• File: src/peft/tuners/boft/model.py
• Lines: 31-132

Signature

class BOFTModel(BaseTuner):
    prefix: str = "boft_"
    # Inherits __init__ from BaseTuner:
    # def __init__(self, model, config, adapter_name):
    #     ...

Import

from peft.tuners.boft import BOFTModel

I/O Contract

Inputs

Name	Type	Required	Description
model	nn.Module	Yes	The pretrained model to adapt
config	BOFTConfig	Yes	Configuration for the BOFT adapter (block size, butterfly factor, dropout, etc.)
adapter_name	str	Yes	Name identifier for the adapter, defaults to "default"

Outputs

Name	Type	Description
adapted_model	BOFTModel	Model with BOFT adapter layers injected into target Linear and Conv2d modules

Usage Examples

Basic Usage

from peft import get_peft_model, BOFTConfig
from transformers import AutoModelForImageClassification

model = AutoModelForImageClassification.from_pretrained("facebook/dinov2-large", num_labels=100)
config = BOFTConfig(
    boft_block_size=8,
    boft_n_butterfly_factor=1,
    target_modules=["query", "value", "key", "output.dense", "mlp.fc1", "mlp.fc2"],
    boft_dropout=0.1,
    bias="boft_only",
)
model = get_peft_model(model, config)

Related Pages

• Environment:Huggingface_Peft_Python_Core_Dependencies