Implementation:Huggingface Peft BOFTConfig

Knowledge Sources	Huggingface_Peft
Domains	Deep_Learning, Parameter_Efficient_Finetuning
Last Updated	2026-02-07 14:00 GMT

Overview

Concrete configuration dataclass for the BOFT (Butterfly Orthogonal Fine-Tuning) PEFT adapter method provided by the Huggingface PEFT library.

Description

BOFTConfig controls the hyperparameters for the BOFT adapter, which applies parameter-efficient orthogonal finetuning via butterfly factorization to selected model layers. It defines block sizes, butterfly factors, dropout, bias handling, and layer targeting options. This class extends PeftConfig and is based on the paper "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization" (ICLR 2024).

Usage

Import and use BOFTConfig when you want to apply butterfly-factorized orthogonal fine-tuning to a pretrained model, which provides parameter-efficient adaptation while preserving orthogonality constraints on the weight updates.

Code Reference

Source Location

• Repository: Huggingface_Peft
• File: src/peft/tuners/boft/config.py
• Lines: 27-161

Signature

@dataclass
class BOFTConfig(PeftConfig):
    boft_block_size: int = 4
    boft_block_num: int = 0
    boft_n_butterfly_factor: int = 1
    target_modules: Optional[Union[list[str], str]] = None
    exclude_modules: Optional[Union[list[str], str]] = None
    boft_dropout: float = 0.0
    fan_in_fan_out: bool = False
    bias: str = "none"
    modules_to_save: Optional[list[str]] = None
    init_weights: bool = True
    layers_to_transform: Optional[Union[list[int], int]] = None
    layers_pattern: Optional[Union[list[str], str]] = None

Import

from peft import BOFTConfig

I/O Contract

Inputs

Name	Type	Default	Description
boft_block_size	int	4	BOFT block size across different layers. Only specify this or boft_block_num, not both.
boft_block_num	int	0	Number of BOFT blocks per injected layer. Only specify this or boft_block_size, not both.
boft_n_butterfly_factor	int	1	Number of butterfly factors. When set to 1, BOFT is equivalent to vanilla OFT.
target_modules	Optional[Union[list[str], str]]	None	Module names or regex to apply adapter to.
exclude_modules	Optional[Union[list[str], str]]	None	Module names or regex to exclude from adapter.
boft_dropout	float	0.0	Multiplicative dropout probability, setting OFT blocks to identity during training.
fan_in_fan_out	bool	False	Set True if layer stores weight as (fan_in, fan_out), e.g. Conv1D.
bias	str	"none"	Bias type. Can be 'none', 'all', or 'boft_only'.
modules_to_save	Optional[list[str]]	None	Additional modules to set as trainable and save in the final checkpoint.
init_weights	bool	True	Whether to initialize BOFT layer weights with default initialization.
layers_to_transform	Optional[Union[list[int], int]]	None	Specific layer indexes to apply BOFT transformations to.
layers_pattern	Optional[Union[list[str], str]]	None	Layer pattern name for targeting nn.ModuleList when layers_to_transform is set.

Outputs

Name	Type	Description
instance	BOFTConfig	Configuration object to pass to get_peft_model

Usage Examples

Basic Configuration

from peft import BOFTConfig, get_peft_model
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model-name")
config = BOFTConfig(
    boft_block_size=4,
    boft_n_butterfly_factor=2,
    target_modules=["q_proj", "v_proj"],
    boft_dropout=0.1,
)
model = get_peft_model(model, config)

Related Pages

• Environment:Huggingface_Peft_Python_Core_Dependencies