Implementation:Huggingface Peft Constants

Knowledge Sources	Huggingface_Peft
Domains	Deep_Learning, Parameter_Efficient_Finetuning
Last Updated	2026-02-07 14:00 GMT

Overview

Concrete tool for defining default target module mappings and miscellaneous constants used throughout the Huggingface PEFT library.

Description

The constants.py module serves as a centralized registry of default target module mappings and configuration constants for the entire PEFT library. It provides dictionary mappings that map transformer model architecture names (such as "llama", "gpt2", "bert", "mistral", etc.) to the specific module names that each PEFT method should target by default. These mappings cover a wide range of PEFT methods including LoRA, BoFT, LoHa, LoKr, OFT, HRA, Poly, IA3, AdaLoRA, VBLoRA, FourierFT, SHIRA, VeRA, LNTuning, OSF, WaveFT, BONE, C3A, DeLORA, MiSS, RandLoRA, and ROAD.

In addition to target module mappings, the file defines helper functions for prefix-tuning post-processing of specific model architectures. The bloom_model_postprocess_past_key_value function handles the special key-value cache format used by BLOOM models, and starcoder_model_postprocess_past_key_value handles the StarCoder format. These are conditionally registered into the TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING dictionary based on the installed transformers version.

The module also defines miscellaneous constants such as standard file names for adapter weights (WEIGHTS_NAME, SAFETENSORS_WEIGHTS_NAME, CONFIG_NAME), lists of common embedding and classification head layer names, and optimization thresholds like MIN_TARGET_MODULES_FOR_OPTIMIZATION.

Usage

This module is used internally by PEFT whenever default target modules need to be determined for a given model architecture and PEFT method. When a user creates a PEFT configuration without explicitly specifying target_modules, the library looks up the appropriate mapping dictionary to determine which layers to adapt. It is also used during model saving and loading to reference standard file names for adapter weights and configurations.

Code Reference

Source Location

• Repository: Huggingface_Peft
• File: src/peft/utils/constants.py
• Lines: 1-362

Signature

def bloom_model_postprocess_past_key_value(past_key_values):
    ...

def starcoder_model_postprocess_past_key_value(past_key_values):
    ...

# Key mapping dictionaries:
TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING = {}
TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_IA3_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_IA3_FEEDFORWARD_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_ADALORA_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_LNTUNING_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_VBLORA_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_OSF_TARGET_MODULES_MAPPING = { ... }
TRANSFORMERS_MODELS_TO_WAVEFT_TARGET_MODULES_MAPPING = { ... }

# Misc constants:
WEIGHTS_NAME = "adapter_model.bin"
SAFETENSORS_WEIGHTS_NAME = "adapter_model.safetensors"
CONFIG_NAME = "adapter_config.json"
EMBEDDING_LAYER_NAMES = ["embed_tokens", "lm_head"]
SEQ_CLS_HEAD_NAMES = ["score", "classifier"]
INCLUDE_LINEAR_LAYERS_SHORTHAND = "all-linear"
TOKENIZER_CONFIG_NAME = "tokenizer_config.json"
DUMMY_TARGET_MODULES = "dummy-target-modules"
DUMMY_MODEL_CONFIG = {"model_type": "custom"}
MIN_TARGET_MODULES_FOR_OPTIMIZATION = 20

Import

from peft.utils.constants import (
    TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING,
    TRANSFORMERS_MODELS_TO_IA3_TARGET_MODULES_MAPPING,
    TRANSFORMERS_MODELS_TO_ADALORA_TARGET_MODULES_MAPPING,
    WEIGHTS_NAME,
    SAFETENSORS_WEIGHTS_NAME,
    CONFIG_NAME,
    EMBEDDING_LAYER_NAMES,
    INCLUDE_LINEAR_LAYERS_SHORTHAND,
    MIN_TARGET_MODULES_FOR_OPTIMIZATION,
)

I/O Contract

Inputs

Name	Type	Required	Description
past_key_values	torch.Tensor	Yes (for postprocess functions)	The past key values tensor from a prefix-tuning forward pass, used by bloom_model_postprocess_past_key_value and starcoder_model_postprocess_past_key_value.

Outputs

Name	Type	Description
result	tuple	Postprocessed past key values formatted for the specific model architecture (Bloom returns tuple of key-value pairs; StarCoder returns tuple of reshaped tensors).
Mapping dictionaries	dict[str, list[str]]	Each mapping dictionary maps model type strings to lists of target module name strings.
File name constants	str	Standard file name strings used for adapter weight serialization.

Usage Examples

from peft.utils.constants import TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING

# Look up default LoRA target modules for a LLaMA model
target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING["llama"]
# Returns: ["q_proj", "v_proj"]

# Look up default LoRA target modules for GPT-2
target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING["gpt2"]
# Returns: ["c_attn"]

# Use standard file names for saving adapters
from peft.utils.constants import SAFETENSORS_WEIGHTS_NAME, CONFIG_NAME
print(SAFETENSORS_WEIGHTS_NAME)  # "adapter_model.safetensors"
print(CONFIG_NAME)                # "adapter_config.json"

Related Pages

• Environment:Huggingface_Peft_Python_Core_Dependencies