Environment:Huggingface Alignment handbook Python PEFT
| Knowledge Sources | |
|---|---|
| Domains | NLP, Deep_Learning, Optimization |
| Last Updated | 2026-02-07 00:00 GMT |
Overview
Python environment with PEFT >= 0.16.0 providing LoRA adapter configuration and injection for parameter-efficient fine-tuning.
Description
The PEFT (Parameter-Efficient Fine-Tuning) library provides the LoRA adapter infrastructure used by the alignment-handbook's QLoRA and LoRA training paths. The get_peft_config function from TRL creates a LoraConfig object based on the ModelConfig settings (lora_r, lora_alpha, lora_dropout, lora_target_modules). When use_peft: true is set in the recipe config, the trainer automatically wraps the model with LoRA adapters.
Usage
Use this environment when running any LoRA or QLoRA training workflow. Required by the Get_Peft_Config implementation and used implicitly by SFTTrainer and DPOTrainer when peft_config is provided.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| Python | >= 3.10.9 | Required by the alignment-handbook package |
Dependencies
Python Packages
- `peft` >= 0.16.0
- `trl` >= 0.19.1 (provides get_peft_config utility)
Credentials
No additional credentials required.
Quick Install
# Installed as part of alignment-handbook
uv pip install .
# Or install standalone
pip install peft>=0.16.0
Code Evidence
PEFT version requirement from `setup.py:60`:
"peft>=0.16.0",
PEFT config usage in `scripts/sft.py:48,111`:
from trl import ModelConfig, SFTTrainer, TrlParser, get_peft_config, setup_chat_format
peft_config=get_peft_config(model_args),
LoRA configuration from `recipes/zephyr-7b-beta/sft/config_qlora.yaml:8-20`:
use_peft: true
lora_r: 16
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `ImportError: No module named 'peft'` | PEFT not installed | `pip install peft>=0.16.0` |
| `ValueError: Target modules not found in model` | lora_target_modules mismatch | Update target modules to match model architecture (check model's named_modules) |
Compatibility Notes
- LoRA target modules: The alignment-handbook targets all linear projection layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj). This is model-architecture-dependent.
- LoRA rank: SFT uses r=16; DPO QLoRA uses r=128 (higher rank for preference learning).