Environment:Huggingface Trl PEFT LoRA Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Parameter_Efficient_Finetuning |
| Last Updated | 2026-02-06 17:00 GMT |
Overview
Optional PEFT/LoRA environment requiring peft >= 0.8.0 for parameter-efficient fine-tuning across all TRL trainers.
Description
This environment provides the PEFT (Parameter-Efficient Fine-Tuning) library, which enables LoRA, QLoRA, and other adapter-based fine-tuning methods. When PEFT is installed, TRL trainers can accept a peft_config parameter to automatically wrap the base model with adapters. PEFT also enables memory-efficient reference model handling in DPO and GRPO by disabling the adapter instead of loading a full copy of the model.
Usage
Use this environment when you want to fine-tune large models with limited GPU memory. Required whenever passing peft_config to any trainer, or when using the get_peft_config utility from TRL's script utilities.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, macOS, Windows | Cross-platform support via PyTorch |
| Hardware | GPU recommended | LoRA reduces memory; QLoRA requires GPU with bitsandbytes |
| Python | >= 3.10 | Must match TRL core requirements |
Dependencies
Python Packages
- `peft` >= 0.8.0
- `transformers` >= 4.56.2 (from core)
Credentials
No additional credentials required.
Quick Install
# Install TRL with PEFT support
pip install "trl[peft]"
# Or install PEFT separately
pip install "peft>=0.8.0"
Code Evidence
PEFT availability check used throughout the codebase (e.g., `trl/trainer/grpo_trainer.py:97-98`):
if is_peft_available():
from peft import PeftConfig, PeftModel, get_peft_model
PEFT config application in GRPOTrainer (`trl/trainer/grpo_trainer.py:329-331`):
# Create PEFT model
if peft_config is not None:
model = get_peft_model(model, peft_config)
Gradient checkpointing fix for PEFT (`trl/trainer/grpo_trainer.py:333-336`):
# When using gradient checkpointing with PEFT, we need to enable input gradients.
# transformers.Trainer normally handles this, but a bug currently prevents it
if is_peft_available() and is_peft_model(model) and args.gradient_checkpointing:
model.enable_input_require_grads()
Optional dependency definition from `pyproject.toml:63-65`:
peft = [
"peft>=0.8.0"
]
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
ValueError: You passed a PeftModel instance together with a peft_config |
Passing both a pre-wrapped PEFT model and a new config | Merge and unload the existing adapter first, then pass the base model with the new config |
RuntimeError: element 0 does not require grad with gradient checkpointing |
PEFT + gradient checkpointing requires input gradients | TRL handles this automatically; ensure you use TRL's trainer, not manual setup |
HybridCache import error |
peft < 0.18.0 with transformers >= 5.0.0 | Upgrade peft to >= 0.18.0 or TRL will auto-patch |
Compatibility Notes
- PEFT < 0.18.0 + transformers >= 5.0.0: HybridCache import error. TRL auto-patches this via
_patch_transformers_hybrid_cache. - PEFT + GRPO reference model: When using PEFT with GRPO (beta != 0), TRL creates a "ref" adapter as a copy of the "default" adapter instead of loading a separate reference model.
- QLoRA: Requires both PEFT and bitsandbytes. Adapter weights are automatically cast to bf16.