Workflow:Huggingface Peft LoRA Causal LM Finetuning
| Knowledge Sources | |
|---|---|
| Domains | LLMs, Fine_Tuning, Parameter_Efficient |
| Last Updated | 2026-02-07 06:00 GMT |
Overview
End-to-end process for fine-tuning a causal language model using LoRA (Low-Rank Adaptation) or DoRA (Weight-Decomposed Low-Rank Adaptation) with optional 4-bit quantization via BitsAndBytes.
Description
This workflow demonstrates the most fundamental use case of the PEFT library: adapting a pre-trained causal language model to a downstream task by injecting trainable low-rank adapter matrices into the model's attention layers. The base model weights remain frozen while only the small adapter parameters are trained, reducing memory requirements by orders of magnitude. The workflow supports optional 4-bit quantization (QLoRA pattern) to further reduce GPU memory, and can use DoRA for improved training quality. The result is a lightweight adapter checkpoint that can be saved, shared, and loaded independently of the base model.
Usage
Execute this workflow when you have a text dataset and need to adapt a pre-trained causal language model (GPT-2, LLaMA, Mistral, OPT, etc.) for tasks such as instruction following, domain-specific text generation, or language modeling on specialized corpora. This is the recommended starting point for most PEFT users working with decoder-only transformer models.
Execution Steps
Step 1: Load Base Model and Tokenizer
Load a pre-trained causal language model and its corresponding tokenizer from the Hugging Face Hub. If memory is constrained, configure 4-bit quantization using BitsAndBytesConfig to load the model in NF4 (NormalFloat4) format with double quantization enabled. This reduces the model's memory footprint while preserving quality. Set the tokenizer's pad token to the EOS token for models that lack a dedicated pad token.
Key considerations:
- Choose the appropriate quantization type (NF4 recommended for most cases)
- Enable double quantization for additional memory savings
- Set compute dtype to bfloat16 or float16 for quantized models
- Call prepare_model_for_kbit_training if using quantization to handle gradient checkpointing and layer norm casting
Step 2: Configure LoRA Adapter
Define the LoRA adapter configuration specifying the rank, alpha scaling factor, dropout rate, and which model modules to target. Common targets include the attention projection layers (q_proj, v_proj, k_proj, o_proj) and optionally the MLP layers (gate_proj, up_proj, down_proj). For DoRA, enable the use_dora flag in the LoRA config. Set the task type to CAUSAL_LM.
Key considerations:
- Rank (r) controls the adapter capacity; typical values are 8, 16, 32, or 64
- Alpha (lora_alpha) scales the adapter output; commonly set equal to rank or double the rank
- Target modules should match the architecture of the specific base model
- DoRA decomposes weights into magnitude and direction components for improved training
Step 3: Apply Adapter to Model
Wrap the base model with the LoRA adapter using the get_peft_model factory function. This injects the adapter layers into the specified target modules while keeping all original parameters frozen. The resulting PeftModel exposes only the adapter parameters as trainable, dramatically reducing the number of parameters that require gradient computation.
What happens:
- Original weight matrix W remains frozen
- Two small matrices A and B are injected: output = W(x) + (alpha/r) * B(A(x))
- Only A and B matrices are trainable (typically less than 1% of total parameters)
- The model reports trainable vs total parameter counts for verification
Step 4: Prepare Dataset
Load the training dataset and tokenize it for causal language modeling. For instruction-tuning datasets, format examples into a prompt template (e.g., Alpaca format with instruction/input/output fields). Set the labels equal to the input IDs for next-token prediction. Apply padding and truncation to the maximum sequence length.
Key considerations:
- Use a consistent prompt template that matches the intended use case
- Set labels to input_ids for causal LM training (the model predicts the next token)
- Mask padding tokens in labels with -100 to exclude them from loss computation
- Split into train and validation sets for monitoring training progress
Step 5: Train the Adapter
Configure the training arguments (learning rate, batch size, number of epochs, gradient accumulation, mixed precision) and run training using the Hugging Face Trainer. The trainer handles the optimization loop, evaluation, checkpointing, and logging. Only the adapter parameters receive gradient updates during training.
Key considerations:
- Use a learning rate appropriate for adapter training (typically 1e-4 to 3e-4)
- Enable fp16 or bf16 mixed precision training for efficiency
- Use gradient accumulation to simulate larger batch sizes on limited hardware
- Monitor training loss and validation metrics to detect overfitting
Step 6: Save and Load Adapter
Save the trained adapter weights to disk. The saved checkpoint contains only the adapter parameters (typically a few megabytes), not the full base model. To use the adapter later, load the base model and then load the adapter on top using PeftModel.from_pretrained. The adapter can also be pushed to the Hugging Face Hub for sharing.
Key considerations:
- Only adapter weights are saved (not the full model), keeping checkpoints small
- The adapter can be loaded onto any compatible base model
- Use merge_and_unload to permanently fold adapter weights into the base model if needed
- Adapters can be pushed to the Hugging Face Hub for sharing and versioning