Implementation:Intel Ipex llm Get Peft Model LoRA

Knowledge Sources	IPEX-LLM
Domains	NLP, Parameter_Efficient_Finetuning
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tools for preparing a bf16 model and injecting LoRA adapters in standard LoRA mode, provided by IPEX-LLM.

Description

The same three IPEX-LLM functions as QLoRA (prepare_model_for_kbit_training, LoraConfig, get_peft_model) but configured with training_mode="lora". This tells IPEX-LLM to use standard gradient computation on bf16 weights rather than quantization-aware gradients. The prepare step still freezes base parameters and enables gradient computation on adapter layers.

Usage

Use after loading a model with load_in_low_bit="bf16" for standard LoRA training.

Code Reference

Source Location

Repository: IPEX-LLM
File: python/llm/example/GPU/LLM-Finetuning/LoRA/alpaca_lora_finetuning.py
Lines: 196-209

Signature

def prepare_model_for_kbit_training(
    model: PreTrainedModel,
    use_gradient_checkpointing: bool = False
) -> PreTrainedModel

class LoraConfig:
    def __init__(
        self,
        r: int = 8,
        lora_alpha: int = 16,
        target_modules: List[str] = None,
        lora_dropout: float = 0.05,
        bias: str = "none",
        task_type: str = "CAUSAL_LM",
        training_mode: str = "lora",  # Key difference from QLoRA
    )

def get_peft_model(model: PreTrainedModel, peft_config: LoraConfig) -> PeftModel

Import

from ipex_llm.transformers.qlora import (
    get_peft_model, prepare_model_for_kbit_training, LoraConfig
)

I/O Contract

Inputs

Name	Type	Required	Description
model	PreTrainedModel	Yes	bf16 base model from AutoModelForCausalLM.from_pretrained
r	int	No	LoRA rank (default 8)
lora_alpha	int	No	LoRA alpha scaling (default 16)
target_modules	List[str]	No	Target linear layers
training_mode	str	Yes	Must be "lora" for standard LoRA

Outputs

Name	Type	Description
model	PeftModel	Model with LoRA adapters, training_mode="lora"

Usage Examples

from ipex_llm.transformers.qlora import (
    get_peft_model, prepare_model_for_kbit_training, LoraConfig
)

# 1. Prepare bf16 model for training
model = prepare_model_for_kbit_training(model, use_gradient_checkpointing=False)

# 2. Configure LoRA with training_mode="lora"
config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj",
                     "up_proj", "down_proj", "gate_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    training_mode="lora",  # Standard LoRA, not QLoRA
)

# 3. Inject adapters
model = get_peft_model(model, config)
model.print_trainable_parameters()

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment