Implementation:Intel Ipex llm QA LoRA Finetuning

Knowledge Sources	Intel IPEX-LLM
Domains	Finetuning, QA_LoRA, Quantization
Last Updated	2026-02-09 04:00 GMT

Overview

Concrete tool for Quantization-Aware LoRA fine-tuning with 4-bit quantization and distributed training provided by IPEX-LLM.

Description

The train() function implements QA-LoRA (Quantization-Aware LoRA), which combines 4-bit NF4 quantization with LoRA adapter injection for memory-efficient fine-tuning. It uses IPEX-LLM's custom get_peft_model and prepare_model_for_kbit_training from the qlora module, targeting all linear layers by default (q_proj, v_proj, k_proj, o_proj, up_proj, down_proj, gate_proj). Supports distributed training with gradient accumulation and WandB logging.

Usage

Use this for fine-tuning quantized models with LoRA adapters on Intel XPU, particularly when targeting all linear layers for maximum adaptation capability while maintaining 4-bit memory efficiency.

Code Reference

Source Location

Repository: Intel IPEX-LLM
File: python/llm/example/GPU/LLM-Finetuning/QA-LoRA/alpaca_qalora_finetuning.py
Lines: 1-279

Signature

def train(
    base_model: str = "meta-llama/Llama-2-7b-hf",
    saved_low_bit_model: str = None,
    data_path: str = "yahma/alpaca-cleaned",
    output_dir: str = "./bigdl-qlora-alpaca",
    batch_size: int = 128,
    micro_batch_size: int = 2,
    num_epochs: int = 3,
    learning_rate: float = 3e-4,
    lora_r: int = 8,
    lora_alpha: int = 16,
    lora_dropout: float = 0.05,
    lora_target_modules: List[str] = [
        "q_proj", "v_proj", "k_proj", "o_proj",
        "up_proj", "down_proj", "gate_proj"
    ],
    gradient_checkpointing: bool = False,
    deepspeed: str = None,
):
    """QA-LoRA fine-tuning with 4-bit quantization."""

Import

from ipex_llm.transformers import AutoModelForCausalLM
from ipex_llm.transformers.qlora import get_peft_model, prepare_model_for_kbit_training, LoraConfig

I/O Contract

Inputs

Name	Type	Required	Description
base_model	str	Yes	HuggingFace model ID or local path
data_path	str	No	Dataset path (default: yahma/alpaca-cleaned)
saved_low_bit_model	str	No	Pre-quantized model path for faster loading
lora_r	int	No	LoRA rank (default: 8)
lora_target_modules	List[str]	No	Target layers (default: all linear)

Outputs

Name	Type	Description
LoRA adapter weights	Files	Saved to output_dir
Training metrics	Console/WandB	Loss and training progress

Usage Examples

QA-LoRA Fine-tuning

python alpaca_qalora_finetuning.py \
    --base_model "meta-llama/Llama-2-7b-hf" \
    --data_path "yahma/alpaca-cleaned" \
    --output_dir "./qalora-output" \
    --bf16 True

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment