Implementation:Hiyouga LLaMA Factory Liger Kernel

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Kernel Optimization, Training Performance
Last Updated	2026-02-06 19:00 GMT

Overview

Applies Liger Kernel Triton-based fused operator optimizations to supported model architectures, reducing GPU memory usage and improving training throughput.

Description

This module provides a single function apply_liger_kernel that maps model architecture types to their corresponding Liger Kernel patch functions from the liger_kernel.transformers package. It supports over 20 model architectures including Llama, Gemma (1/2/3), Mistral, Mixtral, Qwen2, Qwen2-VL, Qwen2.5-VL, Qwen3, Qwen3-MoE, Phi3, Granite, GLM4, OLMo2, LLaVA, MLlama, PaliGemma, and GPT-OSS. The function intelligently handles the fused_linear_cross_entropy optimization: for training stages that require logits (such as DPO, KTO, reward modeling), it disables the fused linear cross entropy kernel and falls back to standard cross entropy to ensure correctness. It inspects each patch function's signature to only pass supported keyword arguments.

Usage

Use this module by setting enable_liger_kernel: true in the model arguments. It is called automatically during model loading when training mode is active. The Liger Kernel optimizations are only applied during training (not inference) and require the liger-kernel package to be installed.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/model/model_utils/liger_kernel.py
• Lines: 1-97

Signature

def apply_liger_kernel(
    config: "PretrainedConfig",
    model_args: "ModelArguments",
    is_trainable: bool,
    require_logits: bool,
) -> None:
    ...

Import

from llamafactory.model.model_utils.liger_kernel import apply_liger_kernel

I/O Contract

Inputs

Name	Type	Required	Description
config	PretrainedConfig	Yes	Model configuration used to determine the model_type for architecture-specific kernel selection
model_args	ModelArguments	Yes	Model arguments; must have enable_liger_kernel set to True for the function to proceed
is_trainable	bool	Yes	Whether the model is being loaded for training; Liger Kernel is only applied during training
require_logits	bool	Yes	Whether the training stage requires logits (e.g., DPO/KTO); disables fused_linear_cross_entropy when True

Outputs

Name	Type	Description
(side effect)	None	Monkey-patches the model architecture's modules with Liger Kernel's fused Triton implementations

Usage Examples

from llamafactory.model.model_utils.liger_kernel import apply_liger_kernel

# Apply Liger Kernel for SFT training (fused cross entropy enabled)
apply_liger_kernel(
    config=model_config,
    model_args=model_args,
    is_trainable=True,
    require_logits=False,  # SFT/PT stage
)

# Apply Liger Kernel for DPO training (fused cross entropy disabled)
apply_liger_kernel(
    config=model_config,
    model_args=model_args,
    is_trainable=True,
    require_logits=True,  # DPO/KTO/RM stage
)

# No-op for inference
apply_liger_kernel(
    config=model_config,
    model_args=model_args,
    is_trainable=False,  # skips application
    require_logits=False,
)

Related Pages

• Hiyouga_LLaMA_Factory_Model_Loader - Calls apply_liger_kernel during model loading pipeline
• Hiyouga_LLaMA_Factory_Attention_Config - Attention configuration applied alongside kernel optimizations
• Hiyouga_LLaMA_Factory_Gradient_Checkpointing - Another memory optimization technique complementary to Liger Kernel