Implementation:Hiyouga LLaMA Factory Model Patcher

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Model Loading, Configuration Management
Last Updated	2026-02-06 19:00 GMT

Overview

Central patching layer that applies monkey-patches and configuration fixes to models, tokenizers, and processors for cross-model compatibility.

Description

The patcher module is the main orchestration point for all pre-loading and post-loading model modifications. patch_config handles pre-loading configuration by invoking compute dtype inference, attention implementation setup, RoPE scaling, LongLoRA, quantization, MoE configuration, visual model setup, packing, and KV cache configuration. It also handles model-type-specific settings for Qwen, MiniCPM-O, Kimi-VL, and validates architecture compatibility. patch_model applies post-loading fixes including generation config correction, vocabulary resizing, gradient checkpointing, visual model projector dtype casting, and Z3 leaf module registration. patch_tokenizer fixes padding behavior and max length settings. patch_processor sets multimodal processing attributes. patch_valuehead_model adds tie_weights, embedding accessors, rope index functions, and model card methods to TRL value head wrappers. Additional patches target Qwen3 OmniMoE sparse MoE blocks and YouTu VL model forward methods.

Usage

These functions are called internally during the model loading pipeline. patch_config is invoked before model instantiation, patch_tokenizer and patch_processor after tokenizer/processor loading, and patch_model after the model is loaded. patch_valuehead_model is used when a value head is attached for RLHF training.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/model/patcher.py
• Lines: 1-282

Signature

def patch_qwen3_omni_moe_thinker_text_sparse_moe_block() -> None

def patch_youtu_vl_model(model: "PreTrainedModel") -> None

def patch_tokenizer(tokenizer: "PreTrainedTokenizer", model_args: "ModelArguments") -> None

def patch_processor(
    processor: "ProcessorMixin",
    tokenizer: "PreTrainedTokenizer",
    model_args: "ModelArguments",
) -> None

def patch_config(
    config: "PretrainedConfig",
    tokenizer: "PreTrainedTokenizer",
    model_args: "ModelArguments",
    init_kwargs: dict[str, Any],
    is_trainable: bool,
) -> None

def patch_model(
    model: "PreTrainedModel",
    tokenizer: "PreTrainedTokenizer",
    model_args: "ModelArguments",
    is_trainable: bool,
    add_valuehead: bool,
) -> None

def patch_valuehead_model(model: "AutoModelForCausalLMWithValueHead") -> None

Import

from llamafactory.model.patcher import patch_config, patch_model, patch_tokenizer, patch_processor, patch_valuehead_model

I/O Contract

Inputs

Name	Type	Required	Description
config	PretrainedConfig	Yes	Model configuration object to be modified before model instantiation
tokenizer	PreTrainedTokenizer	Yes	Tokenizer to be patched for padding and max length; also used in config patching
model_args	ModelArguments	Yes	Comprehensive model arguments controlling all patch behaviors
init_kwargs	dict[str, Any]	Yes	Mutable dict for model init kwargs; modified by patch_config
is_trainable	bool	Yes	Whether the model is being loaded for training
add_valuehead	bool	Conditional	Whether to prepare the model for value head attachment (in patch_model)
model	PreTrainedModel	Conditional	The loaded model to be patched (in patch_model)
processor	ProcessorMixin	Conditional	The processor to be patched (in patch_processor)

Outputs

Name	Type	Description
None	None	All functions modify their arguments in-place; no return values

Usage Examples

# Typical model loading pipeline using patchers
from llamafactory.model.patcher import patch_config, patch_model, patch_tokenizer

init_kwargs = {}
patch_tokenizer(tokenizer, model_args)
patch_config(config, tokenizer, model_args, init_kwargs, is_trainable=True)
# ... load model with init_kwargs ...
patch_model(model, tokenizer, model_args, is_trainable=True, add_valuehead=False)

Related Pages

• Hiyouga_LLaMA_Factory_Quantization - Called by patch_config for quantization setup
• Hiyouga_LLaMA_Factory_RoPE_Config - Called by patch_config for RoPE scaling
• Hiyouga_LLaMA_Factory_Value_Head - prepare_valuehead_model called by patch_model
• Hiyouga_LLaMA_Factory_Unsloth_Integration - Unsloth flag checked during patch_model