Implementation:Hiyouga LLaMA Factory Model Args

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Machine Learning, LLM Configuration
Last Updated	2026-02-06 19:00 GMT

Overview

Comprehensive dataclass hierarchy defining all model, quantization, processor, export, and inference engine arguments for LLaMA-Factory.

Description

ModelArguments is a composite Python dataclass in the LLaMA-Factory framework that consolidates the complete model configuration surface area. It inherits from seven specialized argument classes: BaseModelArguments (model path, tokenizer, attention, RoPE, checkpointing), QuantizationArguments (BNB/HQQ/EETQ quantization settings), ProcessorArguments (image/video/audio processing limits), ExportArguments (model export and quantization), VllmArguments, SGLangArguments, and KTransformersArguments (inference engine configurations). Built using Python dataclasses with HuggingFace argument parsing, it controls how models are loaded, quantized, processed, and served across all training and inference modes.

Usage

Import ModelArguments when setting up any training or inference pipeline. It is parsed by the argument parser (get_train_args, get_infer_args, get_eval_args) and passed to model loaders, adapter initializers, and inference engines.

Code Reference

Source Location

Repository: Hiyouga_LLaMA_Factory
File: src/llamafactory/hparams/model_args.py
Lines: 1-568

Signature

@dataclass
class BaseModelArguments:
    model_name_or_path: str | None = None
    adapter_name_or_path: str | None = None
    cache_dir: str | None = None
    use_fast_tokenizer: bool = True
    resize_vocab: bool = False
    flash_attn: AttentionFunction = AttentionFunction.AUTO
    shift_attn: bool = False
    use_unsloth: bool = False
    infer_backend: EngineName = EngineName.HF
    trust_remote_code: bool = False
    def __post_init__(self): ...

@dataclass
class QuantizationArguments:
    quantization_method: QuantizationMethod = QuantizationMethod.BNB
    quantization_bit: int | None = None
    quantization_type: Literal["fp4", "nf4"] = "nf4"
    double_quantization: bool = True

@dataclass
class ProcessorArguments:
    image_max_pixels: int = 768 * 768
    video_fps: float = 2.0
    video_maxlen: int = 128

@dataclass
class ModelArguments(
    SGLangArguments, VllmArguments, KTransformersArguments,
    ExportArguments, ProcessorArguments, QuantizationArguments,
    BaseModelArguments,
):
    compute_dtype: torch.dtype | None = None  # derived
    device_map: str | dict | None = None  # derived
    model_max_length: int | None = None  # derived
    block_diag_attn: bool = False  # derived

    @classmethod
    def copyfrom(cls, source: Self, **kwargs) -> Self: ...
    def to_dict(self) -> dict[str, Any]: ...

Import

from llamafactory.hparams import ModelArguments

I/O Contract

Inputs

Name	Type	Required	Description
model_name_or_path	str	Yes	Path to model weight or HuggingFace/ModelScope identifier
adapter_name_or_path	str or None	No	Comma-separated adapter paths for LoRA merging
quantization_bit	int or None	No	Number of bits for on-the-fly quantization (4 or 8)
flash_attn	AttentionFunction	No (default: AUTO)	Attention implementation: auto, fa2, sdpa, eager
shift_attn	bool	No (default: False)	Enable S2-Attn from LongLoRA
infer_backend	EngineName	No (default: HF)	Backend engine: hf, vllm, sglang, kt
image_max_pixels	int	No (default: 589824)	Maximum pixel count for image inputs
export_dir	str or None	No	Directory for exported model files
vllm_maxlen	int	No (default: 4096)	Maximum sequence length for vLLM engine

Outputs

Name	Type	Description
ModelArguments instance	ModelArguments	Validated configuration with all model parameters
compute_dtype	torch.dtype	Derived compute dtype (bf16/fp16) set during argument post-processing
device_map	str or dict	Derived device map set during training/inference setup
to_dict()	dict[str, Any]	Serialized dictionary with auth tokens masked

Usage Examples

from llamafactory.hparams import ModelArguments

# Basic model configuration
model_args = ModelArguments(
    model_name_or_path="meta-llama/Llama-2-7b-hf",
    flash_attn="fa2",
    quantization_bit=4,
    quantization_type="nf4",
)

# Clone with overrides for a reference model
ref_model_args = ModelArguments.copyfrom(
    model_args,
    adapter_name_or_path=None,
    quantization_bit=None,
)

# Multimodal model with processor settings
vlm_args = ModelArguments(
    model_name_or_path="llava-hf/llava-1.5-7b-hf",
    image_max_pixels=1024 * 1024,
    video_fps=1.0,
)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment