Implementation:Hiyouga LLaMA Factory Hparams Parser
| Knowledge Sources | |
|---|---|
| Domains | Machine Learning, Configuration Management |
| Last Updated | 2026-02-06 19:00 GMT |
Overview
Central argument parsing, validation, and post-processing module for all LLaMA-Factory training, inference, and evaluation workflows.
Description
The parser module is the single entry point for argument processing in LLaMA-Factory. It uses HuggingFace's HfArgumentParser to parse arguments from YAML files, JSON files, or CLI flags, then performs extensive cross-argument validation (checking compatibility of DeepSpeed, quantization, PPO requirements, distributed training constraints), verifies extra dependencies, configures device maps and compute dtypes, handles checkpoint resumption logic, and sets environment variables. It supports standard training via get_train_args, Megatron Core Adapter (MCA) training, inference via get_infer_args, and evaluation via get_eval_args.
Usage
Call get_train_args(), get_infer_args(), or get_eval_args() at the beginning of any workflow. These functions return validated tuples of argument dataclass instances (ModelArguments, DataArguments, TrainingArguments, FinetuningArguments, etc.) ready for use by downstream components.
Code Reference
Source Location
- Repository: Hiyouga_LLaMA_Factory
- File: src/llamafactory/hparams/parser.py
- Lines: 1-523
Signature
def read_args(
args: dict[str, Any] | list[str] | None = None
) -> dict[str, Any] | list[str]:
"""Get arguments from the command line or a config file."""
def get_train_args(
args: dict[str, Any] | list[str] | None = None
) -> tuple[ModelArguments, DataArguments, TrainingArguments, FinetuningArguments, GeneratingArguments]:
"""Parse and validate training arguments."""
def get_infer_args(
args: dict[str, Any] | list[str] | None = None
) -> tuple[ModelArguments, DataArguments, FinetuningArguments, GeneratingArguments]:
"""Parse and validate inference arguments."""
def get_eval_args(
args: dict[str, Any] | list[str] | None = None
) -> tuple[ModelArguments, DataArguments, EvaluationArguments, FinetuningArguments]:
"""Parse and validate evaluation arguments."""
def get_ray_args(
args: dict[str, Any] | list[str] | None = None
) -> RayArguments:
"""Parse Ray-specific arguments."""
Import
from llamafactory.hparams.parser import get_train_args, get_infer_args, get_eval_args
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| args | dict or list[str] or None | No | Arguments as dict (from YAML/JSON), list of CLI strings, or None to read from sys.argv |
Outputs
| Name | Type | Description |
|---|---|---|
| get_train_args return | tuple of 5 dataclasses | (ModelArguments, DataArguments, TrainingArguments, FinetuningArguments, GeneratingArguments) |
| get_infer_args return | tuple of 4 dataclasses | (ModelArguments, DataArguments, FinetuningArguments, GeneratingArguments) |
| get_eval_args return | tuple of 4 dataclasses | (ModelArguments, DataArguments, EvaluationArguments, FinetuningArguments) |
Usage Examples
from llamafactory.hparams.parser import get_train_args
# Parse from a YAML config dict
model_args, data_args, training_args, finetuning_args, generating_args = get_train_args({
"model_name_or_path": "meta-llama/Llama-2-7b-hf",
"dataset": "alpaca_en",
"stage": "sft",
"finetuning_type": "lora",
"output_dir": "./output",
"per_device_train_batch_size": 4,
"do_train": True,
})
# Parse from CLI (reads sys.argv automatically)
# python train.py config.yaml
model_args, data_args, training_args, finetuning_args, generating_args = get_train_args()
# Parse for inference
from llamafactory.hparams.parser import get_infer_args
model_args, data_args, finetuning_args, generating_args = get_infer_args({
"model_name_or_path": "meta-llama/Llama-2-7b-hf",
"adapter_name_or_path": "./lora_output",
"finetuning_type": "lora",
})