Implementation:Hiyouga LLaMA Factory Tuner

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Machine Learning, Training Infrastructure
Last Updated	2026-02-06 19:00 GMT

Overview

Top-level training dispatcher and model export orchestrator that routes all training commands to the appropriate stage runner and manages Ray-based distributed training infrastructure.

Description

tuner.py serves as the primary entry point for the entire LLaMA-Factory training framework. It contains three core pieces of functionality:

• run_exp: The main training entry point that parses arguments, optionally initializes Ray for distributed multi-node training, and delegates to the internal _training_function.
• _training_function: The internal dispatcher that assembles callbacks (LogCallback, PissaConvertCallback, SwanLab, EarlyStopping, ReporterCallback) and routes to the correct training stage runner based on finetuning_args.stage:
  • pt - Pre-training
  • sft - Supervised Fine-Tuning
  • rm - Reward Model training
  • ppo - Proximal Policy Optimization
  • dpo - Direct Preference Optimization
  • kto - KTO alignment training
  • MCA (mcore-adapter) variants for pt, sft, and dpo stages
• export_model: Handles model export with dtype conversion, adapter merging, safetensors serialization, tokenizer/processor saving, value-head copying for RM models, HuggingFace Hub pushing, and Ollama Modelfile generation.
• Worker class and _ray_training_function: Manage Ray-based multi-node distributed training with placement groups, environment variable propagation, and worker coordination.

Usage

Use run_exp as the entry point for any training job, whether invoked from CLI (llamafactory-cli train), the WebUI, or programmatically. Use export_model to convert trained models for deployment. The Ray infrastructure is activated when ray_args.use_ray is True, enabling multi-node distributed training.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/train/tuner.py
• Lines: 1-304

Signature

def run_exp(args: Optional[dict[str, Any]] = None, callbacks: Optional[list[TrainerCallback]] = None) -> None

def export_model(args: Optional[dict[str, Any]] = None) -> None

class Worker:
    def __init__(self)
    def _setup_env_visible_devices(self) -> None
    def _training_function(self, config: dict[str, Any]) -> None

Import

from llamafactory.train.tuner import run_exp, export_model

I/O Contract

Inputs

run_exp

Name	Type	Required	Description
args	Optional[dict[str, Any]]	No	Dictionary of training arguments; if None, reads from sys.argv via read_args
callbacks	Optional[list[TrainerCallback]]	No	Additional trainer callbacks to register; LogCallback and ReporterCallback are always added internally

export_model

Name	Type	Required	Description
args	Optional[dict[str, Any]]	No	Dictionary of inference/export arguments; must include export_dir to specify save location

Outputs

run_exp

Name	Type	Description
(none)	None	Executes training as a side effect; model checkpoints are saved to the configured output_dir

export_model

Name	Type	Description
(none)	None	Saves the exported model, tokenizer, and Ollama Modelfile to export_dir; optionally pushes to HuggingFace Hub

Usage Examples

# Run training from Python
from llamafactory.train.tuner import run_exp

run_exp(args={
    "stage": "sft",
    "model_name_or_path": "meta-llama/Llama-2-7b-hf",
    "dataset": "alpaca_en",
    "output_dir": "./output/sft",
    "finetuning_type": "lora",
    "do_train": True,
})

# Export a trained model
from llamafactory.train.tuner import export_model

export_model(args={
    "model_name_or_path": "meta-llama/Llama-2-7b-hf",
    "adapter_name_or_path": "./output/sft/checkpoint-1000",
    "export_dir": "./export/merged_model",
    "finetuning_type": "lora",
    "template": "llama2",
})

Related Pages

• Hiyouga_LLaMA_Factory_Trainer_Utils - Shared utilities invoked by the training stages dispatched from tuner
• Hiyouga_LLaMA_Factory_WebUI_Runner - WebUI process manager that invokes run_exp via subprocess