Implementation:Hiyouga LLaMA Factory Generating Args

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	Configuration, Text Generation
Last Updated	2026-02-06 19:00 GMT

Overview

Dataclass defining text generation and decoding parameters for inference, evaluation, and interactive chat workflows.

Description

The GeneratingArguments dataclass specifies all decoding-related hyperparameters including sampling settings (do_sample, temperature, top_p, top_k), beam search parameters (num_beams), output length constraints (max_length, max_new_tokens), penalty parameters (repetition_penalty, length_penalty), and special token handling (skip_special_tokens). The to_dict method serializes the arguments to a dictionary with intelligent conflict resolution: when max_new_tokens is positive, max_length is removed (and vice versa). When obey_generation_config is True, the method filters to only include keys recognized by HuggingFace's GenerationConfig, ensuring compatibility with the transformers generation pipeline.

Usage

Use this dataclass to configure generation behavior during evaluation predictions, interactive chat sessions, and API inference. It is typically parsed alongside other argument classes via HfArgumentParser and passed to the model's generate method or to inference engines.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: src/llamafactory/hparams/generating_args.py
• Lines: 1-83

Signature

@dataclass
class GeneratingArguments:
    do_sample: bool = True
    temperature: float = 0.95
    top_p: float = 0.7
    top_k: int = 50
    num_beams: int = 1
    max_length: int = 1024
    max_new_tokens: int = 1024
    repetition_penalty: float = 1.0
    length_penalty: float = 1.0
    skip_special_tokens: bool = True

    def to_dict(self, obey_generation_config: bool = False) -> dict[str, Any]

Import

from llamafactory.hparams.generating_args import GeneratingArguments

I/O Contract

Inputs

Name	Type	Required	Description
do_sample	bool	No	Whether to use sampling; if False, uses greedy decoding (default: True)
temperature	float	No	Sampling temperature to modulate next-token probabilities (default: 0.95)
top_p	float	No	Nucleus sampling threshold (default: 0.7)
top_k	int	No	Top-k filtering threshold (default: 50)
num_beams	int	No	Number of beams for beam search; 1 means no beam search (default: 1)
max_length	int	No	Maximum total sequence length including prompt (default: 1024)
max_new_tokens	int	No	Maximum number of new tokens to generate (default: 1024); takes precedence over max_length when positive
repetition_penalty	float	No	Penalty for token repetition; 1.0 means no penalty (default: 1.0)
length_penalty	float	No	Exponential penalty for beam search length (default: 1.0)
skip_special_tokens	bool	No	Whether to strip special tokens from decoded output (default: True)

Outputs

Name	Type	Description
to_dict()	dict[str, Any]	Dictionary of generation parameters with max_length/max_new_tokens conflict resolved and optionally filtered to valid GenerationConfig keys

Usage Examples

from llamafactory.hparams.generating_args import GeneratingArguments

# Create with custom parameters
gen_args = GeneratingArguments(
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    max_new_tokens=512,
    repetition_penalty=1.1,
)

# Convert to dict for model.generate()
gen_kwargs = gen_args.to_dict()
# gen_kwargs will NOT contain "max_length" since max_new_tokens > 0

# Convert with GenerationConfig compatibility filtering
gen_kwargs_safe = gen_args.to_dict(obey_generation_config=True)
# gen_kwargs_safe only contains keys recognized by transformers GenerationConfig

# Use with HfArgumentParser
from transformers import HfArgumentParser
from llamafactory.hparams.generating_args import GeneratingArguments

parser = HfArgumentParser(GeneratingArguments)
gen_args = parser.parse_args_into_dataclasses()[0]

Related Pages

• Hiyouga_LLaMA_Factory_Data_Args - Companion argument class for dataset and data processing configuration
• Hiyouga_LLaMA_Factory_Evaluator - Uses generation parameters during benchmark evaluation inference