Implementation:Princeton nlp SimPO H4ArgumentParser Parse

Knowledge Sources	SimPO HfArgumentParser
Domains	Configuration, Training_Infrastructure
Last Updated	2026-02-08 04:30 GMT

Overview

Concrete tool for parsing YAML configuration files with CLI overrides into typed training argument dataclasses, provided by the SimPO alignment package.

Description

H4ArgumentParser extends HuggingFace's HfArgumentParser to support a two-layer config system: YAML files provide base configuration, and command-line arguments override individual values. The parser dispatches arguments to three dataclasses: ModelArguments (model path, quantization, LoRA settings), DataArguments (dataset mixer, chat template, preprocessing), and SimPOConfig (extends TrainingArguments with SimPO loss parameters). The parse() method auto-detects whether input is YAML-only, YAML+CLI, or CLI-only.

Usage

Import and use at the beginning of any SimPO training script. This is always the first operational step after environment setup.

Code Reference

Source Location

• Repository: SimPO
• File: alignment/configs.py (Lines 33-105)
• File: scripts/simpo_config.py (Lines 6-71)

Signature

class H4ArgumentParser(HfArgumentParser):
    def parse_yaml_and_args(
        self,
        yaml_arg: str,
        other_args: Optional[List[str]] = None
    ) -> List[dataclass]:
        """
        Parse a YAML file and overwrite the default/loaded values
        with the values provided to the command line.

        Args:
            yaml_arg: The path to the config file used.
            other_args: A list of strings to parse as command line
                        arguments, e.g. ['--arg=val', '--arg2=val2'].

        Returns:
            A list of dataclasses with the values from the
            YAML file and the command line.
        """

    def parse(self) -> DataClassType | Tuple[DataClassType]:
        """
        Auto-dispatch parser:
        - Single YAML arg -> parse YAML only
        - YAML + CLI args -> parse YAML then override with CLI
        - CLI only -> standard argparse
        """

@dataclass
class ModelArguments:
    model_name_or_path: Optional[str] = None
    model_revision: str = "main"
    torch_dtype: Optional[str] = None
    trust_remote_code: bool = False
    attn_implementation: Optional[str] = None
    use_peft: bool = False
    lora_r: Optional[int] = 16
    lora_alpha: Optional[int] = 32
    lora_dropout: Optional[float] = 0.05
    lora_target_modules: Optional[List[str]] = None
    load_in_8bit: bool = False
    load_in_4bit: bool = False
    bnb_4bit_quant_type: Optional[str] = "nf4"
    use_bnb_nested_quant: bool = False

@dataclass
class DataArguments:
    chat_template: Optional[str] = None
    dataset_mixer: Optional[Dict[str, float]] = None
    dataset_splits: Optional[List[str]] = field(default_factory=lambda: ["train", "test"])
    truncation_side: Optional[str] = None
    auto_insert_empty_system_msg: bool = True
    preprocessing_num_workers: Optional[int] = None

@dataclass
class SimPOConfig(TrainingArguments):
    max_length: Optional[int] = None
    max_prompt_length: Optional[int] = None
    beta: float = 2.0
    gamma_beta_ratio: float = 0.25
    sft_weight: float = 0.0
    loss_type: Literal["sigmoid", "hinge"] = "sigmoid"
    label_smoothing: float = 0
    disable_dropout: bool = True

Import

from alignment import H4ArgumentParser, ModelArguments, DataArguments
from simpo_config import SimPOConfig

I/O Contract

Inputs

Name	Type	Required	Description
dataclass_types	Tuple[Type]	Yes	Tuple of dataclass types to parse into, e.g. (ModelArguments, DataArguments, SimPOConfig)
YAML file	str (via sys.argv)	Yes	Path to YAML configuration file
CLI overrides	List[str] (via sys.argv)	No	Command-line arguments in --key=value format

Outputs

Name	Type	Description
model_args	ModelArguments	Model selection, quantization, and LoRA parameters
data_args	DataArguments	Dataset mixer, chat template, preprocessing settings
training_args	SimPOConfig	Training hyperparameters including SimPO-specific loss config

Usage Examples

Standard Training Launch

from alignment import H4ArgumentParser, ModelArguments, DataArguments
from simpo_config import SimPOConfig

# Parse from YAML config file (the typical usage)
parser = H4ArgumentParser((ModelArguments, DataArguments, SimPOConfig))
model_args, data_args, training_args = parser.parse()

# Access parsed values
print(model_args.model_name_or_path)  # e.g., "meta-llama/Meta-Llama-3-8B-Instruct"
print(training_args.beta)             # e.g., 2.0
print(training_args.gamma_beta_ratio) # e.g., 0.25
print(data_args.dataset_mixer)        # e.g., {"princeton-nlp/llama3-ultrafeedback": 1.0}

CLI Launch With YAML + Overrides

# YAML provides base config, CLI overrides specific values
accelerate launch scripts/run_simpo.py \
    training_configs/llama-3-8b-instruct-simpo.yaml \
    --beta=2.5 \
    --learning_rate=5e-7

Example YAML Config

# training_configs/llama-3-8b-instruct-simpo.yaml
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
torch_dtype: bfloat16
attn_implementation: flash_attention_2

dataset_mixer:
  princeton-nlp/llama3-ultrafeedback: 1.0
dataset_splits:
  - train_prefs
  - test_prefs

beta: 2.0
gamma_beta_ratio: 0.5
loss_type: sigmoid
max_length: 2048
max_prompt_length: 1800
learning_rate: 5.0e-7
num_train_epochs: 1
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
output_dir: data/llama-3-8b-instruct-simpo

Related Pages

Implements Principle

• Principle:Princeton_nlp_SimPO_Configuration_Parsing