Implementation:Huggingface Alignment handbook Trainer Save And Push

Knowledge Sources	Alignment Handbook Transformers Trainer HuggingFace Hub
Domains	NLP, Training, MLOps
Last Updated	2026-02-07 00:00 GMT

Overview

Concrete tool for saving trained models locally and publishing to HuggingFace Hub, using Transformers Trainer methods as configured by the alignment-handbook scripts.

Description

The alignment-handbook uses Trainer.save_model, Trainer.create_model_card, and Trainer.push_to_hub from HuggingFace Transformers for model persistence. The SFT script (scripts/sft.py) has the most complete implementation including EOS alignment and model card creation. DPO and ORPO scripts use a simpler save-and-push flow.

Usage

These methods are called at the end of every training script. The save/push behavior is controlled by training_args.output_dir and training_args.push_to_hub.

Code Reference

Source Location

Repository: alignment-handbook
File: scripts/sft.py (lines 132-168), scripts/dpo.py (lines 150-153), scripts/orpo.py (lines 150-152)

Signature

# From Transformers Trainer (used by all alignment-handbook scripts)
class Trainer:
    def save_model(self, output_dir: Optional[str] = None):
        """Save the model to output_dir. For PEFT models, saves only adapter weights."""

    def create_model_card(self, **kwargs):
        """Create a model card with training metadata.
        kwargs: model_name, dataset_name, tags, etc.
        """

    def push_to_hub(self, **kwargs):
        """Push model, tokenizer, and model card to HuggingFace Hub.
        kwargs: dataset_name, etc.
        """

Import

# No separate import needed; these are methods on the trainer instance
# from trl import SFTTrainer  (or DPOTrainer, ORPOTrainer)

I/O Contract

Inputs

Name	Type	Required	Description
training_args.output_dir	str	Yes	Local directory path for saving model files
training_args.push_to_hub	bool	No	Whether to push to HuggingFace Hub (default: False)
training_args.hub_model_id	str	No	HuggingFace Hub model ID for publishing
kwargs.tags	list[str]	No	Model card tags (default: ["alignment-handbook"])
kwargs.dataset_name	str	No	Dataset name for model card metadata

Outputs

Name	Type	Description
model files	Files	Model weights (or LoRA adapter weights) saved to output_dir
model_card	File	Auto-generated README.md model card
config	File	Model config with updated EOS token and use_cache=True
hub_url	str	HuggingFace Hub URL (if push_to_hub=True)

Usage Examples

SFT Save Flow (Most Complete)

# From scripts/sft.py:L132-168

# 1. Align EOS token
trainer.model.generation_config.eos_token_id = tokenizer.eos_token_id
trainer.model.config.eos_token_id = tokenizer.eos_token_id

# 2. Save model
trainer.save_model(training_args.output_dir)

# 3. Create model card (main process only)
kwargs = {
    "model_name": training_args.hub_model_id if training_args.push_to_hub else None,
    "dataset_name": script_args.dataset_name,
    "tags": ["alignment-handbook"],
}
if trainer.accelerator.is_main_process:
    trainer.create_model_card(**kwargs)
    trainer.model.config.use_cache = True
    trainer.model.config.save_pretrained(training_args.output_dir)

# 4. Push to hub
if training_args.push_to_hub:
    trainer.push_to_hub(**kwargs)

DPO/ORPO Save Flow (Simpler)

# From scripts/dpo.py:L150-153
trainer.save_model(training_args.output_dir)
if training_args.push_to_hub:
    trainer.push_to_hub(dataset_name=script_args.dataset_name)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment