Implementation:Intel Ipex llm Save Pretrained DPO

Knowledge Sources	IPEX-LLM
Domains	NLP, Model_Deployment
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for saving DPO-trained model artifacts for deployment, used in the IPEX-LLM DPO workflow.

Description

After DPO training, dpo_trainer.model.save_pretrained() saves the LoRA adapter weights, and tokenizer.save_pretrained() saves the tokenizer. For full model export, the shared export_merged_model.py script calls merge_adapter to fold adapters into the base model.

Usage

Use immediately after dpo_trainer.train() to save model artifacts.

Code Reference

Source Location

• Repository: IPEX-LLM
• File: python/llm/example/GPU/LLM-Finetuning/DPO/dpo_finetuning.py
• Lines: 175-179
• File: python/llm/example/GPU/LLM-Finetuning/DPO/export_merged_model.py
• Lines: 28-44

Signature

# Save adapter
dpo_trainer.model.save_pretrained(output_path: str) -> None
tokenizer.save_pretrained(output_path: str) -> None

# Merge adapter (via export_merged_model.py)
# python export_merged_model.py --base_model MODEL --adapter_path ADAPTER --output_path OUTPUT

Import

# save_pretrained is a method on the model/tokenizer objects
# merge_adapter is from common.utils (see Merge_Adapter implementation)

I/O Contract

Inputs

Name	Type	Required	Description
output_path	str	Yes	Directory to save adapter weights and tokenizer

Outputs

Name	Type	Description
adapter files	Files	adapter_model.bin, adapter_config.json saved to output_path
tokenizer files	Files	tokenizer.json, tokenizer_config.json saved to output_path

Usage Examples

# After DPO training
dpo_trainer.train()

# Save adapter + tokenizer
dpo_trainer.model.save_pretrained("./dpo-output")
tokenizer.save_pretrained("./dpo-output")

# Optionally merge adapter for standalone deployment
# python export_merged_model.py \
#     --base_model teknium/OpenHermes-2.5-Mistral-7B \
#     --adapter_path ./dpo-output \
#     --output_path ./dpo-merged

Related Pages

Implements Principle

• Principle:Intel_Ipex_llm_DPO_Model_Export

Requires Environment

• Environment:Intel_Ipex_llm_XPU_Finetuning_Environment