Implementation:Intel Ipex llm Chatml Format

Knowledge Sources	IPEX-LLM
Domains	NLP, RLHF, Data_Processing
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete function for converting preference pair examples into ChatML format for DPO training, provided in the IPEX-LLM DPO example.

Description

The chatml_format function takes a dataset example with system, question, chosen, and rejected fields, and converts them into the ChatML template format using the tokenizer's apply_chat_template method. It returns a dict with prompt, chosen, and rejected keys suitable for DPOTrainer.

Usage

Use as a map function over HuggingFace datasets to format preference pairs for DPO training.

Code Reference

Source Location

• Repository: IPEX-LLM
• File: python/llm/example/GPU/LLM-Finetuning/DPO/dpo_finetuning.py
• Lines: 47-69

Signature

def chatml_format(example: Dict) -> Dict:
    """Convert a preference pair example to ChatML format.

    Args:
        example: Dict with keys 'system', 'question', 'chosen', 'rejected'

    Returns:
        Dict with keys 'prompt', 'chosen', 'rejected' in ChatML format
    """

Import

# Defined in dpo_finetuning.py (not a library import)
# The function depends on the global tokenizer variable

I/O Contract

Inputs

Name	Type	Required	Description
example	Dict	Yes	Dataset row with 'system', 'question', 'chosen', 'rejected' keys

Outputs

Name	Type	Description
result	Dict	Contains 'prompt' (ChatML formatted), 'chosen' (with im_end), 'rejected' (with im_end)

Usage Examples

from datasets import load_dataset
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("teknium/OpenHermes-2.5-Mistral-7B")
tokenizer.pad_token = tokenizer.eos_token

def chatml_format(example):
    if len(example['system']) > 0:
        message = {"role": "system", "content": example['system']}
        system = tokenizer.apply_chat_template([message], tokenize=False)
    else:
        system = ""
    message = {"role": "user", "content": example['question']}
    prompt = tokenizer.apply_chat_template([message], tokenize=False,
                                            add_generation_prompt=True)
    chosen = example['chosen'] + "<|im_end|>\n"
    rejected = example['rejected'] + "<|im_end|>\n"
    return {"prompt": system + prompt, "chosen": chosen, "rejected": rejected}

# Format dataset
dataset = load_dataset("Intel/orca_dpo_pairs")['train']
original_columns = dataset.column_names
dataset = dataset.map(chatml_format, remove_columns=original_columns)

Related Pages

Implements Principle

• Principle:Intel_Ipex_llm_DPO_Data_Preparation

Requires Environment

• Environment:Intel_Ipex_llm_XPU_Finetuning_Environment