Principle:Huggingface Open r1 Chat Template Formatting

Metadata

Field	Value
Sources	Doc: HuggingFace Chat Templates
Domains	NLP, Data_Preprocessing
Last Updated	2026-02-08 00:00 GMT

Overview

A data preprocessing technique that converts raw text prompts into structured chat message format with system/user roles to match the model's expected input schema for instruction-following tasks.

Description

Modern language models trained with RLHF or instruction tuning expect inputs in a structured chat format (e.g., ChatML with role/content pairs). Raw dataset prompts must be converted into this format before training. The Chat_Template_Formatting principle covers:

1. Constructing message lists — building a list of chat messages with an optional system prompt and a user message, following the role/content pair convention.
2. Handling different prompt column names — supporting heterogeneous datasets where the prompt text may live under different column names (e.g., problem, question, prompt).
3. Applying the tokenizer's chat template — using apply_chat_template to convert message lists into model-ready input strings that match the expected token format.
4. Removing original message columns — dropping the original messages column (if present) to avoid conflicts with trainer expectations.

For GRPO training, the prompts remain as message lists (the trainer applies the template internally), while for pass-rate filtering, apply_chat_template is called explicitly to prepare inputs for vLLM generation.

Usage

Use this principle when preparing datasets for GRPO training or pass-rate filtering where prompts need to be in chat message format. It is applicable to:

• Any training pipeline that expects structured chat messages rather than raw text strings.
• Scenarios where datasets have varying column names for the prompt field.
• Preprocessing steps that must produce tokenizer-compatible chat format strings for inference engines such as vLLM.

Theoretical Basis

The core transformation converts a flat text prompt into a structured list of role/content dictionaries. The optional system prompt provides task-level instructions, while the user message contains the actual prompt text. This structure mirrors the chat interface used during RLHF alignment and ensures the model receives inputs in the format it was trained to follow.

The pseudocode below illustrates the general algorithm:

def make_conversation(example, prompt_column, system_prompt=None):
    messages = []
    if system_prompt:
        messages.append({"role": "system", "content": system_prompt})
    messages.append({"role": "user", "content": example[prompt_column]})
    return {"prompt": messages}

dataset = dataset.map(make_conversation)
if "messages" in dataset.columns:
    dataset = dataset.remove_columns("messages")

This algorithm processes each example individually — optionally prepending a system message, then appending the user prompt extracted from the configured column. After mapping, the original messages column is removed to prevent schema conflicts during training. For pass-rate filtering, an additional apply_chat_template step serializes the message list into a formatted string suitable for vLLM batch generation.

Related Pages

• Implementation:Huggingface_Open_r1_Make_Conversation