Principle:Hpcaitech ColossalAI SFT Data Preparation
| Knowledge Sources | |
|---|---|
| Domains | NLP, Data_Engineering |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A data engineering process that converts raw conversational datasets into tokenized, instruction-formatted sequences suitable for supervised fine-tuning of language models.
Description
SFT Data Preparation addresses the critical gap between raw text data and the structured input format required by language models during fine-tuning. Raw conversational data (in JSONL format with message roles and content) must be tokenized according to a specific conversation template, with loss masks applied to distinguish instruction tokens (which should not contribute to loss) from response tokens (which should). This process also handles sequence length truncation and padding to create uniform-length batches.
The ColossalAI implementation uses a CLI script that processes multiple dataset sources in parallel, applying conversation templates that vary by model architecture (LLaMA, Qwen, ChatGLM, etc.).
Usage
Use this principle when preparing instruction-following datasets for supervised fine-tuning. It is required before any SFT training can begin. The data preparation is a separate offline step that produces Arrow-format datasets consumed by the training pipeline.
Theoretical Basis
The core transformation follows the instruction-tuning paradigm:
- Raw conversations are parsed into role-tagged turns (system, human, assistant)
- Each turn is formatted using a model-specific conversation template
- The formatted text is tokenized into input_ids
- Loss masks are generated: 0 for instruction/prompt tokens, 1 for response tokens
- Sequences exceeding max_length are truncated; shorter sequences are padded
Pseudo-code:
# Abstract algorithm
for sample in raw_dataset:
formatted = apply_conversation_template(sample, template_config)
tokens = tokenizer.encode(formatted)
loss_mask = generate_loss_mask(tokens, response_boundaries)
if len(tokens) > max_length:
tokens = tokens[:max_length]
save_to_arrow(tokens, loss_mask)