Implementation:Hiyouga LLaMA Factory KTO En Demo Data

Knowledge Sources	Hiyouga_LLaMA_Factory
Domains	NLP, Training_Data
Last Updated	2026-02-06 19:00 GMT

Overview

kto_en_demo.json provides English Kahneman-Tversky Optimization (KTO) demo data with binary quality labels for demonstrating and testing feedback-based preference training workflows in LLaMA Factory.

Description

The file contains a JSON array of records, each with a messages list (multi-turn conversation in a messages format with "role" and "content" fields) and a boolean label field indicating whether the final assistant response is considered good (true) or bad (false). Unlike DPO data which requires paired chosen/rejected responses, KTO uses unpaired binary feedback signals, making it simpler to collect and more representative of real-world feedback scenarios.

The conversations cover a wide range of topics including science, general knowledge, and classification tasks. The messages format uses "role" (with values "user" and "assistant") and "content" tags, which is why explicit tag mappings are specified in the dataset registry entry.

Usage

This demo dataset is used for testing KTO training pipelines. Users reference it by name (kto_en_demo) with --stage kto in training configuration. KTO training only requires a binary signal per example rather than paired preferences, making it suitable when only thumbs-up/thumbs-down feedback is available.

Code Reference

Source Location

• Repository: Hiyouga_LLaMA_Factory
• File: data/kto_en_demo.json

Data Format

[
  {
    "messages": [
      {
        "content": "The Federal Trade Commission is going after spyware...\nQ: Which is the best summary?\n(A). World\n(B). Sports\n(C). Business\n(D). Science/Tech\nI think the answer is",
        "role": "user"
      },
      {
        "content": "The best summary of this article is... D! Science/Tech!",
        "role": "assistant"
      }
    ],
    "label": true
  },
  {
    "messages": [
      {
        "content": "Compare and contrast the roles of the hippocampus and the prefrontal cortex...",
        "role": "user"
      },
      {
        "content": "The human brain is a highly complex organ...",
        "role": "assistant"
      },
      {
        "content": "Discuss the mechanisms through which the prefrontal cortex modulates...",
        "role": "user"
      },
      {
        "content": "The prefrontal cortex (PFC) and the hippocampus have a dynamic relationship...",
        "role": "assistant"
      }
    ],
    "label": true
  }
]

I/O Contract

Schema

Field	Type	Required	Description
messages	array	Yes	List of conversation turns with "role" ("user" or "assistant") and "content" (message text)
label	boolean	Yes	Binary quality signal: true for good responses, false for bad responses

Message Turn Schema

Field	Type	Required	Description
role	string	Yes	Role identifier: "user" or "assistant"
content	string	Yes	The message content

Dataset Registry Entry

Property	Value
Key	kto_en_demo
file_name	kto_en_demo.json
formatting	sharegpt
columns.messages	messages
columns.kto_tag	label
tags.role_tag	role
tags.content_tag	content
tags.user_tag	user
tags.assistant_tag	assistant
Lines	5398

Usage Examples

# Train with KTO using binary feedback
# llamafactory-cli train \
#     --dataset kto_en_demo \
#     --stage kto \
#     --model_name_or_path meta-llama/Llama-2-7b-hf \
#     --output_dir output/kto_demo

# Loading the data manually for inspection
import json

with open("data/kto_en_demo.json", "r", encoding="utf-8") as f:
    data = json.load(f)

print(f"Number of samples: {len(data)}")

# Count positive vs negative labels
positive = sum(1 for item in data if item["label"])
negative = len(data) - positive
print(f"Positive (good) samples: {positive}")
print(f"Negative (bad) samples: {negative}")
print(f"Positive ratio: {positive/len(data):.2%}")

# Inspect first sample
sample = data[0]
print(f"Turns in first conversation: {len(sample['messages'])}")
print(f"Label: {sample['label']}")

Related Pages

• Implementation:Hiyouga_LLaMA_Factory_DPO_En_Demo_Data - Alternative preference format using paired chosen/rejected responses
• Implementation:Hiyouga_LLaMA_Factory_DPO_Zh_Demo_Data - Chinese DPO preference data
• Implementation:Hiyouga_LLaMA_Factory_Dataset_Info_Registry - Central dataset registry that indexes this file
• Implementation:Hiyouga_LLaMA_Factory_Alpaca_En_Demo_Data - English SFT demo data (non-preference)