Implementation:Langfuse Langfuse Seeder Data Generators

Knowledge Sources	Langfuse
Domains	Database Seeding, Test Data Generation, ClickHouse
Last Updated	2026-02-14 00:00 GMT

Overview

A singleton DataGenerator class that produces realistic test data for ClickHouse traces, observations, scores, dataset run items, and support chat sessions.

Description

The DataGenerator class is a singleton responsible for generating all ClickHouse-targeted seed data. It produces data in three primary categories:

1. Dataset Experiment Data:

• generateDatasetRunItem() -- Creates dataset run item records linking dataset items to experiment traces.
• generateDatasetTrace() -- Creates trace records from dataset items, transforming item input/output into natural language questions/answers (e.g., "What is the capital of France?" for countries dataset).
• generateDatasetObservation() -- Creates GENERATION-type observations for dataset traces with randomized token usage (30-150 input, 10-80 output) and costs.
• generateDatasetScore() -- Creates scores for dataset traces.
• generateDatasetRunScore() -- Creates run-level scores for dataset runs.

2. Evaluation Data:

• generateEvaluationTraces() -- Creates traces in the "langfuse-evaluation" environment with prefixed IDs for evaluator testing.
• generateEvaluationObservations() -- Creates multiple observations per evaluation trace.
• generateEvaluationScores() -- Creates exactly one NUMERIC score per evaluation trace (source: EVAL), skipping every 10th trace to simulate failures.

3. Synthetic Data:

• generateSyntheticTraces() -- Creates large-scale traces in the "default" environment with realistic names, tags, user IDs, session IDs, and randomized metadata. Uses loaded file content (nested JSON, heavy markdown, ChatML JSON) for realistic input/output data.
• generateSyntheticObservations() -- Creates observations of various types (SPAN, GENERATION, AGENT, TOOL, CHAIN, RETRIEVER, EVALUATOR, EMBEDDING, GUARDRAIL) with realistic names from constant arrays, randomized usage details, cost calculations, and model assignments.
• generateSyntheticScores() -- Creates scores of types: NUMERIC, BOOLEAN, CATEGORICAL, and CORRECTION with realistic values.

4. Support Chat Session Data:

• generateSupportChatSessionData() -- Creates a realistic multi-turn support chat session with 5 conversation turns, each containing a trace, generation observation, and helpfulness/safety/resolved scores.

The class uses a FileContent object (loaded from external JSON/markdown files) to populate trace input/output with realistic content. It relies on constants from clickhouse-seed-constants for realistic names and models, and helper functions from seed-helpers for deterministic ID generation.

Usage

Use this class when:

• Generating ClickHouse seed data for development or testing.
• Creating specific data scenarios (dataset experiments, evaluations, synthetic load).
• Adding new data generation patterns for seeding.

Code Reference

Source Location

• Repository: Langfuse
• File: packages/shared/scripts/seeder/utils/data-generators.ts
• Lines: 1-1323

Signature

export class DataGenerator {
  private static instance: DataGenerator;
  private fileContent: FileContent | null;

  static getInstance(): DataGenerator;
  setFileContent(content: FileContent): void;

  // Dataset experiment methods
  generateDatasetRunItem(input: DatasetItemInput & { runCreatedAt: number }, projectId: string): DatasetRunItemRecordInsertType;
  generateDatasetTrace(input: DatasetItemInput, projectId: string): TraceRecordInsertType;
  generateDatasetObservation(trace: TraceRecordInsertType, input: DatasetItemInput, projectId: string): ObservationRecordInsertType;
  generateDatasetScore(trace: TraceRecordInsertType, input: DatasetItemInput, projectId: string, scoreNames: string[]): ScoreRecordInsertType;
  generateDatasetRunScore(datasetId: string, input: {...}, projectId: string, scoreNames: string[]): ScoreRecordInsertType;

  // Evaluation methods
  generateEvaluationTraces(projectId: string, count: number): TraceRecordInsertType[];
  generateEvaluationObservations(traces: TraceRecordInsertType[], obsPerTrace: number, projectId: string): ObservationRecordInsertType[];
  generateEvaluationScores(traces: TraceRecordInsertType[], observations: ObservationRecordInsertType[], projectId: string): ScoreRecordInsertType[];

  // Synthetic data methods
  generateSyntheticTraces(projectId: string, count: number): TraceRecordInsertType[];
  generateSyntheticObservations(traces: TraceRecordInsertType[], obsPerTrace: number): ObservationRecordInsertType[];
  generateSyntheticScores(traces: TraceRecordInsertType[], observations: ObservationRecordInsertType[], scoresPerTrace: number): ScoreRecordInsertType[];

  // Support chat session
  generateSupportChatSessionData(projectId: string): { traces: TraceRecordInsertType[]; observations: ObservationRecordInsertType[]; scores: ScoreRecordInsertType[] };
}

Import

import { DataGenerator } from "./utils/data-generators";

const generator = DataGenerator.getInstance();

I/O Contract

Inputs

Name	Type	Required	Description
projectId	string	Yes	The project ID to associate generated data with
input	DatasetItemInput	Varies	Dataset item data including datasetName, itemIndex, item, runNumber
fileContent	FileContent	No	External file content (nested JSON, markdown, ChatML) for realistic I/O data
count / obsPerTrace / scoresPerTrace	number	Yes	Count parameters controlling how many records to generate

Outputs

Name	Type	Description
TraceRecordInsertType[]	Array of trace records	Trace records ready for ClickHouse insertion
ObservationRecordInsertType[]	Array of observation records	Observation records with usage, cost, and model details
ScoreRecordInsertType[]	Array of score records	Score records (NUMERIC, BOOLEAN, CATEGORICAL, CORRECTION)
DatasetRunItemRecordInsertType	Dataset run item record	Links dataset items to experiment trace runs

Usage Examples

import { DataGenerator } from "./utils/data-generators";

const generator = DataGenerator.getInstance();

// Generate synthetic traces for a project
const traces = generator.generateSyntheticTraces("project-123", 100);
const observations = generator.generateSyntheticObservations(traces, 15);
const scores = generator.generateSyntheticScores(traces, observations, 10);

// Generate evaluation data
const evalTraces = generator.generateEvaluationTraces("project-123", 50);
const evalObs = generator.generateEvaluationObservations(evalTraces, 10, "project-123");
const evalScores = generator.generateEvaluationScores(evalTraces, evalObs, "project-123");

// Generate dataset experiment trace
const trace = generator.generateDatasetTrace({
  datasetName: "demo-countries-dataset",
  itemIndex: 0,
  item: { input: { country: "France" }, output: "Paris" },
  runNumber: 0,
}, "project-123");