Implementation:Langfuse Langfuse ExperimentsRouter CreateExperiment

Knowledge Sources	Langfuse
Domains	LLM Experimentation, Job Orchestration
Last Updated	2026-02-14 00:00 GMT

Overview

Concrete tool for creating a dataset run record with experiment metadata and dispatching an asynchronous processing job, provided by Langfuse.

Description

experimentsRouter.createExperiment is a tRPC mutation procedure in the Langfuse web application. When invoked, it performs two actions:

1. Creates a datasetRuns record in PostgreSQL via Prisma, storing the full experiment configuration (prompt ID, model provider, model name, model parameters, optional structured output schema, and dataset version) as JSON metadata.
2. Enqueues a job to ExperimentCreateQueue, a BullMQ queue backed by Redis, with the project, dataset, and run identifiers as the payload.

The mutation is gated by RBAC authorization, requiring the promptExperiments:CUD scope on the project. It also requires Redis to be available; if Redis is not configured, the mutation throws an UnauthorizedError because the experiment cannot be processed without the queue.

The ExperimentCreateQueue is configured with 10 retry attempts and exponential backoff starting at 10 seconds. The worker that consumes this queue (createExperimentJobClickhouse) retrieves the dataset run record, extracts the metadata, and processes each dataset item.

Usage

This mutation is called when a user submits the experiment creation form in the Langfuse UI. The frontend calls trpc.experiments.createExperiment.mutate() and, upon receiving the runId in the response, navigates the user to the experiment run detail page where results will appear as items are processed.

Code Reference

Source Location

• Repository: langfuse
• File: web/src/features/experiments/server/router.ts
• Lines: 177-259

Signature

experimentsRouter.createExperiment: protectedProjectProcedure
  .input(
    z.object({
      projectId: z.string(),
      name: z.string().min(1, "Please enter an experiment name"),
      runName: z.string().min(1, "Run name is required"),
      promptId: z.string().min(1, "Please select a prompt"),
      datasetId: z.string().min(1, "Please select a dataset"),
      datasetVersion: z.coerce.date().optional(),
      description: z.string().max(1000).optional(),
      modelConfig: z.object({
        provider: z.string().min(1, "Please select a provider"),
        model: z.string().min(1, "Please select a model"),
        modelParams: ZodModelConfig,
      }),
      structuredOutputSchema: z.record(z.string(), z.any()).optional(),
    }),
  )
  .mutation(async ({ input, ctx }) => { ... })

Import

import { experimentsRouter } from "@/src/features/experiments/server/router";

I/O Contract

Inputs

Name	Type	Required	Description
projectId	string	Yes	The project under which the experiment is created.
name	string	Yes	Human-readable experiment name (minimum 1 character).
runName	string	Yes	Name for the dataset run record. Appears in the runs table.
promptId	string	Yes	ID of the prompt to use for LLM calls.
datasetId	string	Yes	ID of the dataset whose items will be processed.
datasetVersion	Date (coerced)	No	Optional version timestamp to filter dataset items.
description	string (max 1000)	No	Optional description for the experiment run.
modelConfig.provider	string	Yes	LLM provider name (e.g. "openai", "anthropic").
modelConfig.model	string	Yes	Model identifier (e.g. "gpt-4", "claude-3-opus").
modelConfig.modelParams	ZodModelConfig	Yes	Model parameters including temperature, maxTokens, topP, etc.
structuredOutputSchema	Record<string, any>	No	Optional JSON schema for structured output mode.

Outputs

Name	Type	Description
success	boolean	Always true on successful creation.
datasetId	string	The dataset ID passed in the input, echoed back for convenience.
runId	string	The ID of the newly created datasetRuns record.
runName	string	The run name passed in the input, echoed back for convenience.

Usage Examples

Creating an Experiment

const result = await trpc.experiments.createExperiment.mutate({
  projectId: "proj_abc123",
  name: "GPT-4 vs Claude QA Experiment",
  runName: "gpt4-run-001",
  promptId: "prompt_ghi789",
  datasetId: "ds_def456",
  description: "Testing GPT-4 on our QA dataset with default params",
  modelConfig: {
    provider: "openai",
    model: "gpt-4",
    modelParams: {
      temperature: 0.7,
      maxTokens: 1024,
      topP: 1,
    },
  },
});

console.log(`Experiment created: run ${result.runId}`);
// Navigate to run detail page
router.push(`/project/${result.datasetId}/runs/${result.runId}`);

Creating an Experiment with Structured Output

const result = await trpc.experiments.createExperiment.mutate({
  projectId: "proj_abc123",
  name: "Structured Extraction Experiment",
  runName: "extraction-run-001",
  promptId: "prompt_extraction",
  datasetId: "ds_documents",
  modelConfig: {
    provider: "openai",
    model: "gpt-4",
    modelParams: { temperature: 0 },
  },
  structuredOutputSchema: {
    type: "object",
    properties: {
      entities: { type: "array", items: { type: "string" } },
      sentiment: { type: "string", enum: ["positive", "negative", "neutral"] },
    },
    required: ["entities", "sentiment"],
  },
});

Related Pages

Implements Principle

• Principle:Langfuse_Langfuse_Experiment_Run_Creation