Implementation:Promptfoo Promptfoo readTests

Knowledge Sources	Promptfoo Promptfoo Test Cases
Domains	Testing, Evaluation
Last Updated	2026-02-14 08:00 GMT

Overview

Concrete tool for loading test cases from CSV, YAML, JSON files, and inline objects into a uniform TestCase array, provided by the Promptfoo framework.

Description

The readTests function accepts test specifications in multiple formats (file paths, glob patterns, inline objects) and normalizes them into a flat array of TestCase objects. Each test case has vars (template variables), assert (assertions to grade), and options (per-test settings).

Usage

Import this function when loading test cases from external files or when test data is specified in heterogeneous formats that need normalization.

Code Reference

Source Location

• Repository: promptfoo
• File: src/util/testCaseReader.ts
• Lines: L410-454

Signature

export async function readTests(
  tests: TestSuiteConfig['tests'],
  basePath?: string,
): Promise<TestCase[]>

Import

import { readTests } from './util/testCaseReader';

I/O Contract

Inputs

Name	Type	Required	Description
tests	TestSuiteConfig['tests']	Yes	String path, object with path, or array of test case specs
basePath	string	No	Base directory for resolving relative file paths

Outputs

Name	Type	Description
(return)	TestCase[]	Flat array of test cases with vars, assert, and options fields

Usage Examples

Load Tests from CSV

import { readTests } from './util/testCaseReader';

const tests = await readTests('test_data.csv', '/path/to/project');
// Each CSV row becomes a TestCase with columns as vars
console.log(tests.length); // Number of rows

Inline and File Mixed

const tests = await readTests([
  { vars: { input: 'hello' }, assert: [{ type: 'contains', value: 'hi' }] },
  'more_tests.yaml',
]);

Related Pages

Implements Principle

• Principle:Promptfoo_Promptfoo_Test_Suite_Construction