Implementation:Promptfoo Promptfoo evaluate

Knowledge Sources	Promptfoo Promptfoo Node Package
Domains	Evaluation, Testing
Last Updated	2026-02-14 08:00 GMT

Overview

Concrete tool for orchestrating the full evaluation of a test suite against LLM providers, provided by the Promptfoo framework.

Description

The evaluate function is the top-level entry point for running evaluations. It creates an Evaluator instance and delegates to its evaluate() method, which manages the full lifecycle: matrix generation, concurrent execution via the `async` library, progress reporting, and result aggregation.

The Evaluator class (L831-2386) contains the bulk of the logic, including conversation management for multi-turn tests, abort signal handling, and OpenTelemetry tracing integration.

Usage

Import this function for programmatic evaluation. It is also called internally by the CLI `eval` command via `doEval`.

Code Reference

Source Location

Repository: promptfoo
File: src/evaluator.ts
Lines: L2388-2395 (evaluate function), L831-2386 (Evaluator class), L290-830 (runEval)

Signature

export function evaluate(
  testSuite: TestSuite,
  evalRecord: Eval,
  options: EvaluateOptions,
): Promise<Eval>

Import

import { evaluate } from './evaluator';

I/O Contract

Inputs

Name	Type	Required	Description
testSuite	TestSuite	Yes	Complete test suite with providers, prompts, tests, and assertions
evalRecord	Eval	Yes	Database evaluation record to populate with results
options	EvaluateOptions	Yes	Execution options: maxConcurrency, repeat, delay, cache settings

Outputs

Name	Type	Description
(return)	Eval	Populated evaluation record with all results, metrics, and status

Usage Examples

Programmatic Evaluation

import { evaluate } from './evaluator';

const evalRecord = new Eval(config);
const result = await evaluate(testSuite, evalRecord, {
  maxConcurrency: 4,
  showProgressBar: true,
});

console.log(`Completed: ${result.results.length} test cases`);

Related Pages

Implements Principle

Principle:Promptfoo_Promptfoo_Evaluation_Execution

Requires Environment

Uses Heuristic

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment