Implementation:Promptfoo Promptfoo Exit Code Logic

Knowledge Sources	Promptfoo CI/CD Integration
Domains	CI_CD, Quality_Assurance
Last Updated	2026-02-14 08:00 GMT

Overview

Concrete pattern for translating evaluation success/failure metrics into process exit codes for CI/CD pipeline integration, implemented in the Promptfoo eval command.

Description

The exit code logic at the end of doEval aggregates metrics (successes, failures, errors) from the evaluation results and sets `process.exitCode = 1` when any failures or errors are detected. This is a Pattern Doc documenting user-observable behavior rather than a callable API.

Usage

This pattern is automatically applied. No import or configuration is needed. The exit code is set by the `doEval` function after evaluation completes.

Code Reference

Source Location

• Repository: promptfoo
• File: src/commands/eval.ts
• Lines: L629-648

Signature

// Pattern: exit code logic within doEval()
// Not a standalone function - embedded in eval command handler

// Metrics aggregation (conceptual):
const { successes, failures, errors } = aggregateMetrics(evalRecord);
if (failures > 0 || errors > 0) {
  process.exitCode = 1;
}

Import

// Not importable - this is a behavior pattern within doEval
// The exit code is set automatically when running:
// npx promptfoo eval -c config.yaml

I/O Contract

Inputs

Name	Type	Required	Description
evalRecord	Eval	Yes	Completed evaluation record with per-prompt metrics

Outputs

Name	Type	Description
process.exitCode	0 or 1	0 when all tests pass, 1 when any fail or error

Usage Examples

CI Pipeline Gate

# In CI pipeline:
npx promptfoo eval -c config.yaml --no-cache

# Exit code 0: all tests pass → pipeline continues
# Exit code 1: failures detected → pipeline blocked
echo "Exit code: $?"

Related Pages

Implements Principle

• Principle:Promptfoo_Promptfoo_Quality_Gate_Enforcement