Implementation:Promptfoo Promptfoo Promptfoo CLI

Knowledge Sources	Promptfoo CLI Reference CI/CD Guide
Domains	CI_CD, DevOps
Last Updated	2026-02-14 08:00 GMT

Overview

Concrete CLI tool for running LLM evaluations and red team scans in CI/CD pipelines, provided by the Promptfoo framework.

Description

The promptfoo CLI provides `eval` and `redteam run` commands that serve as the execution interface for CI/CD integration. The CLI is registered via Commander.js in `src/main.ts` and delegates to doEval for evaluation execution. Exit codes are set based on test results: 0 for all tests passing, 1 when failures or errors are detected.

Usage

Use the CLI in any CI pipeline YAML configuration. Install via `npx promptfoo@latest` or as a project dependency.

Code Reference

Source Location

• Repository: promptfoo
• File: src/main.ts (CLI registration)
• Lines: L1-370
• File: src/commands/eval.ts (doEval handler)
• Lines: L85-800

Signature

# Evaluation
npx promptfoo eval -c <config-path> [--no-cache] [-o <output-path>] [--verbose]

# Red team scan
npx promptfoo redteam run -c <config-path> [-o <output-path>]

# Exit codes:
# 0 = all tests pass
# 1 = failures or errors detected

Import

# No import needed - CLI tool
npx promptfoo@latest eval
# Or install as dependency:
npm install promptfoo

I/O Contract

Inputs

Name	Type	Required	Description
-c, --config	file path(s)	No	Config file paths (default: promptfooconfig.yaml in cwd)
-o, --output	file path(s)	No	Output paths for results (JSON, CSV, HTML, YAML)
--no-cache	flag	No	Disable result caching for fresh API calls
--verbose	flag	No	Enable debug logging
--env-file	file path	No	Load environment variables from file

Outputs

Name	Type	Description
Exit code	integer	0 for success, 1 for failures/errors
Output files	files	Written to -o paths in specified formats
Console output	text	Progress bar, results table, summary statistics

Usage Examples

GitHub Actions

# .github/workflows/llm-eval.yml
name: LLM Evaluation
on: [push, pull_request]
jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npx promptfoo@latest eval -c promptfooconfig.yaml --no-cache -o results.json
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: eval-results
          path: results.json

Related Pages

Implements Principle

• Principle:Promptfoo_Promptfoo_CI_Pipeline_Configuration

Requires Environment

• Environment:Promptfoo_Promptfoo_Node_Runtime