Principle:Promptfoo Promptfoo Quality Gate Enforcement

Knowledge Sources	Promptfoo CI/CD Integration
Domains	CI_CD, Quality_Assurance
Last Updated	2026-02-14 08:00 GMT

Overview

A pass/fail gate mechanism that converts evaluation results into CI-interpretable exit codes for automated quality enforcement.

Description

Quality Gate Enforcement translates LLM evaluation results (pass counts, failure counts, error counts) into process exit codes that CI/CD systems use to gate deployments. When any test fails or errors occur, the process exits with code 1, blocking the pipeline.

This is the critical bridge between LLM evaluation outcomes and deployment decisions, ensuring that quality regressions are caught automatically.

Usage

This principle is automatically applied at the end of every `promptfoo eval` CLI execution. It requires no explicit configuration; any test failure triggers a non-zero exit code.

Theoretical Basis

Pseudo-code Logic:

1. After evaluation completes, aggregate metrics:
   successes = sum of passing tests across all prompts
   failures = sum of failing tests across all prompts
   errors = sum of error tests across all prompts
2. If failures > 0 OR errors > 0:
   process.exitCode = 1  (CI gate fails)
3. Else:
   process.exitCode = 0  (CI gate passes)

Related Pages

Implemented By

• Implementation:Promptfoo_Promptfoo_Exit_Code_Logic