Environment:Promptfoo Promptfoo Python Runtime

Knowledge Sources	Promptfoo Python Provider Docs
Domains	Runtime, Custom_Providers
Last Updated	2026-02-14 08:00 GMT

Overview

Optional Python 3 runtime environment required for Python custom providers, Python assertions, and Python prompt processors.

Description

Promptfoo supports custom providers, assertions, and prompt processors written in Python. The Python runtime is optional and only needed when using Python-based features. The system uses the `python-shell` npm package to spawn Python subprocesses. Python path detection is platform-aware: on Windows it tries `where python`, the `py` launcher, and direct `python` commands; on Unix it tries `python3` first, then `python`. Each candidate is validated with a `--version` call (2.5-second timeout per path).

Usage

Use this environment when configuring a Python custom provider (`provider: python:my_script.py`), Python assertions (`type: python`), or Python prompt processors. Not required for standard LLM provider usage.

System Requirements

Category	Requirement	Notes
Runtime	Python 3.x	Any Python 3 version; detected automatically
OS	Linux, macOS, Windows	Platform-specific detection logic

Dependencies

System Packages

`python3` (or `python` on Windows)

Node.js Packages

`python-shell` ^5.0.0 (bundled with promptfoo)

Credentials

The following environment variables configure the Python runtime:

`PROMPTFOO_PYTHON`: Custom Python executable path (overrides auto-detection)
`PROMPTFOO_PYTHON_WORKERS`: Number of worker processes for Python provider (default: 1)
`PROMPTFOO_PYTHON_DEBUG_ENABLED`: Enable debug logging for Python execution

Quick Install

# Verify Python 3 is available
python3 --version

# Or set a custom Python path
export PROMPTFOO_PYTHON=/path/to/python3

Code Evidence

Python path resolution from `src/python/pythonUtils.ts:32-49`:

/**
 * Resolves the Python executable path from explicit config and environment.
 * This centralizes the fallback logic: configPath > PROMPTFOO_PYTHON env var.
 *
 * Note: Does NOT apply the final 'python' default - that's handled by
 * validatePythonPath. This preserves the distinction between "explicitly
 * configured" (should fail if invalid) and "using system default" (should
 * try fallback detection).
 */
export function getConfiguredPythonPath(configPath?: string): string | undefined {
  if (configPath) {
    return configPath;
  }
  const envPath = getEnvString('PROMPTFOO_PYTHON');
  return envPath || undefined;
}

Windows Store stub filtering from `src/python/pythonUtils.ts:78-80`:

// Skip Microsoft Store stubs and non-executables
if (trimmedPath.includes('WindowsApps') || !trimmedPath.endsWith('.exe')) {
  continue;
}

Common Errors

Error Message	Cause	Solution
`Python 3 is required but not found`	Python not installed or not on PATH	Install Python 3 or set `PROMPTFOO_PYTHON` env var
`Python validation timed out`	Python executable hangs on `--version`	Check for broken Python installations; set explicit path
`Windows Store stub detected`	Windows redirects `python` to Microsoft Store	Install Python from python.org instead of Store

Compatibility Notes

Windows: Auto-detects Python via `where python`, then `py` launcher. Microsoft Store stubs are filtered out automatically.
Unix/macOS: Tries `python3` first, then `python`. Set `PROMPTFOO_PYTHON` to override.
Virtual Environments: Supported; set `PROMPTFOO_PYTHON` to the venv Python path.
Workers: `PROMPTFOO_PYTHON_WORKERS` controls parallelism for Python providers (default: 1, can be overridden by config.workers).

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment