Implementation:Openai Evals Pip Install Evals

Knowledge Sources	OpenAI Evals PyPI evals
Domains	Evaluation, DevOps
Last Updated	2026-02-14 10:00 GMT

Overview

Concrete tool for installing the OpenAI Evals framework and configuring CLI entry points provided by pip and setuptools.

Description

The evals package is distributed as a standard Python package using pyproject.toml with setuptools as the build backend. Installation registers two console script entry points: oaieval (single eval runner) and oaievalset (batch eval runner). The package requires Python 3.9+ and pulls in dependencies including openai, pyyaml, blobfile, lz4, pydantic, tqdm, numpy, and requests.

Usage

Use this when setting up a new environment for running OpenAI evaluations. This is the first step in any evaluation workflow and must be completed before using any other evals API.

Code Reference

Source Location

• Repository: openai/evals
• File: pyproject.toml (lines 1-89)

Configuration

[project]
name = "evals"
version = "3.0.1.post1"
requires-python = ">=3.9"
dependencies = [
    "openai",
    "pyyaml",
    "blobfile",
    "lz4",
    "pydantic",
    "tqdm",
    "numpy",
    "requests",
    "zstandard",
]

[project.scripts]
oaieval = "evals.cli.oaieval:main"
oaievalset = "evals.cli.oaievalset:main"

Import

# Installation command
pip install evals

# Or from source
pip install -e .

I/O Contract

Inputs

Name	Type	Required	Description
Python environment	Python 3.9+	Yes	Compatible Python interpreter
OPENAI_API_KEY	str (env var)	Yes	OpenAI API key for model access
Network access	—	Yes	To download package and dependencies

Outputs

Name	Type	Description
oaieval CLI	console script	Entry point for running single evaluations
oaievalset CLI	console script	Entry point for running eval sets
evals package	Python package	Importable evaluation framework

Usage Examples

Basic Installation and Verification

# Install the evals package
pip install evals

# Set required environment variable
export OPENAI_API_KEY="sk-your-key-here"

# Verify the CLI is available
oaieval --help

# Run a quick test eval
oaieval gpt-3.5-turbo test-match --max_samples 5

Development Installation

# Clone and install in editable mode
git clone https://github.com/openai/evals.git
cd evals
pip install -e .

# Install with optional dependencies for development
pip install -e ".[dev]"

Related Pages

Implements Principle

• Principle:Openai_Evals_Environment_Setup

Requires Environment

• Environment:Openai_Evals_Python_Runtime