Implementation:Arize ai Phoenix Evals Public API

Overview

The Evals Public API is the top-level __init__.py module of the arize-phoenix-evals package (phoenix.evals). It serves as the main entry point for the package, re-exporting symbols from all submodules and providing a unified namespace for both the modern (v2) evaluator framework and the legacy (v1) evaluation system.

Description

This module consolidates the entire public API of phoenix-evals into a single import namespace. It re-exports classes, functions, constants, and submodules from:

• phoenix.evals.evaluators -- Core evaluator framework (v2), including base classes and evaluation utilities.
• phoenix.evals.legacy -- Legacy (v1) evaluation system, including all template constants, model classes, and evaluator wrappers.
• phoenix.evals.llm -- The LLM class for configuring judge models.
• phoenix.evals.metrics -- Built-in metric evaluators (re-exported as a submodule).
• phoenix.evals.templating -- Template utilities (re-exported as a submodule).
• phoenix.evals.tracing -- Tracing instrumentation (re-exported as a submodule).
• phoenix.evals.utils -- Utility functions (re-exported as a submodule).

The module also exposes the package version via __version__, read from the installed package metadata.

Usage

from phoenix.evals import *

Or import specific symbols:

from phoenix.evals import ClassificationEvaluator, LLM, Score, evaluate_dataframe

Code Reference

Property	Value
Source File	packages/phoenix-evals/src/phoenix/evals/__init__.py
Module	phoenix.evals
Lines	~184
Package	arize-phoenix-evals
Domain	API Surface

Exported Symbols

The __all__ list is organized into two sections: evals 1.0 (legacy) and evals 2.0 (modern).

Evals 2.0 (Modern Framework)

Symbol	Type	Description
ClassificationEvaluator	Class	Base class for LLM-based classification evaluators.
EvalInput	Type alias	Dict[str, Any] -- the standard input type for evaluators.
Evaluator	Class	Abstract base class for all evaluators (code-based and LLM-based).
LLMEvaluator	Class	Base class for LLM-powered evaluators.
Score	Dataclass	The standard return type for evaluation results.
ToolSchema	Type alias	Optional[Dict[str, Any]] -- schema for tool definitions.
KindType	Type alias	Literal["human", "llm", "heuristic", "code"].
create_classifier	Function	Factory for creating classification evaluators.
create_evaluator	Decorator	Decorator to convert a function into an evaluator.
async_evaluate_dataframe	Function	Asynchronous DataFrame evaluation utility.
evaluate_dataframe	Function	Synchronous DataFrame evaluation utility.
bind_evaluator	Function	Binds an evaluator to specific field mappings.
LLM	Class	Unified LLM client for configuring judge models.
metrics	Module	Submodule containing built-in metric evaluators.
templating	Module	Submodule for template utilities.
tracing	Module	Submodule for OpenTelemetry tracing instrumentation.
utils	Module	Submodule for utility functions.

Evals 1.0 (Legacy System)

Model Classes

Symbol	Description
OpenAIModel	OpenAI model wrapper.
AnthropicModel	Anthropic model wrapper.
GeminiModel	Google Gemini model wrapper.
GoogleGenAIModel	Google GenAI model wrapper.
VertexAIModel	Google Vertex AI model wrapper.
BedrockModel	AWS Bedrock model wrapper.
LiteLLMModel	LiteLLM unified model wrapper.
MistralAIModel	Mistral AI model wrapper.

Legacy Evaluators

Symbol	Description
HallucinationEvaluator	Legacy hallucination detection evaluator.
QAEvaluator	Legacy question-answering evaluator.
RelevanceEvaluator	Legacy relevance evaluator.
SQLEvaluator	Legacy SQL generation evaluator.
SummarizationEvaluator	Legacy summarization evaluator.
ToxicityEvaluator	Legacy toxicity evaluator.

Core Functions

Symbol	Description
llm_classify	Legacy LLM classification function.
llm_generate	Legacy LLM generation function.
run_evals	Legacy batch evaluation runner.
compute_precisions_at_k	Precision@K computation for retrieval evaluation.
download_benchmark_dataset	Download standard benchmark datasets.

Template Classes

Symbol	Description
PromptTemplate	Legacy prompt template class.
ClassificationTemplate	Legacy classification template class.

Prompt Template Constants

The module re-exports numerous prompt template constants organized by evaluation type. Each type provides up to four variants: base template, rails map, standard template, and template with explanation.

Evaluation Type	Constant Prefix
Hallucination	HALLUCINATION_PROMPT_*
Question Answering	QA_PROMPT_*
RAG Relevancy	RAG_RELEVANCY_PROMPT_*
Code Functionality	CODE_FUNCTIONALITY_PROMPT_*
Code Readability	CODE_READABILITY_PROMPT_*
Human vs. AI	HUMAN_VS_AI_PROMPT_*
Reference Link Correctness	REFERENCE_LINK_CORRECTNESS_PROMPT_*
SQL Generation	SQL_GEN_EVAL_PROMPT_*
Tool Calling	TOOL_CALLING_*
Toxicity	TOXICITY_PROMPT_*
User Frustration	USER_FRUSTRATION_PROMPT_*

Span Templates

Symbol	Description
HALLUCINATION_SPAN_PROMPT_TEMPLATE	Span-level hallucination prompt template.
QA_SPAN_PROMPT_TEMPLATE	Span-level QA prompt template.
TOOL_CALLING_SPAN_PROMPT_TEMPLATE	Span-level tool calling prompt template.

Other

Symbol	Description
NOT_PARSABLE	Sentinel value for unparsable LLM responses.

I/O Contract

This module is a re-export surface and does not define its own I/O contract. Each re-exported symbol has its own I/O contract defined in its respective source module.

Usage Examples

Using the Modern (v2) API

from phoenix.evals import ClassificationEvaluator, LLM, evaluate_dataframe
from phoenix.evals.metrics import CorrectnessEvaluator
import pandas as pd

llm = LLM(provider="openai", model="gpt-4o-mini")
evaluator = CorrectnessEvaluator(llm=llm)

df = pd.DataFrame({
    "input": ["What is 2+2?"],
    "output": ["4"],
})

results = evaluate_dataframe(df, evaluator)

Using the Legacy (v1) API

from phoenix.evals import (
    OpenAIModel,
    llm_classify,
    HALLUCINATION_PROMPT_TEMPLATE,
    HALLUCINATION_PROMPT_RAILS_MAP,
)

model = OpenAIModel(model="gpt-4o-mini")
results = llm_classify(
    dataframe=df,
    model=model,
    template=HALLUCINATION_PROMPT_TEMPLATE,
    rails=list(HALLUCINATION_PROMPT_RAILS_MAP.values()),
)

Related Pages

• Arize_ai_Phoenix_Legacy_Public_API -- The legacy (v1) submodule that provides backward-compatible evaluation tools.
• Arize_ai_Phoenix_CorrectnessEvaluator -- Built-in correctness metric evaluator.
• Arize_ai_Phoenix_FaithfulnessEvaluator -- Built-in faithfulness metric evaluator.
• Arize_ai_Phoenix_PrecisionRecallFScore -- Built-in precision/recall/F-score metric evaluator.
• Arize_ai_Phoenix_ExactMatch -- Built-in exact match metric evaluator.