Implementation:Arize ai Phoenix Legacy Span Templates

Overview

The Legacy Span Templates module defines predefined prompt templates for span-level evaluations in the Phoenix Evals subsystem. Span-level evaluations operate on individual OpenTelemetry spans rather than standalone text records, allowing evaluation of specific steps within an LLM application pipeline (e.g., a single retrieval-augmented generation call or tool invocation).

The module provides three ClassificationTemplate constants: HALLUCINATION_SPAN_PROMPT_TEMPLATE (detecting hallucinations in span outputs), QA_SPAN_PROMPT_TEMPLATE (evaluating question-answering correctness at the span level), and TOOL_CALLING_SPAN_PROMPT_TEMPLATE (assessing tool call correctness). Each constant includes both a base template and an explanation template for chain-of-thought reasoning.

The span templates use system_message, user_message, and output as their primary template variables (for hallucination and QA), and question, tool_call, and tool_definitions (for tool calling), reflecting the structure of traced LLM conversation spans.

Code Reference

Attribute	Details
Source File	`packages/phoenix-evals/src/phoenix/evals/legacy/span_templates.py`
Repository	Arize-ai/phoenix
Lines	309
Module	`phoenix.evals.legacy.span_templates`
Key Symbols	`HALLUCINATION_SPAN_PROMPT_TEMPLATE`, `QA_SPAN_PROMPT_TEMPLATE`, `TOOL_CALLING_SPAN_PROMPT_TEMPLATE`
Dependencies	`phoenix.evals.legacy.templates.ClassificationTemplate`

I/O Contract

Template Constants

Template	Rails	Template Variables	Purpose
`HALLUCINATION_SPAN_PROMPT_TEMPLATE`	`["hallucinated", "factual"]`	`{system_message}`, `{user_message}`, `{output}`	Evaluates whether an AI assistant's answer contains hallucinated facts relative to the system/user context provided in a span.
`QA_SPAN_PROMPT_TEMPLATE`	`["correct", "incorrect"]`	`{system_message}`, `{user_message}`, `{output}`	Evaluates whether an AI assistant correctly answered a question based on the reference context in a span.
`TOOL_CALLING_SPAN_PROMPT_TEMPLATE`	`["correct", "incorrect"]`	`{question}`, `{tool_call}`, `{tool_definitions}`	Evaluates whether the correct tool was called with correct parameters for a given question.

All templates use scores [1, 0] where the first rail receives a score of 1 and the second receives 0.

Rails Maps

Constant	Mapping
`HALLUCINATION_SPAN_PROMPT_RAILS_MAP`	`OrderedDict({True: "hallucinated", False: "factual"})`
`QA_SPAN_PROMPT_RAILS_MAP`	`OrderedDict({True: "correct", False: "incorrect"})`
`TOOL_CALLING_SPAN_PROMPT_RAILS_MAP`	`OrderedDict({True: "correct", False: "incorrect"})`

Template Structure

Each template constant is built from paired base/explanation raw template strings:

Base Template	Explanation Template	Description
`HALLUCINATION_SPAN_PROMPT_BASE_TEMPLATE`	`HALLUCINATION_SPAN_PROMPT_TEMPLATE_WITH_EXPLANATION`	Direct "factual"/"hallucinated" classification vs. step-by-step reasoning
`QA_SPAN_PROMPT_BASE_TEMPLATE`	`QA_SPAN_PROMPT_TEMPLATE_WITH_EXPLANATION`	Direct "correct"/"incorrect" classification vs. step-by-step reasoning
`TOOL_CALLING_SPAN_BASE_TEMPLATE`	`TOOL_CALLING_SPAN_TEMPLATE_WITH_EXPLANATION`	Direct "correct"/"incorrect" classification vs. step-by-step reasoning

Usage Examples

from phoenix.evals.legacy.span_templates import (
    HALLUCINATION_SPAN_PROMPT_TEMPLATE,
    QA_SPAN_PROMPT_TEMPLATE,
    TOOL_CALLING_SPAN_PROMPT_TEMPLATE,
)
from phoenix.evals.legacy.classify import llm_classify
from phoenix.evals.legacy.models import OpenAIModel
import pandas as pd

model = OpenAIModel(model="gpt-4")

# Span-level hallucination evaluation
spans_df = pd.DataFrame({
    "system_message": ["You are a helpful assistant. Use the context to answer."],
    "user_message": ["What is the population of Tokyo? Context: Tokyo has 14M people."],
    "output": ["The population of Tokyo is 37 million."],
})

result = llm_classify(
    data=spans_df,
    model=model,
    template=HALLUCINATION_SPAN_PROMPT_TEMPLATE,
    rails=["hallucinated", "factual"],
    provide_explanation=True,
)

# Span-level tool calling evaluation
tool_spans_df = pd.DataFrame({
    "question": ["What is the weather in Paris?"],
    "tool_call": ["get_weather(city='Paris', unit='celsius')"],
    "tool_definitions": ["get_weather(city: str, unit: str) - returns current weather"],
})

result = llm_classify(
    data=tool_spans_df,
    model=model,
    template=TOOL_CALLING_SPAN_PROMPT_TEMPLATE,
    rails=["correct", "incorrect"],
)

Related Pages

Implementation:Arize_ai_Phoenix_Legacy_Default_Templates - Document-level templates that import span-level templates
Implementation:Arize_ai_Phoenix_Legacy_Classify - llm_classify() function that consumes these templates
Implementation:Arize_ai_Phoenix_Legacy_Templates - ClassificationTemplate base class
Implementation:Arize_ai_Phoenix_Legacy_Evaluators - LLMEvaluator classes using EvalCriteria span-level entries

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment