Implementation:Arize ai Phoenix LLM Prompts

Overview

The LLM Prompts module provides the prompt and message template abstraction layer for the Phoenix evaluator framework. It resides at phoenix.evals.llm.prompts and defines the core types used across all LLM adapters for representing prompts, messages, and template rendering. The module supports two template formats -- Mustache ({{variable}}) and F-String ({variable}) -- with automatic format detection, and provides a hierarchical template system from individual content parts up to complete multi-message prompt templates.

Description

The module is organized into several layers of abstraction:

Core Types

• TemplateFormat (Enum): Defines the two supported template formats: MUSTACHE and F_STRING.
• MessageRole (Enum): Defines message roles: USER ("user"), AI ("assistant"), and SYSTEM ("system").
• TextContentPart (TypedDict): Represents a text content part with type: "text" and text: str fields, following the OpenAI content part format.
• ContentPart: Type alias for TextContentPart (extensible for future multimodal support).
• Message (TypedDict): A message with role: MessageRole and content: Union[str, List[ContentPart]].
• PromptLike: Type alias Union[str, List[Dict[str, Any]], List[Message]] representing all accepted prompt formats.

Template Formatters

• TemplateFormatter (ABC): Abstract base class with render() and extract_variables() methods.
• MustacheFormatter: Uses the pystache library for Mustache template rendering and variable extraction via recursive parse tree traversal.
• FStringFormatter: Uses Python's built-in string.Formatter with a custom _DotKeyFormatter that supports dotted key access (e.g., {data.name}).
• FormatterFactory: Factory class that creates formatters by type, or auto-detects format from a template string.

Format Detection

The detect_template_format() function analyzes a template string to determine whether it uses Mustache or F-String syntax. It distinguishes between genuine template variables and escaped JSON content (e.g., {{ used for JSON escaping in f-strings) through heuristic analysis of the match content.

Content Part Templates

• ContentPartTemplate (ABC): Abstract base class for content part templates with variables() and render() methods.
• TextContentPartTemplate: Template for text content parts that renders to a TextContentPart TypedDict.
• create_content_part_template(): Factory function that creates the appropriate template subclass from a content part dictionary.

Message and Prompt Templates

• MessageTemplate: Template for a single message with role and content. Handles both string content (rendered back to string) and structured content parts (rendered to list of ContentPart). Validates roles and extracts variables across all content parts.
• Template (deprecated): Simple string template class. Emits a DeprecationWarning in favor of PromptTemplate.
• PromptTemplate: The main template class supporting string templates, message lists, and copy construction from other templates. Renders to a List[Message] with dedented string content.

Usage

from phoenix.evals.llm.prompts import (
    PromptTemplate, MessageRole, Message, TemplateFormat
)

# Simple string template with auto-detected Mustache format
template = PromptTemplate(template="Classify the following text: {{text}}")
messages = template.render({"text": "I love this product!"})
# Returns: [Message(role=MessageRole.USER, content="Classify the following text: I love this product!")]

# Message list template with F-String format
template = PromptTemplate(
    template=[
        {"role": "system", "content": "You are a {role}."},
        {"role": "user", "content": "Analyze: {text}"},
    ],
    template_format=TemplateFormat.F_STRING,
)
messages = template.render({"role": "classifier", "text": "Great movie!"})

# Structured content parts within messages
template = PromptTemplate(
    template=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Score this review: {{review}}"},
            ],
        }
    ]
)
variables = template.variables  # ["review"]
messages = template.render({"review": "Excellent service."})

Code Reference

Symbol	Kind	Location	Lines
TemplateFormat	Enum	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	26-28
MessageRole	Enum	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	31-34
TextContentPart	TypedDict	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	37-41
Message	TypedDict	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	49-51
PromptLike	Type Alias	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	55
MustacheFormatter	Class	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	87-135
FStringFormatter	Class	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	138-177
detect_template_format()	Function	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	180-244
MessageTemplate	Class	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	413-514
Template	Class (deprecated)	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	517-588
PromptTemplate	Class	packages/phoenix-evals/src/phoenix/evals/llm/prompts.py	591-746

I/O Contract

PromptTemplate.render()

Direction	Type	Description
Input	Dict[str, Any]	A dictionary of variable names to their values for template substitution
Input	Optional[Tracer]	Optional OpenTelemetry tracer (currently unused but accepted for API compatibility)
Output	List[Message]	A list of rendered Message TypedDicts. String templates produce a single user message; message list templates produce one message per template entry.
Raises	TypeError	If variables is not a dictionary

detect_template_format()

Direction	Type	Description
Input	str	A template string to analyze
Output	TemplateFormat	The detected format: MUSTACHE or F_STRING

Usage Examples

Format Auto-Detection

from phoenix.evals.llm.prompts import detect_template_format, TemplateFormat

# Mustache detection
fmt = detect_template_format("Hello {{name}}, welcome to {{place}}!")
assert fmt == TemplateFormat.MUSTACHE

# F-String detection
fmt = detect_template_format("Hello {name}, welcome to {place}!")
assert fmt == TemplateFormat.F_STRING

Copy Construction

# Copy a template with a different format override
original = PromptTemplate(template="Score: {{score}}")
copy = PromptTemplate(template=original, template_format=TemplateFormat.F_STRING)

Variable Extraction

template = PromptTemplate(
    template=[
        {"role": "system", "content": "You are a {{role}}."},
        {"role": "user", "content": "Analyze: {{text}}"},
    ]
)
print(template.variables)  # ["role", "text"] (order may vary)

Related Pages

• Arize_ai_Phoenix_LLM_Base_Types - Uses PromptLike type alias for adapter method signatures
• Arize_ai_Phoenix_Anthropic_Adapter - Consumes Message and MessageRole types
• Arize_ai_Phoenix_OpenAI_Adapter - Consumes Message and MessageRole types
• Arize_ai_Phoenix_Google_Adapter - Consumes Message and MessageRole types
• Arize_ai_Phoenix_LangChain_Adapter - Consumes Message, MessageRole, and ContentPart types
• Arize_ai_Phoenix_LiteLLM_Adapter - Consumes Message and MessageRole types
• Arize_ai_Phoenix_Evals_Tracing - Accepts an optional Tracer parameter in template rendering