Implementation:Arize ai Phoenix LiteLLM Adapter
Overview
The LiteLLMAdapter is a concrete implementation of the BaseLLMAdapter abstract class that wraps the LiteLLM unified API for use within the Phoenix evaluator framework. It resides in the phoenix-evals sub-package and provides a single adapter that supports multiple LLM providers -- OpenAI, Anthropic, Vertex AI, Amazon Bedrock, and generic LiteLLM -- through LiteLLM's model routing and provider-specific message translation. The adapter supports text, image, and audio content in prompts, and provides both synchronous and asynchronous generation paths.
Description
The module defines the LiteLLMAdapter class and a supporting function get_litellm_rate_limit_errors(). Key design characteristics:
- Multi-provider registration: The adapter is decorated with five
@register_providerdecorators, each specifying a different provider name (openai,anthropic,vertex,bedrock,litellm) with corresponding client factories and dependency lists. This means the adapter can be selected for any of these providers. - LiteLLM client validation: The adapter requires a
LiteLLMClientinstance (a lightweight wrapper holding the model string and configuration). It useslitellm.completion()andlitellm.acompletion()directly rather than going through a provider SDK. - Dynamic capability detection: For object generation, the adapter queries
litellm.get_supported_openai_params()to determine whether the model supportsresponse_format(structured output) and/ortools(tool calling). - Schema normalization: The
_ensure_additional_properties_false()method recursively adds"additionalProperties": falseto all object types in the schema, which is required by OpenAI's structured output API and is generally harmless for other providers. - Multimodal support: The
_build_messages()method handles legacyMultimodalPromptobjects, converting text, image (base64 or URL), and audio parts into OpenAI-compatible message content arrays. - Unified rate limiting: All providers share the same
litellm.RateLimitErrortype.
Usage
from phoenix.evals.llm.adapters.litellm.adapter import LiteLLMAdapter
from phoenix.evals.llm.adapters.litellm.client import LiteLLMClient
# Create a LiteLLM client for OpenAI
client = LiteLLMClient(model="gpt-4o", api_key="sk-...")
adapter = LiteLLMAdapter(client=client, model="gpt-4o")
# Text generation (uses litellm.completion() internally)
response = adapter.generate_text("What is the capital of France?")
# Async text generation (uses litellm.acompletion() internally)
response = await adapter.async_generate_text("Summarize this document.")
# Structured output with automatic method selection
schema = {
"type": "object",
"properties": {
"label": {"type": "string", "enum": ["positive", "negative"]},
"score": {"type": "number"}
},
"required": ["label", "score"]
}
# AUTO mode: prefers structured output, falls back to tool calling
result = adapter.generate_object("Classify: I love this!", schema=schema)
print(result) # {"label": "positive", "score": 0.95}
Code Reference
I/O Contract
generate_text / async_generate_text
| Direction | Type | Description |
|---|---|---|
| Input | Union[PromptLike, MultimodalPrompt] |
A string prompt, list of message dicts, list of Message TypedDicts, or legacy MultimodalPrompt (with text, image, and audio parts)
|
| Input | **kwargs |
Additional keyword arguments forwarded to litellm.completion() or litellm.acompletion()
|
| Output | str |
The choices[0].message.content from the LiteLLM response
|
| Raises | ValueError |
If the returned content is None
|
| Raises | PhoenixUnsupportedAudioFormat |
If an audio part uses an unsupported format |
generate_object / async_generate_object
| Direction | Type | Description |
|---|---|---|
| Input | Union[PromptLike, MultimodalPrompt] |
A prompt in any supported format |
| Input | Dict[str, Any] |
A JSON schema defining the expected output structure |
| Input | ObjectGenerationMethod |
AUTO (default: prefers structured output), STRUCTURED_OUTPUT, or TOOL_CALLING
|
| Output | Dict[str, Any] |
Parsed JSON from response_format content, or parsed function arguments from tool calls
|
| Raises | ValueError |
If the model does not support the requested method, or if the returned content is None
|
Usage Examples
Multi-Provider Routing
# LiteLLM uses model prefixes for provider routing
# OpenAI models: "gpt-4o", "gpt-4o-mini"
# Anthropic models: "anthropic/claude-sonnet-4-20250514"
# Vertex models: "vertex_ai/gemini-2.0-flash"
# Bedrock models: "bedrock/anthropic.claude-v2"
client = LiteLLMClient(model="anthropic/claude-sonnet-4-20250514")
adapter = LiteLLMAdapter(client=client, model="claude-sonnet-4-20250514")
response = adapter.generate_text("Hello!")
Multimodal Prompts
from phoenix.evals.legacy.templates import MultimodalPrompt, PromptPart, PromptPartContentType
# Image + text prompt
prompt = MultimodalPrompt(parts=[
PromptPart(content_type=PromptPartContentType.TEXT, content="Describe this image:"),
PromptPart(content_type=PromptPartContentType.IMAGE, content="https://example.com/img.jpg"),
])
response = adapter.generate_text(prompt)
Related Pages
- Arize_ai_Phoenix_LLM_Base_Types - Abstract base class
BaseLLMAdapterand core type definitions - Arize_ai_Phoenix_OpenAI_Adapter - Direct OpenAI adapter (without LiteLLM routing)
- Arize_ai_Phoenix_Anthropic_Adapter - Direct Anthropic adapter (without LiteLLM routing)
- Arize_ai_Phoenix_LLM_Prompts - Prompt template and message abstraction layer
- Arize_ai_Phoenix_Evals_Rate_Limiters - Rate limiting subsystem integrated via
litellm.RateLimitError - Arize_ai_Phoenix_Evals_Executors - Task execution framework that orchestrates adapter calls
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment