Implementation:Arize ai Phoenix LLM Class

Knowledge Sources	Phoenix
Domains	LLM Evaluation, AI Observability, Provider Abstraction
Last Updated	2026-02-14 00:00 GMT

Overview

Concrete tool for initializing and interacting with large language model providers provided by arize-phoenix-evals.

Description

The LLM class is a unified wrapper that delegates text generation, structured object generation, and classification to any supported LLM provider SDK installed in the Python environment. It creates both synchronous and asynchronous adapter instances at initialization, builds rate limiters for each known rate-limit error type, and exposes a uniform set of generation methods. This class is the primary entry point for configuring an LLM instance that will be consumed by Phoenix evaluators such as ClassificationEvaluator.

Usage

Use LLM when you need to:

Create an LLM instance for passing into any Phoenix evaluator.
Generate freeform text, structured JSON objects, or classification labels from a prompt.
Control rate limiting behavior for high-throughput evaluation workloads.
Use provider-specific SDK parameters (API keys, base URLs, timeouts) without coupling to a single provider.

Code Reference

Source Location

Repository: Phoenix
File: packages/phoenix-evals/src/phoenix/evals/llm/wrapper.py (lines 98-247)

Signature

class LLM:
    def __init__(
        self,
        *,
        provider: Optional[str] = None,
        model: Optional[str] = None,
        client: Optional[str] = None,
        initial_per_second_request_rate: Optional[float] = None,
        sync_client_kwargs: Optional[Dict[str, Any]] = None,
        async_client_kwargs: Optional[Dict[str, Any]] = None,
        **kwargs: Any,
    ) -> None

Import

from phoenix.evals import LLM

I/O Contract

Inputs

Name	Type	Required	Description
provider	`Optional[str]`	Yes	Name of the LLM provider (e.g., `"openai"`, `"anthropic"`, `"azure"`, `"google"`). Used to look up the appropriate adapter from the provider registry.
model	`Optional[str]`	Yes	Model identifier for the provider (e.g., `"gpt-4o"`, `"claude-3-opus"`). Must be specified together with provider.
client	`Optional[str]`	No	Optionally selects a specific client name when multiple clients are registered for a provider. If omitted, the first available client is used.
initial_per_second_request_rate	`Optional[float]`	No	Sets the starting per-second request rate for the adaptive rate limiter. If omitted, the default rate is used.
sync_client_kwargs	`Optional[Dict[str, Any]]`	No	Keyword arguments forwarded exclusively to the synchronous SDK client constructor. Overrides matching keys in `**kwargs` for the sync client only.
async_client_kwargs	`Optional[Dict[str, Any]]`	No	Keyword arguments forwarded exclusively to the asynchronous SDK client constructor. Overrides matching keys in `**kwargs` for the async client only.
**kwargs	`Any`	No	Additional keyword arguments forwarded to both sync and async SDK client constructors (e.g., `api_key`, `base_url`, `api_version`).

Outputs

Name	Type	Description
LLM instance	`LLM`	A fully initialized wrapper containing `_sync_adapter`, `_async_adapter`, and `_rate_limiters` ready for use by evaluators and direct generation calls.

Key Methods

Method	Signature	Description
generate_text	`generate_text(prompt, **kwargs) -> str`	Synchronous text generation from a prompt string or multimodal prompt.
generate_object	`generate_object(prompt, schema, **kwargs) -> Dict[str, Any]`	Synchronous structured object generation guided by a JSON schema.
generate_classification	`generate_classification(prompt, labels, include_explanation=True, **kwargs) -> Dict[str, Any]`	Synchronous classification producing a label and optional explanation.
async_generate_text	`async_generate_text(prompt, **kwargs) -> str`	Asynchronous variant of `generate_text`.
async_generate_object	`async_generate_object(prompt, schema, **kwargs) -> Dict[str, Any]`	Asynchronous variant of `generate_object`.
async_generate_classification	`async_generate_classification(prompt, labels, include_explanation=True, **kwargs) -> Dict[str, Any]`	Asynchronous variant of `generate_classification`.

Usage Examples

Basic OpenAI Initialization

from phoenix.evals import LLM

llm = LLM(provider="openai", model="gpt-4o")
response = llm.generate_text(prompt="Explain quantum computing in one sentence.")
print(response)

Azure OpenAI with Custom Parameters

from phoenix.evals import LLM

llm = LLM(
    provider="azure",
    model="gpt-4o",
    api_key="your-azure-api-key",
    api_version="2024-02-01",
    base_url="https://your-resource.openai.azure.com/",
)

Separate Sync/Async Timeouts

from phoenix.evals import LLM

llm = LLM(
    provider="openai",
    model="gpt-4o",
    api_key="your-api-key",
    sync_client_kwargs={"timeout": 60.0},
    async_client_kwargs={"timeout": 120.0},
)

Structured Object Generation

from phoenix.evals import LLM

llm = LLM(provider="openai", model="gpt-4o")
result = llm.generate_object(
    prompt="Extract the person's name and age from: 'John is 30 years old.'",
    schema={
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
        },
        "required": ["name", "age"],
    },
)
print(result)  # {"name": "John", "age": 30}

Classification with Labels

from phoenix.evals import LLM

llm = LLM(provider="openai", model="gpt-4o")
result = llm.generate_classification(
    prompt="Is this statement factual? 'Water boils at 100 degrees Celsius at sea level.'",
    labels=["factual", "not_factual"],
    include_explanation=True,
)
print(result)
# {"label": "factual", "explanation": "Water does boil at 100 degrees Celsius ..."}

Custom Rate Limiting

from phoenix.evals import LLM

# Limit initial throughput to 5 requests per second
llm = LLM(
    provider="openai",
    model="gpt-4o",
    initial_per_second_request_rate=5.0,
)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment