Implementation:Arize ai Phoenix Evals Tracing

Overview

The Evals Tracing module provides OpenTelemetry-based tracing decorators for instrumenting evaluation functions within the Phoenix evaluator framework. It resides at phoenix.evals.tracing and defines the trace() decorator and get_tracer() utility function. The trace() decorator transparently wraps both synchronous and asynchronous functions with span creation, input/output attribute recording, error handling, and automatic trace ID injection, all while guaranteeing that tracing failures never disrupt the underlying business logic.

Description

The module contains the following key components:

get_tracer() Function

Obtains a tracer instance following a priority chain:

1. Use the explicitly provided Tracer if given.
2. Pull from the global TracerProvider via trace_api.get_tracer_provider().
3. Fall back to NoOpTracer if all else fails.

The function also ensures the tracer is wrapped in an OITracer (from openinference.instrumentation) with a default TraceConfig if it is not already an OITracer instance. This ensures consistent OpenInference semantic conventions.

trace() Decorator

A parameterized decorator with the following capabilities:

• Span creation: Creates a span using the resolved tracer. The span name defaults to the function's __qualname__ but can be overridden with a string or a callable that receives BoundArguments.
• Span kind: Optionally sets the OPENINFERENCE_SPAN_KIND attribute using OpenInferenceSpanKindValues.
• Input processing: Accepts a mapping of attribute names to callables that extract values from the function's bound arguments and set them as span attributes.
• Output processing: Accepts a mapping of attribute names to callables that extract values from the function's return value and set them as span attributes.
• Trace ID injection: If the decorated function has a trace_id parameter, the decorator automatically injects the hex-encoded trace ID from the span context into the function's kwargs.
• Tracer resolution from arguments: If the decorated function has a tracer argument, it is extracted and used preferentially over the decorator-level tracer.
• Error recording: Exceptions raised by the function are recorded on the span with record_exception() and the span status is set to ERROR.
• Fault tolerance: The decorator is designed so that tracing failures never prevent the function from executing. If span creation fails, the function is called without tracing. If function execution succeeds but tracing teardown fails, the result is still returned.
• Sync/async support: Automatically detects whether the decorated function is a coroutine and wraps it with the appropriate sync or async wrapper.

Helper Functions

• _format_trace_id(): Converts an integer trace ID to a hex-encoded string representation (16 bytes, big-endian).
• _otel_attribute_value(): Converts arbitrary values to OpenTelemetry-compatible AttributeValue types (bool, int, float, str, or list of strings).

Usage

from phoenix.evals.tracing import trace, get_tracer
from openinference.semconv.trace import OpenInferenceSpanKindValues

# Basic tracing of a sync function
@trace(span_name="my_evaluation")
def evaluate_text(text: str) -> str:
    return "positive"

# Tracing with span kind and input/output processing
@trace(
    span_name="classify",
    span_kind=OpenInferenceSpanKindValues.EVALUATOR,
    process_input={"input.text": lambda bound: bound.arguments.get("text")},
    process_output={"output.label": lambda result: result},
)
def classify(text: str) -> str:
    return "relevant"

# Async function tracing
@trace(span_name="async_eval")
async def async_evaluate(text: str) -> str:
    return await some_async_llm_call(text)

# Automatic trace_id injection
@trace(span_name="eval_with_trace")
def evaluate_with_trace(text: str, trace_id: str = "") -> str:
    # trace_id is automatically populated with the hex trace ID
    print(f"Trace ID: {trace_id}")
    return "result"

Code Reference

Symbol	Kind	Location	Lines
get_tracer()	Function	packages/phoenix-evals/src/phoenix/evals/tracing.py	28-53
_format_trace_id()	Function	packages/phoenix-evals/src/phoenix/evals/tracing.py	60-69
trace()	Decorator	packages/phoenix-evals/src/phoenix/evals/tracing.py	96-331
_wrapper_sync()	Inner Function	packages/phoenix-evals/src/phoenix/evals/tracing.py	133-228
_wrapper_async()	Inner Function	packages/phoenix-evals/src/phoenix/evals/tracing.py	230-325
_otel_attribute_value()	Function	packages/phoenix-evals/src/phoenix/evals/tracing.py	334-348

I/O Contract

trace() Decorator Parameters

Parameter	Type	Description
span_name	Union[str, Callable[[BoundArguments], str], None]	Span name or a callable that computes it from bound arguments. Defaults to func.__qualname__.
span_kind	Optional[OpenInferenceSpanKindValues]	The OpenInference span kind (e.g., EVALUATOR, LLM)
tracer	Optional[Tracer]	Explicit tracer to use. If not provided, resolved from function args or global provider.
process_input	Optional[Mapping[str, Callable[[BoundArguments], Any]]]	Mapping of span attribute keys to input extraction functions
process_output	Optional[Mapping[str, Callable[[ReturnValue], Any]]]	Mapping of span attribute keys to output extraction functions

get_tracer()

Direction	Type	Description
Input	Optional[Tracer]	An optional tracer to use directly
Output	Tracer	An OITracer-wrapped tracer instance, or NoOpTracer as fallback

Usage Examples

Dynamic Span Name

@trace(
    span_name=lambda bound: f"eval_{bound.arguments.get('eval_name', 'unknown')}",
)
def run_evaluation(eval_name: str, data: dict) -> str:
    return "result"

# Creates span with name "eval_hallucination" when called as:
run_evaluation(eval_name="hallucination", data={})

Fault-Tolerant Tracing

# Even if tracing completely fails, the function still executes
@trace(span_name="safe_eval")
def critical_evaluation(text: str) -> str:
    return "important_result"

# This will always return "important_result", even if the tracer is misconfigured
result = critical_evaluation("test")

Related Pages

• Arize_ai_Phoenix_LLM_Prompts - PromptTemplate.render() accepts an optional Tracer parameter
• Arize_ai_Phoenix_Evals_Executors - The execution framework that invokes traced evaluation functions
• Arize_ai_Phoenix_Evals_Utils - Utility functions that may be decorated with trace()
• Arize_ai_Phoenix_LLM_Base_Types - Adapter methods that are candidates for tracing decoration