Jump to content

Connect SuperML | Leeroopedia MCP: Equip your AI agents with best practices, code verification, and debugging knowledge. Powered by Leeroo — building Organizational Superintelligence. Contact us at founders@leeroo.com.

Implementation:Openai Openai python Eval List Response

From Leeroopedia
Knowledge Sources
Domains API_Types, Python
Last Updated 2026-02-15 00:00 GMT

Overview

Concrete type for evaluation list response objects provided by the openai-python SDK.

Description

The EvalListResponse Pydantic model represents an individual Eval object returned when listing evaluations. It has the same structure as EvalCreateResponse and EvalRetrieveResponse: an id, created_at timestamp, data_source_config (discriminated union of custom, logs, or stored_completions configs), optional metadata, a name, object type "eval", and testing_criteria list of grader variants. An Eval represents a task for your LLM integration, such as improving chatbot quality or comparing model performance.

Usage

Import this type when iterating over results from client.evals.list() to type-hint individual eval items.

Code Reference

Source Location

Signature

class EvalListResponse(BaseModel):
    id: str
    created_at: int
    data_source_config: DataSourceConfig
    metadata: Optional[Metadata] = None
    name: str
    object: Literal["eval"]
    testing_criteria: List[TestingCriterion]

DataSourceConfig = Annotated[
    Union[EvalCustomDataSourceConfig, DataSourceConfigLogs, EvalStoredCompletionsDataSourceConfig],
    PropertyInfo(discriminator="type"),
]

TestingCriterion = Union[
    LabelModelGrader, StringCheckGrader,
    TestingCriterionEvalGraderTextSimilarity,
    TestingCriterionEvalGraderPython,
    TestingCriterionEvalGraderScoreModel,
]

Import

from openai.types import EvalListResponse

I/O Contract

Fields

Name Type Required Description
id str Yes Unique identifier for the evaluation
created_at int Yes Unix timestamp (seconds) when the eval was created
data_source_config DataSourceConfig Yes Configuration of data sources (custom, logs, or stored_completions)
metadata Optional[Metadata] No Up to 16 key-value pairs for additional information
name str Yes Name of the evaluation
object Literal["eval"] Yes Object type, always "eval"
testing_criteria List[TestingCriterion] Yes List of graders for the evaluation

Usage Examples

from openai import OpenAI

client = OpenAI()

for eval_obj in client.evals.list(limit=20, order="desc"):
    print(f"{eval_obj.id}: {eval_obj.name}")
    print(f"  Data source: {eval_obj.data_source_config.type}")
    print(f"  Criteria count: {len(eval_obj.testing_criteria)}")

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment