Implementation:Evidentlyai Evidently LLM Models

Knowledge Sources	Evidentlyai_Evidently
Domains	LLM, Data Models
Last Updated	2026-02-14 12:00 GMT

Overview

Defines core data models for LLM (Large Language Model) interactions, including the LLMMessage class for conversation messages and the LLMResponse type alias.

Description

The LLM Models module provides foundational types used across the Evidently LLM evaluation and monitoring subsystem:

• LLMMessage -- A frozen (immutable) Pydantic model representing a single message in an LLM conversation. Each message has a role (e.g., "user", "system", "assistant") and content (the text of the message). The class provides two factory class methods: user(message) and system(message) for convenient construction of common message types. It also supports validation from tuples (role, content) and dictionaries via the __get_validators__ / validate_from_tuple class methods, enabling flexible input parsing when constructing messages from configuration or serialized data.

• LLMResponse -- A type alias defined as Dict[str, Any], representing the raw response dictionary returned by an LLM provider.

LLMMessage inherits from FrozenBaseModel, making instances hashable and immutable, which is important for caching and deduplication in evaluation pipelines.

Usage

Use LLMMessage when constructing prompts or conversation histories for LLM-based evaluations in Evidently. Use LLMResponse as the type annotation for raw LLM response dictionaries. These types are used internally by Evidently's LLM descriptors and evaluation metrics.

Code Reference

Source Location

• Repository: Evidentlyai_Evidently
• File: src/evidently/llm/models.py

Signature

class LLMMessage(FrozenBaseModel):
    role: str
    content: str

    @classmethod
    def user(cls, message: str) -> "LLMMessage":
        ...

    @classmethod
    def system(cls, message: str) -> "LLMMessage":
        ...

    @classmethod
    def validate_from_tuple(cls, value: Any) -> "LLMMessage":
        ...

LLMResponse = Dict[str, Any]

Import

from evidently.llm.models import LLMMessage, LLMResponse

I/O Contract

Inputs

Name	Type	Required	Description
role	str	Yes	The role of the message sender (e.g., "user", "system", "assistant")
content	str	Yes	The text content of the message
message	str	Yes (for factory methods)	The message text passed to user() or system() factory methods
value	Any	Yes (for validate_from_tuple)	A tuple (role, content), a dict with role/content keys, or an LLMMessage instance

Outputs

Name	Type	Description
LLMMessage instance	LLMMessage	An immutable message object with role and content fields
LLMResponse	Dict[str, Any]	Type alias for raw LLM response dictionaries

Usage Examples

from evidently.llm.models import LLMMessage, LLMResponse

# Create messages using factory methods
system_msg = LLMMessage.system("You are a helpful assistant.")
user_msg = LLMMessage.user("Evaluate this text for sentiment.")

# Create message directly
msg = LLMMessage(role="assistant", content="The sentiment is positive.")

# Validate from tuple
msg_from_tuple = LLMMessage.validate_from_tuple(("user", "Hello!"))

# Validate from dict
msg_from_dict = LLMMessage.validate_from_tuple({"role": "system", "content": "Analyze this."})

# Type annotation for LLM responses
response: LLMResponse = {"choices": [{"message": {"content": "result"}}]}

Related Pages

• Environment:Evidentlyai_Evidently_Python_Core_Environment