Implementation:Confident ai Deepeval ConversationSimulator
| Sources | Domains | Last Updated |
|---|---|---|
| DeepEval | Synthetic_Data, LLM_Evaluation | 2026-02-14 09:00 GMT |
Overview
The ConversationSimulator class simulates multi-turn conversations between a synthetic user (played by an LLM) and a chatbot, producing ConversationalTestCase data for evaluating multi-turn interactions.
Description
ConversationSimulator takes a model_callback function (the chatbot under test) and an optional simulator_model (the LLM playing the user role). It orchestrates multi-turn dialogue by alternating between the simulator model generating user messages and the chatbot callback producing responses. The resulting conversations are structured as ConversationalTestCase objects with ordered Turn entries. The class supports configurable concurrency, language selection, and optional remote execution on the Confident AI platform.
Usage
Instantiate with a chatbot callback function, then call the simulation method to generate conversational test cases.
Code Reference
Source Location: Repository: confident-ai/deepeval, File: deepeval/simulator/conversation_simulator.py (L34-65)
Signature:
class ConversationSimulator:
def __init__(
self,
model_callback: Callable[[str], str],
simulator_model: Optional[Union[str, DeepEvalBaseLLM]] = None,
max_concurrent: int = 5,
async_mode: bool = True,
language: str = "English",
run_remote: bool = False,
):
...
Import:
from deepeval.simulator import ConversationSimulator
I/O Contract
Inputs:
| Parameter | Type | Required | Description |
|---|---|---|---|
| model_callback | Callable[[str], str] | Yes | Chatbot function that takes a user message string and returns a response string |
| simulator_model | Optional[Union[str, DeepEvalBaseLLM]] | No | LLM that plays the user role in simulated conversations |
| max_concurrent | int | No | Maximum number of parallel conversation simulations (default: 5) |
| async_mode | bool | No | Enable asynchronous simulation execution (default: True) |
| language | str | No | Language for generated conversations (default: "English") |
| run_remote | bool | No | Run simulation on Confident AI cloud platform (default: False) |
Outputs:
- ConversationSimulator instance -- configured simulator that produces List[ConversationalTestCase] when its simulation method is invoked, with each test case containing multi-turn Turn objects (role, content, tool calls)
Usage Examples
from deepeval.simulator import ConversationSimulator
def my_chatbot(message: str) -> str:
return llm.invoke(message)
simulator = ConversationSimulator(model_callback=my_chatbot)