Implementation:Confident ai Deepeval TaskCompletionMetric

Metadata

Knowledge Sources	DeepEval
Domains	LLM_Evaluation AI_Agents
Last Updated	2026-02-14 09:00 GMT

Overview

Concrete evaluation metric class that measures whether an AI agent successfully completes its assigned task. The TaskCompletionMetric uses an LLM-as-judge approach to score the agent's output against the task description, producing a continuous score between 0 and 1.

Description

The TaskCompletionMetric evaluates end-to-end task success. It takes the agent's output (and optionally the full execution trace) and uses a judge LLM to assess whether the stated task was accomplished. The metric supports automatic task extraction from agent traces when the task parameter is not explicitly provided.

Key capabilities:

• LLM-as-judge evaluation -- uses a configurable model to assess task completion quality.
• Automatic task extraction -- when task is None, the metric attempts to extract the task description from the agent's execution trace.
• Configurable threshold -- the threshold parameter sets the minimum score for a passing evaluation.
• Reason generation -- when include_reason=True, the metric produces a human-readable explanation of the score.
• Async support -- supports asynchronous evaluation via async_mode.

Usage

Import and instantiate for agent evaluation:

from deepeval.metrics import TaskCompletionMetric

Code Reference

Source Location

• Repository: confident-ai/deepeval
• File: deepeval/metrics/task_completion/task_completion.py (lines 26--254)

Signature

class TaskCompletionMetric(BaseMetric):
    def __init__(
        self,
        threshold: float = 0.5,
        task: Optional[str] = None,
        model: Optional[str] = None,
        include_reason: bool = True,
        async_mode: bool = True,
        strict_mode: bool = False,
        verbose_mode: bool = False,
    ):
        ...

Import

from deepeval.metrics import TaskCompletionMetric

Parent Class

• BaseMetric

I/O Contract

Inputs (Constructor Parameters)

Input Contract

Name	Type	Default	Description
threshold	float	0.5	Minimum score (0--1) for the evaluation to pass.
task	Optional[str]	None	Task description to evaluate against. If None, automatically extracted from the agent trace.
model	Optional[str]	None	LLM model to use as the evaluation judge. Falls back to default if not specified.
include_reason	bool	True	Whether to generate a human-readable reason for the score.
async_mode	bool	True	Whether to run evaluation asynchronously.
strict_mode	bool	False	When enabled, scores are binarized to 0 or 1 based on the threshold.
verbose_mode	bool	False	When enabled, prints detailed evaluation information during execution.

Outputs

Output Contract

Name	Type	Description
score	float	A value between 0 and 1 indicating the degree of task completion.
reason	Optional[str]	Human-readable explanation of the score (when include_reason=True).
success	bool	Whether the score meets or exceeds the threshold.

Usage Examples

Example 1: Basic Task Completion Evaluation

Create a metric with a custom threshold and explicit task description.

from deepeval.metrics import TaskCompletionMetric

metric = TaskCompletionMetric(
    threshold=0.7,
    task="Answer the user's question about weather",
)

• The threshold=0.7 requires a 70% completion score to pass.
• The task parameter explicitly defines what the agent should accomplish.

Example 2: Automatic Task Extraction

Use with framework instrumentation where the task is extracted from the trace.

from deepeval.metrics import TaskCompletionMetric
from deepeval.integrations.langchain import CallbackHandler

metric = TaskCompletionMetric(threshold=0.5)
handler = CallbackHandler(metrics=[metric], name="my-agent")
agent.invoke({"input": "Book a flight to Paris"}, config={"callbacks": [handler]})

• The task parameter is omitted, so the metric extracts the task from the captured trace.

Related Pages

• Principle:Confident_ai_Deepeval_Task_Completion_Evaluation