Implementation:Confident ai Deepeval Error Hierarchy

Knowledge Sources	Confident_ai_Deepeval
Domains	Error_Handling, Evaluation
Last Updated	2026-02-14 09:30 GMT

Overview

Custom exception hierarchy that distinguishes framework-originated errors from user application errors, with specialized subclasses for test case validation failures.

Description

The Error Hierarchy (`deepeval/errors.py`) defines four exception classes that enable the framework to handle errors with appropriate granularity:

• DeepEvalError (extends `Exception`): Base class for framework-originated errors. When raised and not handled, it aborts the current operation. Instances can also be stringified and attached to traces or spans as non-fatal diagnostics.
• UserAppError (extends `Exception` independently): Represents exceptions thrown by user LLM applications or tools. These are recorded on traces or spans but allow the overall evaluation run to continue, enabling "ignore errors" mode.
• MissingTestCaseParamsError (extends `DeepEvalError`): Raised when required test case fields are absent.
• MismatchedTestCaseInputsError (extends `DeepEvalError`): Raised when inputs provided to a metric or test case are inconsistent or invalid.

The separation between `DeepEvalError` and `UserAppError` is critical for the evaluation engine's error boundary logic, where user code failures are recorded but do not terminate the evaluation run.

Usage

Import these exception classes when you need to raise framework errors, catch specific error types in evaluation pipelines, or distinguish between framework bugs and user application failures in error handling logic.

Code Reference

Source Location

• Repository: Confident_ai_Deepeval
• File: deepeval/errors.py
• Lines: 1-24

Signature

class DeepEvalError(Exception):
    """Base class for framework-originated errors.
    If raised and not handled, it will abort the current operation.
    We may also stringify instances of this class and attach them to traces or spans to surface
    non-fatal diagnostics while allowing the run to continue.
    """


class UserAppError(Exception):
    """Represents exceptions thrown by user LLM apps/tools.
    We record these on traces or spans and keep the overall evaluation run alive.
    """


class MissingTestCaseParamsError(DeepEvalError):
    """Required test case fields are missing."""
    pass


class MismatchedTestCaseInputsError(DeepEvalError):
    """Inputs provided to a metric or test case are inconsistent or invalid."""
    pass

Import

from deepeval.errors import DeepEvalError, UserAppError
from deepeval.errors import MissingTestCaseParamsError, MismatchedTestCaseInputsError

I/O Contract

Inputs

Name	Type	Required	Description
message	str	No	Exception message string (standard Python exception constructor)

Outputs

Name	Type	Description
DeepEvalError	Exception	Framework error that aborts the current operation or surfaces as a non-fatal diagnostic
UserAppError	Exception	User application error that is recorded but does not terminate the evaluation run
MissingTestCaseParamsError	DeepEvalError	Indicates required test case fields are missing
MismatchedTestCaseInputsError	DeepEvalError	Indicates inconsistent or invalid metric/test case inputs

Usage Examples

Raising Framework Errors

from deepeval.errors import MissingTestCaseParamsError, MismatchedTestCaseInputsError

# Validate required test case fields
def validate_test_case(test_case):
    if test_case.input is None:
        raise MissingTestCaseParamsError(
            "Test case is missing required 'input' field."
        )
    if test_case.actual_output is None and test_case.expected_output is None:
        raise MismatchedTestCaseInputsError(
            "Test case must have either 'actual_output' or 'expected_output'."
        )

Catching Errors in Evaluation Pipeline

from deepeval.errors import DeepEvalError, UserAppError

try:
    result = run_evaluation(test_case, metric)
except UserAppError as e:
    # User's LLM app threw an error -- record it but continue
    trace.record_error(str(e))
except DeepEvalError as e:
    # Framework error -- abort this test case
    raise

Related Pages

• Environment:Confident_ai_Deepeval_Python_3_9_Runtime