Implementation:Langchain ai Langchain ChatModelUnitTests

Knowledge Sources	Langchain_ai_Langchain
Domains	Testing, Chat Models, Standard Tests
Last Updated	2026-02-11 00:00 GMT

Overview

Standard unit test suite for validating LangChain chat model implementations, covering initialization, serialization, tool binding, structured output, and standard parameter compliance.

Description

ChatModelUnitTests is an abstract test suite class in the langchain-tests (standard-tests) package that extends ChatModelTests. It provides a comprehensive set of unit tests (no network calls) for BaseChatModel implementations. The suite verifies model initialization, environment variable loading, streaming mode support, Pydantic tool binding, structured output generation, standard LangSmith parameter compliance, serialization/deserialization, and initialization benchmarks. Test subclasses configure what features to test by overriding boolean properties (tool calling, structured output, image/audio/video inputs, JSON mode, etc.).

Usage

Import ChatModelUnitTests when writing unit tests for a custom chat model integration. Subclass it and implement the required chat_model_class and chat_model_params properties.

Code Reference

Source Location

• Repository: Langchain_ai_Langchain
• File: libs/standard-tests/langchain_tests/unit_tests/chat_models.py
• Lines: 1-1148

Signature

class ChatModelTests(BaseStandardTests):
    @property
    @abstractmethod
    def chat_model_class(self) -> type[BaseChatModel]: ...

    @property
    def chat_model_params(self) -> dict[str, Any]: ...

class ChatModelUnitTests(ChatModelTests):
    @property
    def standard_chat_model_params(self) -> dict[str, Any]: ...

    @property
    def init_from_env_params(
        self,
    ) -> tuple[dict[str, str], dict[str, Any], dict[str, Any]]: ...

Import

from langchain_tests.unit_tests.chat_models import ChatModelUnitTests

I/O Contract

Required Properties

Name	Type	Required	Description
chat_model_class	type[BaseChatModel]	Yes	The chat model class to test (e.g., ChatParrotLink).
chat_model_params	dict[str, Any]	Yes	Initialization parameters for the chat model.

Configurable Feature Properties

Name	Type	Default	Description
has_tool_calling	bool	Auto-detected	Whether the model supports tool calling. Auto-detected from bind_tools override.
has_tool_choice	bool	Auto-detected	Whether the model supports tool_choice parameter. Auto-detected from bind_tools signature.
has_structured_output	bool	Auto-detected	Whether the model supports structured output. Auto-detected from with_structured_output or bind_tools overrides.
structured_output_kwargs	dict[str, Any]	{}	Additional kwargs passed to with_structured_output() in tests.
supports_json_mode	bool	False	Whether the model supports method="json_mode" in with_structured_output.
supports_image_inputs	bool	False	Whether the model supports image inputs.
supports_image_urls	bool	False	Whether the model supports image inputs from URLs.
supports_pdf_inputs	bool	False	Whether the model supports PDF inputs.
supports_audio_inputs	bool	False	Whether the model supports audio inputs.
supports_video_inputs	bool	False	Whether the model supports video inputs.
returns_usage_metadata	bool	True	Whether the model returns usage metadata on responses.
supports_anthropic_inputs	bool	False	Whether the model supports Anthropic-style input content blocks.
supports_image_tool_message	bool	False	Whether the model supports ToolMessage with image content.
supports_pdf_tool_message	bool	False	Whether the model supports ToolMessage with PDF content.
supports_model_override	bool	True	Whether the model accepts a model kwarg at runtime.
model_override_value	str or None	None	Alternative model name for testing model override.
enable_vcr_tests	bool	False	Whether to enable VCR-cached HTTP tests.
init_from_env_params	tuple[dict, dict, dict]	({}, {}, {})	Environment variables, init args, and expected attributes for env init testing.

Outputs

Name	Type	Description
Test results	pytest outcomes	Pass/fail results for each standard test method.

Test Methods

Test Method	Description
test_init	Tests model initialization with standard and custom parameters.
test_init_from_env	Tests initialization from environment variables. Skipped if init_from_env_params not set.
test_init_streaming	Tests model can be initialized with streaming=True.
test_bind_tool_pydantic	Tests bind_tools with Pydantic models, functions, and JSON schemas. Skipped if has_tool_calling is False.
test_with_structured_output	Tests with_structured_output with Pydantic models using json_schema, function_calling, and json_mode methods. Skipped if has_structured_output is False.
test_standard_params	Tests that _get_ls_params() returns valid LangSmith parameters (ls_provider, ls_model_name, ls_model_type, etc.).
test_serdes	Tests serialization and deserialization. Skipped if model is not LangChain-serializable.
test_init_time	Benchmark test measuring model initialization time (10 iterations).

Usage Examples

Basic Usage

from typing import Type

from langchain_tests.unit_tests.chat_models import ChatModelUnitTests
from my_package.chat_models import MyChatModel


class TestMyChatModelUnit(ChatModelUnitTests):
    @property
    def chat_model_class(self) -> Type[MyChatModel]:
        return MyChatModel

    @property
    def chat_model_params(self) -> dict:
        return {"model": "model-001", "temperature": 0}

With Environment Variable Testing

from langchain_tests.unit_tests.chat_models import ChatModelUnitTests
from my_package.chat_models import MyChatModel


class TestMyChatModelUnit(ChatModelUnitTests):
    @property
    def chat_model_class(self):
        return MyChatModel

    @property
    def chat_model_params(self) -> dict:
        return {"model": "model-001"}

    @property
    def init_from_env_params(self):
        return (
            {"MY_API_KEY": "api_key"},       # env vars to set
            {"model": "model-001"},           # init args
            {"my_api_key": "api_key"},        # expected attributes
        )

With Custom Structured Output Method

from langchain_tests.unit_tests.chat_models import ChatModelUnitTests
from my_package.chat_models import MyChatModel


class TestMyChatModelUnit(ChatModelUnitTests):
    @property
    def chat_model_class(self):
        return MyChatModel

    @property
    def chat_model_params(self) -> dict:
        return {"model": "model-001"}

    @property
    def structured_output_kwargs(self) -> dict:
        return {"method": "json_schema"}

Standard Chat Model Parameters

The ChatModelUnitTests class provides default standard parameters used for all model initializations in tests:

{
    "temperature": 0,
    "max_tokens": 100,
    "timeout": 60,
    "stop": [],
    "max_retries": 2,
    "api_key": "test",  # Added by ChatModelUnitTests
}

These are merged with chat_model_params when creating model instances in the model fixture.