Principle:Langchain ai Langchain Standard Unit Testing

Overview

A standardized test suite that validates chat model implementations against a comprehensive set of unit tests without requiring network access.

Description

The Standard Unit Testing principle defines how LangChain partner integrations use a shared, reusable test suite to verify that their chat model implementations conform to the framework's expectations. Rather than each integration writing its own set of boilerplate tests, LangChain provides the ChatModelUnitTests base class in the langchain-tests package that includes 30+ pre-built test methods.

How it works:

1. The integration author creates a test class that inherits from ChatModelUnitTests.
2. The author implements two required properties: chat_model_class (returns the model class) and chat_model_params (returns initialization parameters).
3. The inherited test methods automatically exercise the model against a comprehensive suite of checks.

What the standard tests validate:

• Initialization: test_init verifies the model can be instantiated with the given parameters.
• Serialization: test_serdes tests that the model can be serialized and deserialized.
• Standard parameters: test_standard_params checks that standard LLM parameters (temperature, max_tokens, etc.) are handled correctly.
• Tool calling: test_bind_tool_pydantic, test_tool_message_content_is_str verify tool binding works with Pydantic models.
• Structured output: test_with_structured_output validates the structured output interface.
• Secret handling: test_init_from_env checks environment variable-based initialization.
• Declarative methods: Tests for configurable_fields, configurable_alternatives, etc.

Key constraint: All unit tests run with network sockets disabled (--disable-socket pytest flag). This ensures tests are fast, deterministic, and can run in any CI environment without API credentials.

Feature flags: Test subclasses can control which tests run by overriding boolean properties such as has_tool_calling, has_tool_choice, has_structured_output, and supports_json_mode. Tests for unsupported features are automatically skipped.

Usage

Apply this principle when:

• Creating a new LangChain chat model integration and writing its unit test suite.
• Validating that a model implementation meets the minimum LangChain interface requirements.
• Debugging why a model fails standard framework operations (tool calling, structured output, etc.).

Theoretical Basis

The pattern follows the Abstract Test Case design: a base test class defines a battery of tests, and concrete subclasses only need to provide the system-under-test. This ensures consistent quality across all integrations while minimizing per-integration test maintenance.

from langchain_tests.unit_tests import ChatModelUnitTests
from my_package.chat_models import ChatMyProvider


class TestChatMyProviderUnit(ChatModelUnitTests):
    @property
    def chat_model_class(self) -> type[ChatMyProvider]:
        return ChatMyProvider

    @property
    def chat_model_params(self) -> dict:
        return {"model": "model-001", "api_key": "test_key"}

Running with:

uv run --group test pytest --disable-socket --allow-unix-socket tests/unit_tests/

Related Pages

• Implementation:Langchain_ai_Langchain_ChatModelUnitTests_Subclass