Implementation:Langchain ai Langchain BaseRateLimiter Acquire

Knowledge Sources	LangChain
Domains	Concurrency, API_Management
Last Updated	2026-02-11 00:00 GMT

Overview

Concrete tool for rate-limiting LLM API requests provided by langchain-core.

Description

The BaseRateLimiter is an abstract interface with an acquire() method. LangChain provides InMemoryRateLimiter as a built-in implementation using the token bucket algorithm. The rate limiter is checked within the invoke() method of BaseChatModel before the actual API call is made.

The rate limiter field is declared on BaseChatModel as: rate_limiter: BaseRateLimiter | None = Field(default=None, exclude=True)

Usage

Pass a rate limiter instance when initializing a chat model to automatically throttle API calls.

Code Reference

Source Location

• Repository: langchain
• File: libs/core/langchain_core/language_models/chat_models.py
• Lines: L296-297 (field declaration), L389-437 (rate limit check in invoke)

Signature

class BaseRateLimiter(ABC):
    @abstractmethod
    def acquire(self, *, blocking: bool = True) -> bool:
        """Acquire a permit. Blocks if blocking=True."""
        ...

    @abstractmethod
    async def aacquire(self, *, blocking: bool = True) -> bool:
        """Async version of acquire."""
        ...

Import

from langchain_core.rate_limiters import InMemoryRateLimiter

I/O Contract

Inputs

Name	Type	Required	Description
blocking	bool	No (default: True)	Whether to block until a permit is available

Outputs

Name	Type	Description
return	bool	True if permit was acquired, False if non-blocking and unavailable

Usage Examples

Configuring Rate Limiting

from langchain_core.rate_limiters import InMemoryRateLimiter
from langchain_openai import ChatOpenAI

# Allow 10 requests per second with burst capacity of 20
rate_limiter = InMemoryRateLimiter(
    requests_per_second=10,
    check_every_n_seconds=0.1,
    max_bucket_size=20,
)

llm = ChatOpenAI(
    model="gpt-4o-mini",
    rate_limiter=rate_limiter,
)

# Rate limiting is applied automatically on each invoke/stream call
for question in large_question_list:
    response = llm.invoke(question)

Related Pages

Implements Principle

• Principle:Langchain_ai_Langchain_Rate_Limiting