Implementation:BerriAI Litellm Constants

Attribute	Value
Sources	litellm/constants.py
Domains	Configuration, Defaults, Provider Settings, Infrastructure
last_updated	2026-02-15 16:00 GMT

Overview

The Constants module is the central repository of default values, thresholds, provider lists, model sets, and configuration constants used throughout the LiteLLM library and proxy server.

Description

This module defines over 300 constants organized into the following categories. Most numeric constants are configurable via environment variables with sensible defaults.

Infrastructure and networking:

• request_timeout (6000s), REDIS_SOCKET_TIMEOUT (0.1s), REDIS_CONNECTION_POOL_TIMEOUT (5s)
• AIOHTTP_CONNECTOR_LIMIT (300), AIOHTTP_KEEPALIVE_TIMEOUT (120s)
• DEFAULT_SSL_CIPHERS -- TLS cipher priority list for fast handshakes
• REALTIME_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES
• _DEFAULT_TTL_FOR_HTTPX_CLIENTS (3600s)

Retry and reliability:

• DEFAULT_MAX_RETRIES (2), INITIAL_RETRY_DELAY (0.5s), MAX_RETRY_DELAY (8s), JITTER (0.75)
• DEFAULT_COOLDOWN_TIME_SECONDS (5), DEFAULT_FAILURE_THRESHOLD_PERCENT (0.5)
• REPEATED_STREAMING_CHUNK_LIMIT (100), ROUTER_MAX_FALLBACKS (5)

Token counting:

• DEFAULT_MAX_TOKENS (4096), DEFAULT_IMAGE_TOKEN_COUNT (250)
• FUNCTION_DEFINITION_TOKEN_COUNT (9), SYSTEM_MESSAGE_TOKEN_COUNT (4)

Spend tracking:

• DEFAULT_REPLICATE_GPU_PRICE_PER_SECOND, OPENAI_FILE_SEARCH_COST_PER_1K_CALLS
• FIREWORKS_AI_* and TOGETHER_AI_* parameter tier thresholds

Logging and callbacks:

• LOGGING_WORKER_CONCURRENCY (100), LOGGING_WORKER_MAX_QUEUE_SIZE (50000)
• MAX_CALLBACKS (100), MAX_LANGFUSE_INITIALIZED_CLIENTS (50)

Provider and model lists:

• LITELLM_CHAT_PROVIDERS -- 70+ supported chat providers
• openai_compatible_providers, openai_compatible_endpoints
• BEDROCK_CONVERSE_MODELS, clarifai_models, nebius_models, WANDB_MODELS, etc.
• OPENAI_CHAT_COMPLETION_PARAMS, OPENAI_FINISH_REASONS

Proxy server:

• DEFAULT_SOFT_BUDGET (50.0), MAX_SPENDLOG_ROWS_TO_QUERY (1M)
• PROXY_BATCH_WRITE_AT (10s), APScheduler tuning constants
• SENTRY_DENYLIST, SENTRY_PII_DENYLIST -- lists of sensitive field names to scrub

Reasoning effort budgets:

• DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET for disable, minimal, low, medium, high
• Gemini model-specific minimal budgets

Usage

Import individual constants as needed:

from litellm.constants import DEFAULT_MAX_RETRIES, REDIS_SOCKET_TIMEOUT

Code Reference

Source Location

/litellm/constants.py (1462 lines)

Selected Constants

Constant	Default	Env Var	Description
DEFAULT_MAX_RETRIES	2	DEFAULT_MAX_RETRIES	Maximum automatic retries for failed requests
DEFAULT_MAX_TOKENS	4096	DEFAULT_MAX_TOKENS	Default max tokens when not specified
REDIS_SOCKET_TIMEOUT	0.1	REDIS_SOCKET_TIMEOUT	Redis socket timeout in seconds
LOGGING_WORKER_CONCURRENCY	100	LOGGING_WORKER_CONCURRENCY	Max concurrent logging coroutines
LOGGING_WORKER_MAX_QUEUE_SIZE	50000	LOGGING_WORKER_MAX_QUEUE_SIZE	Max logging queue depth
MAX_CALLBACKS	100	LITELLM_MAX_CALLBACKS	Max registered callbacks to prevent CPU overload
DEFAULT_COOLDOWN_TIME_SECONDS	5	DEFAULT_COOLDOWN_TIME_SECONDS	Cooldown period after deployment failures
request_timeout	6000.0	REQUEST_TIMEOUT	Global request timeout in seconds

Import

from litellm.constants import (
    DEFAULT_MAX_RETRIES,
    LITELLM_CHAT_PROVIDERS,
    OPENAI_CHAT_COMPLETION_PARAMS,
    REDIS_SOCKET_TIMEOUT,
    LOGGING_WORKER_CONCURRENCY,
)

I/O Contract

This module exports only constant values. It has no callable functions. All numeric constants are read from environment variables at import time with fallback defaults.

Outputs

Type	Count	Description
int / float constants	~200	Numeric thresholds and defaults
str constants	~30	String identifiers and prefixes
List[str]	~15	Provider lists, parameter lists, endpoint lists
set	~10	Model name sets for specific providers
dict	~5	Tokenizer configs, default parameter value maps

Usage Examples

from litellm.constants import (
    DEFAULT_MAX_RETRIES,
    DEFAULT_COOLDOWN_TIME_SECONDS,
    LITELLM_CHAT_PROVIDERS,
    OPENAI_CHAT_COMPLETION_PARAMS,
)

# Use in retry logic
for attempt in range(DEFAULT_MAX_RETRIES + 1):
    try:
        response = make_request()
        break
    except Exception:
        if attempt == DEFAULT_MAX_RETRIES:
            raise

# Check if a provider supports chat
if "anthropic" in LITELLM_CHAT_PROVIDERS:
    print("Anthropic is a supported chat provider")

# Validate parameters
unsupported = [p for p in user_params if p not in OPENAI_CHAT_COMPLETION_PARAMS]

Related Pages

• Implementation:BerriAI_Litellm_Redis_Client - uses REDIS_SOCKET_TIMEOUT and REDIS_CONNECTION_POOL_TIMEOUT
• Implementation:BerriAI_Litellm_Logging_Worker - uses LOGGING_WORKER_* constants
• Implementation:BerriAI_Litellm_Duration_Parser - uses time-unit constants like HOURS_IN_A_DAY, DAYS_IN_A_MONTH
• Implementation:BerriAI_Litellm_Get_Supported_Openai_Params - references provider lists from this module