Implementation:BerriAI Litellm Lazy Imports

Attribute	Value
Sources	litellm/_lazy_imports.py
Domains	Performance, Module Loading, Import System
last_updated	2026-02-15 16:00 GMT

Overview

The Lazy Imports module implements a deferred attribute resolution system for the litellm package, ensuring that heavy dependencies are only loaded when first accessed rather than at import time.

Description

This module is the runtime engine of LiteLLM's lazy loading infrastructure. When a user accesses an attribute on the litellm namespace (e.g., litellm.ModelResponse), Python's __getattr__ mechanism in __init__.py delegates to this module. A global registry maps every lazy-loadable attribute name to a category-specific handler function. Each handler calls the generic _generic_lazy_import workhorse, which:

1. Looks up the module path and attribute name from a per-category import map (defined in _lazy_imports_registry.py).
2. Imports the target module via importlib.import_module.
3. Caches the resolved value in the litellm module's global dictionary so subsequent accesses are free.

Special handlers exist for HTTP client singletons (module_level_aclient, module_level_client), the LLM client cache singleton, and utils module attributes (which cache into litellm.utils.__dict__ instead of litellm.__dict__). Three standalone lazy loaders (_get_default_encoding, _get_modified_max_tokens, _get_token_counter_new) avoid importing tiktoken and token_counter at startup.

Usage

This module is used internally by the litellm.__init__ module. Consumers should never import from litellm._lazy_imports directly; instead they should access attributes through the top-level litellm namespace. The lazy system is transparent to callers.

Code Reference

Source Location

/litellm/_lazy_imports.py (439 lines)

Key Functions

Function	Signature	Purpose
_get_lazy_import_registry	def _get_lazy_import_registry() -> dict[str, Callable[[str], Any]]	Builds and caches the name-to-handler mapping on first access
_generic_lazy_import	def _generic_lazy_import(name: str, import_map: dict[str, tuple[str, str]], category: str) -> Any	Core workhorse that resolves, imports, caches, and returns an attribute
_get_default_encoding	def _get_default_encoding() -> Any	Lazily loads the default tiktoken encoding
_get_modified_max_tokens	def _get_modified_max_tokens() -> Any	Lazily loads the get_modified_max_tokens function
_get_token_counter_new	def _get_token_counter_new() -> Any	Lazily loads the token_counter function
_lazy_import_http_handlers	def _lazy_import_http_handlers(name: str) -> Any	Creates HTTP client instances on demand
_lazy_import_llm_client_cache	def _lazy_import_llm_client_cache(name: str) -> Any	Handles class and singleton instance for LLM client cache

Import

# Not intended for direct import. Accessed via:
import litellm
litellm.ModelResponse  # triggers lazy import internally

I/O Contract

Inputs

Parameter	Type	Description
name	str	The attribute name being accessed (e.g., "ModelResponse")
import_map	dict[str, tuple[str, str]]	Mapping of attribute names to (module_path, attr_name) tuples
category	str	Human-readable category label for error messages

Outputs

Return	Type	Description
Resolved attribute	Any	The lazily imported class, function, or object

Raises AttributeError if the name is not found in the registry.

Usage Examples

# Transparent usage -- the lazy import system works behind the scenes
import litellm

# First access triggers the import of litellm.utils -> ModelResponse
response = litellm.ModelResponse(id="test", model="gpt-4")

# Subsequent accesses use the cached value (no re-import)
response2 = litellm.ModelResponse(id="test2", model="gpt-4")

# HTTP client singletons are created on demand
client = litellm.module_level_client  # creates HTTPHandler instance

# Cache class and singleton
cache_class = litellm.LLMClientCache
cache_instance = litellm.in_memory_llm_clients_cache

Related Pages

• BerriAI_Litellm_Lazy_Imports_Registry - defines the name tuples and import maps consumed by this module
• BerriAI_Litellm_Logging_Setup - one of the modules lazily imported through this system
• BerriAI_Litellm_Constants - provides configuration values used by lazily created HTTP clients