Implementation:BerriAI Litellm Cache Endpoints

Knowledge Sources	https://github.com/BerriAI/litellm
Domains	Caching, Operations, API Design, LLM Infrastructure
Last Updated	2026-02-15

Overview

Concrete HTTP endpoints for cache lifecycle management provided by the LiteLLM proxy's caching_routes FastAPI router.

Description

The litellm/proxy/caching_routes.py module defines a FastAPI APIRouter mounted at the /cache prefix with four endpoints for managing the LiteLLM proxy's response cache at runtime:

• GET /cache/ping: Checks whether the cache is initialized and healthy. For Redis caches, it issues a PING command and a test set operation. Returns a CachePingResponse Pydantic model with status, cache type, ping result, and masked configuration parameters. For non-Redis caches, it confirms initialization and returns the cache type.

• POST /cache/delete: Accepts a JSON body with a keys list and deletes those keys from the Redis cache. Only supported for Redis backends. The cache keys are exposed in every proxy response via the x-litellm-cache-key header.

• GET /cache/redis/info: Returns Redis server information and client connection details. Uses a helper function _get_redis_client_info that gracefully handles the case where CLIENT LIST is restricted on managed Redis instances.

• POST /cache/flushall: Flushes all entries from the Redis cache. Only supported for Redis backends. This is a destructive operation.

All endpoints are protected by the user_api_key_auth dependency. A helper function _extract_cache_params safely extracts and masks cache configuration parameters using HealthCheckCacheParams and SensitiveDataMasker.

Usage

These endpoints are available automatically when the LiteLLM proxy is running with caching enabled. Interact with them via HTTP requests using the proxy's base URL and an authorized API key.

Code Reference

Source Location	litellm/proxy/caching_routes.py, lines 16-257
Router Definition	router = APIRouter(prefix="/cache", tags=["caching"])
Endpoints	GET /cache/ping, POST /cache/delete, GET /cache/redis/info, POST /cache/flushall
Import	from litellm.proxy.caching_routes import router

Router setup:

from fastapi import APIRouter, Depends, HTTPException, Request

router = APIRouter(
    prefix="/cache",
    tags=["caching"],
)

Key endpoint signatures:

@router.get("/ping", response_model=CachePingResponse, dependencies=[Depends(user_api_key_auth)])
async def cache_ping(): ...

@router.post("/delete", tags=["caching"], dependencies=[Depends(user_api_key_auth)])
async def cache_delete(request: Request): ...

@router.get("/redis/info", dependencies=[Depends(user_api_key_auth)])
async def cache_redis_info(): ...

@router.post("/flushall", tags=["caching"], dependencies=[Depends(user_api_key_auth)])
async def cache_flushall(): ...

Helper functions:

def _extract_cache_params() -> Dict[str, Any]:
    """Safely extracts and cleans cache parameters with sensitive data masking."""
    if litellm.cache is None:
        return {}
    cache_params = vars(litellm.cache.cache)
    cleaned_params = HealthCheckCacheParams(**cache_params).model_dump() if cache_params else {}
    return masker.mask_dict(cleaned_params)

def _get_redis_client_info(cache_instance) -> Tuple[List, int]:
    """Safely get Redis client list; returns -1 for num_clients if CLIENT LIST is unavailable."""
    try:
        client_list = cache_instance.client_list()
        return client_list, len(client_list)
    except Exception:
        return ["CLIENT LIST command not available on this Redis instance"], -1

I/O Contract

GET /cache/ping

Inputs:

Parameter	Type	Description
(none)	--	No request body required. Authentication via Authorization: Bearer <api-key> header.

Outputs:

Field	Type	Description
status	str	"healthy" on success
cache_type	str	The cache backend type (e.g., "redis", "local")
ping_response	Optional[bool]	True if Redis PING succeeded (Redis only)
set_cache_response	Optional[str]	"success" if test write succeeded (Redis only)
litellm_cache_params	Optional[str]	JSON string of masked top-level cache parameters
health_check_cache_params	Optional[dict]	Masked backend connection parameters (host, port, namespace, etc.)

POST /cache/delete

Inputs:

Parameter	Type	Description
keys	List[str]	A list of cache keys to delete. Provided in JSON request body: {"keys": ["key1", "key2"]}

Outputs:

Field	Type	Description
status	str	"success" on successful deletion

GET /cache/redis/info

Inputs:

Parameter	Type	Description
(none)	--	No request body required. Authentication via header.

Outputs:

Field	Type	Description
num_clients	int	Number of connected Redis clients (-1 if CLIENT LIST is unavailable)
clients	List	List of client connection details, or a descriptive message if unavailable
info	dict	Raw Redis INFO command output (memory, server version, keyspace stats, etc.)

POST /cache/flushall

Inputs:

Parameter	Type	Description
(none)	--	No request body required. Authentication via header.

Outputs:

Field	Type	Description
status	str	"success" on successful flush

Usage Examples

Checking cache health:

curl -X GET "http://0.0.0.0:4000/cache/ping" \
  -H "Authorization: Bearer sk-1234"

Example response (Redis):

{
  "status": "healthy",
  "cache_type": "redis",
  "ping_response": true,
  "set_cache_response": "success",
  "litellm_cache_params": "{\"type\": \"redis\", \"namespace\": \"prod\", ...}",
  "health_check_cache_params": {
    "host": "re****.example.com",
    "port": 6379,
    "namespace": "prod"
  }
}

Deleting specific cache keys:

curl -X POST "http://0.0.0.0:4000/cache/delete" \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"keys": ["a3f2b8c9d4e5f6a7...", "b7e23ec29af22b0b..."]}'

Inspecting Redis server info:

curl -X GET "http://0.0.0.0:4000/cache/redis/info" \
  -H "Authorization: Bearer sk-1234"

Flushing all cached entries:

curl -X POST "http://0.0.0.0:4000/cache/flushall" \
  -H "Authorization: Bearer sk-1234"

Example response:

{
  "status": "success"
}

Related Pages

• Principle:BerriAI_Litellm_Cache_Management