Implementation:BerriAI Litellm Anthropic Cache Control Hook

Attribute	Value
Sources	litellm/integrations/anthropic_cache_control_hook.py
Domains	Prompt Management, Anthropic, Cache Control, Message Transformation
Last Updated	2026-02-15 16:00 GMT

Overview

The AnthropicCacheControlHook is a prompt management hook that injects Anthropic cache control directives (cache_control) into chat completion messages at user-specified injection points.

Description

AnthropicCacheControlHook extends CustomPromptManagement to process cache_control_injection_points from completion parameters. Each injection point specifies a target message by index or role and a cache control directive (defaulting to {"type": "ephemeral"}). The hook creates a deep copy of the messages, then injects cache control directives into the targeted messages. For string content, it adds cache_control at the message level; for list content (multiple content blocks), it adds it to the last content block per Anthropic's API specification. The hook supports negative indices, role-based targeting, and both sync and async interfaces. It integrates with LiteLLM's prompt management system but is not a true prompt management client -- it only modifies messages in transit.

Usage

This hook is automatically activated when cache_control_injection_points is present in the completion parameters. It can also be used via the static helper method should_use_anthropic_cache_control_hook().

Code Reference

Source Location

litellm/integrations/anthropic_cache_control_hook.py

Signature

class AnthropicCacheControlHook(CustomPromptManagement):
    def get_chat_completion_prompt(
        self,
        model: str,
        messages: List[AllMessageValues],
        non_default_params: dict,
        prompt_id: Optional[str],
        prompt_variables: Optional[dict],
        dynamic_callback_params: StandardCallbackDynamicParams,
        prompt_spec: Optional[PromptSpec] = None,
        prompt_label: Optional[str] = None,
        prompt_version: Optional[int] = None,
        ignore_prompt_manager_model: Optional[bool] = False,
        ignore_prompt_manager_optional_params: Optional[bool] = False,
    ) -> Tuple[str, List[AllMessageValues], dict]

Import

from litellm.integrations.anthropic_cache_control_hook import AnthropicCacheControlHook

I/O Contract

Inputs

Parameter	Type	Required	Description
model	str	Yes	The model name.
messages	List[AllMessageValues]	Yes	Chat completion messages.
non_default_params	dict	Yes	Completion params, must contain cache_control_injection_points.
prompt_id	Optional[str]	No	Not used by this hook.
prompt_variables	Optional[dict]	No	Not used by this hook.
dynamic_callback_params	StandardCallbackDynamicParams	Yes	Dynamic callback parameters.

Key Methods

Method	Returns	Description
get_chat_completion_prompt(...)	Tuple[str, List, dict]	Injects cache control directives into messages (sync).
async_get_chat_completion_prompt(...)	Tuple[str, List, dict]	Async version -- delegates to sync.
should_use_anthropic_cache_control_hook(non_default_params)	bool	Static check if hook should be activated.
get_custom_logger_for_anthropic_cache_control_hook(non_default_params)	Optional[CustomLogger]	Static factory that returns the hook if applicable.

Outputs

Output	Type	Description
Return tuple	Tuple[str, List[AllMessageValues], dict]	Returns (model, modified_messages, params_without_injection_points).

Usage Examples

import litellm

# Use cache_control_injection_points in the completion call
response = litellm.completion(
    model="claude-3-opus-20240229",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
    cache_control_injection_points=[
        {
            "location": "message",
            "index": 0,  # Target the system message
            "control": {"type": "ephemeral"},
        },
        {
            "location": "message",
            "role": "user",  # Target all user messages
            "control": {"type": "ephemeral"},
        },
    ],
)

# Negative index targeting (last message)
response = litellm.completion(
    model="claude-3-opus-20240229",
    messages=[
        {"role": "user", "content": "First message"},
        {"role": "user", "content": "Second message"},
    ],
    cache_control_injection_points=[
        {"location": "message", "index": -1, "control": {"type": "ephemeral"}},
    ],
)

Related Pages

• BerriAI_Litellm_Humanloop_Integration - another prompt management integration