Implementation:BerriAI Litellm Custom Prompt Management Template

Attribute	Value
Sources	litellm/proxy/custom_prompt_management.py
Domains	Proxy, Prompt Management, Extensibility, Templates
Last Updated	2026-02-15 16:00 GMT

Overview

custom_prompt_management is an example template demonstrating how to implement a custom prompt management integration for the LiteLLM proxy, showing how to intercept and modify chat completion prompts at runtime.

Description

This module provides a reference implementation of custom prompt management by defining the X42PromptManagement class, which extends CustomPromptManagement. The class overrides get_chat_completion_prompt to intercept completion calls and optionally modify the model, messages, and non-default parameters based on a prompt ID and prompt variables. In this template, the method is a pass-through that returns the original values unchanged, but developers can add custom logic to fetch prompt templates from external systems, inject system prompts, or modify parameters. The integration_name property returns "x42-prompt-management". A module-level singleton instance x42_prompt_management is created for direct use.

Usage

Copy and modify this template when you need to integrate a custom prompt management system with the LiteLLM proxy. Configure it in your proxy's callback list so it intercepts completion calls and applies prompt templates.

Code Reference

Source Location

litellm/proxy/custom_prompt_management.py

Signature

class X42PromptManagement(CustomPromptManagement):
    def get_chat_completion_prompt(
        self, model: str, messages: List[AllMessageValues], non_default_params: dict,
        prompt_id: Optional[str], prompt_variables: Optional[dict],
        dynamic_callback_params: StandardCallbackDynamicParams,
        prompt_spec: Optional[PromptSpec] = None, prompt_label: Optional[str] = None,
        prompt_version: Optional[int] = None, ignore_prompt_manager_model: Optional[bool] = False,
        ignore_prompt_manager_optional_params: Optional[bool] = False,
    ) -> Tuple[str, List[AllMessageValues], dict]

    @property
    def integration_name(self) -> str

x42_prompt_management = X42PromptManagement()

Import

from litellm.proxy.custom_prompt_management import x42_prompt_management

I/O Contract

Inputs

Parameter	Type	Description
model	str	The model name for the completion call.
messages	List[AllMessageValues]	The chat messages from the client.
non_default_params	dict	Non-default parameters (temperature, max_tokens, etc.).
prompt_id	Optional[str]	The prompt template identifier.
prompt_variables	Optional[dict]	Variables to interpolate into the prompt template.
dynamic_callback_params	StandardCallbackDynamicParams	Dynamic callback parameters for the call.
prompt_spec	Optional[PromptSpec]	Prompt specification object.
prompt_label	Optional[str]	Label for the prompt version.
prompt_version	Optional[int]	Version number of the prompt.

Outputs

Return Type	Description
Tuple[str, List[AllMessageValues], dict]	A tuple of (model, messages, non_default_params), potentially modified by the prompt management logic.

Usage Examples

from litellm.integrations.custom_prompt_management import CustomPromptManagement
from litellm.types.llms.openai import AllMessageValues
from typing import List, Optional, Tuple

class MyPromptManagement(CustomPromptManagement):
    def get_chat_completion_prompt(
        self, model, messages, non_default_params, prompt_id,
        prompt_variables, dynamic_callback_params, **kwargs
    ) -> Tuple[str, List[AllMessageValues], dict]:
        # Inject a system prompt based on prompt_id
        if prompt_id == "customer-support":
            system_msg = {"role": "system", "content": "You are a helpful customer support agent."}
            messages = [system_msg] + messages
            non_default_params["temperature"] = 0.3
        return model, messages, non_default_params

    @property
    def integration_name(self) -> str:
        return "my-prompt-management"

Related Pages

• Implementation:BerriAI_Litellm_Prompt_Management_Base - The abstract base class for prompt management integrations
• Implementation:BerriAI_Litellm_Custom_Auth_Template - Custom authentication template for the proxy