Implementation:BerriAI Litellm Prompt Management Base

Attribute	Value
Sources	litellm/integrations/prompt_management_base.py
Domains	Prompt Management, Integrations, Extensibility
Last Updated	2026-02-15 16:00 GMT

Overview

PromptManagementBase is an abstract base class that defines the interface for prompt management integrations, allowing external prompt management tools to compile, merge, and inject prompt templates into LiteLLM chat completion calls.

Description

The PromptManagementBase class provides the foundation for integrating third-party prompt management systems (such as Langfuse, Helicone, or custom systems) with LiteLLM. It defines abstract methods for compiling prompts (_compile_prompt_helper, async_compile_prompt_helper), determining whether prompt management should run (should_run_prompt_management), and an integration_name property. The class provides concrete methods for merging prompt templates with client messages, post-processing compiled prompts to extract models and optional parameters, and both synchronous and asynchronous workflows for getting chat completion prompts. The PromptManagementClient TypedDict defines the standard return structure containing the prompt template, model, optional parameters, and completed messages.

Usage

Import and subclass PromptManagementBase when building an integration with a prompt management service. Implement the required abstract methods to connect to your prompt management tool and compile prompts at runtime.

Code Reference

Source Location

litellm/integrations/prompt_management_base.py

Signature

class PromptManagementClient(TypedDict):
    prompt_id: Optional[str]
    prompt_template: List[AllMessageValues]
    prompt_template_model: Optional[str]
    prompt_template_optional_params: Optional[Dict[str, Any]]
    completed_messages: Optional[List[AllMessageValues]]

class PromptManagementBase(ABC):
    @property
    def integration_name(self) -> str  # abstract
    def should_run_prompt_management(self, prompt_id, prompt_spec, dynamic_callback_params) -> bool  # abstract
    def _compile_prompt_helper(self, prompt_id, prompt_spec, prompt_variables, dynamic_callback_params, prompt_label=None, prompt_version=None) -> PromptManagementClient  # abstract
    async def async_compile_prompt_helper(self, prompt_id, prompt_variables, dynamic_callback_params, prompt_spec=None, prompt_label=None, prompt_version=None) -> PromptManagementClient  # abstract
    def merge_messages(self, prompt_template, client_messages) -> List[AllMessageValues]
    def compile_prompt(self, prompt_id, prompt_variables, client_messages, dynamic_callback_params, ...) -> PromptManagementClient
    async def async_compile_prompt(self, prompt_id, prompt_variables, client_messages, dynamic_callback_params, ...) -> PromptManagementClient
    def get_chat_completion_prompt(self, model, messages, non_default_params, prompt_id, prompt_variables, ...) -> Tuple[str, List[AllMessageValues], dict]
    async def async_get_chat_completion_prompt(self, model, messages, non_default_params, prompt_id, prompt_variables, ...) -> Tuple[str, List[AllMessageValues], dict]

Import

from litellm.integrations.prompt_management_base import PromptManagementBase, PromptManagementClient

I/O Contract

Inputs

Parameter	Type	Description
prompt_id	Optional[str]	The identifier for the prompt template to retrieve.
prompt_variables	Optional[dict]	Variables to interpolate into the prompt template.
client_messages / messages	List[AllMessageValues]	The user-provided messages to merge with the prompt template.
dynamic_callback_params	StandardCallbackDynamicParams	Dynamic parameters passed through callbacks.
prompt_spec	Optional[PromptSpec]	Prompt specification object.
prompt_label	Optional[str]	Label for the prompt version.
prompt_version	Optional[int]	Version number of the prompt.
model	str	The model name for the completion call.
non_default_params	dict	Non-default parameters to potentially override with prompt template settings.
ignore_prompt_manager_model	Optional[bool]	If True, do not override the model from the prompt template.
ignore_prompt_manager_optional_params	Optional[bool]	If True, do not merge optional params from the prompt template.

Outputs

Method	Return Type	Description
compile_prompt / async_compile_prompt	PromptManagementClient	A TypedDict containing the prompt template, model, optional params, and completed messages.
get_chat_completion_prompt / async_get_chat_completion_prompt	Tuple[str, List[AllMessageValues], dict]	A tuple of (model, messages, non_default_params) ready for completion.

Usage Examples

from litellm.integrations.prompt_management_base import PromptManagementBase, PromptManagementClient
from litellm.types.utils import StandardCallbackDynamicParams

class MyPromptManager(PromptManagementBase):
    @property
    def integration_name(self) -> str:
        return "my-prompt-manager"

    def should_run_prompt_management(self, prompt_id, prompt_spec, dynamic_callback_params):
        return prompt_id is not None

    def _compile_prompt_helper(self, prompt_id, prompt_spec, prompt_variables,
                               dynamic_callback_params, prompt_label=None, prompt_version=None):
        # Fetch prompt template from your system
        return PromptManagementClient(
            prompt_id=prompt_id,
            prompt_template=[{"role": "system", "content": "You are a helpful assistant."}],
            prompt_template_model="gpt-4",
            prompt_template_optional_params={"temperature": 0.7},
            completed_messages=None,
        )

    async def async_compile_prompt_helper(self, prompt_id, prompt_variables,
                                           dynamic_callback_params, prompt_spec=None,
                                           prompt_label=None, prompt_version=None):
        return self._compile_prompt_helper(prompt_id, prompt_spec, prompt_variables,
                                           dynamic_callback_params, prompt_label, prompt_version)

Related Pages

• BerriAI_Litellm_Custom_Prompt_Management_Template - Concrete example template for custom prompt management
• BerriAI_Litellm_Custom_LLM_Handler - Custom LLM handler that may use prompt templates