Implementation:BerriAI Litellm Humanloop Integration

Attribute	Value
Sources	`litellm/integrations/humanloop.py`
Domains	Prompt Management, Integrations, Template Compilation
Last Updated	2026-02-15 16:00 GMT

Overview

The HumanloopLogger and HumanLoopPromptManager provide integration with Humanloop for fetching, caching, and compiling prompt templates in LLM completion calls.

Description

This module contains two main classes. HumanLoopPromptManager extends DualCache to fetch prompt templates from the Humanloop API (/v5/prompts/{prompt_id}), cache them locally with a configurable TTL (litellm.HUMANLOOP_PROMPT_CACHE_TTL_SECONDS), compile templates by substituting variables (using Template:Variable syntax), and extract model and optional parameters from the template. HumanloopLogger extends CustomLogger and implements get_chat_completion_prompt() to intercept completion calls, fetch the prompt template via the prompt manager, compile it with provided variables, and return the updated model, messages, and parameters. A module-level singleton prompt_manager instance is shared across all logger instances.

Usage

Import and register HumanloopLogger when using Humanloop for prompt management. Requires a HUMANLOOP_API_KEY environment variable or dynamic callback parameter, and a prompt_id in the completion call.

Code Reference

Source Location

litellm/integrations/humanloop.py

Signature

class HumanLoopPromptManager(DualCache):
    @property
    def integration_name(self) -> str: ...
    def _get_prompt_from_id(self, humanloop_prompt_id: str, humanloop_api_key: str) -> PromptManagementClient: ...
    def compile_prompt(self, prompt_template: List[AllMessageValues], prompt_variables: Optional[dict]) -> List[AllMessageValues]: ...

class HumanloopLogger(CustomLogger):
    def get_chat_completion_prompt(
        self,
        model: str,
        messages: List[AllMessageValues],
        non_default_params: dict,
        prompt_id: Optional[str],
        prompt_variables: Optional[dict],
        dynamic_callback_params: StandardCallbackDynamicParams,
        prompt_spec: Optional[PromptSpec] = None,
        prompt_label: Optional[str] = None,
        prompt_version: Optional[int] = None,
        ignore_prompt_manager_model: Optional[bool] = False,
        ignore_prompt_manager_optional_params: Optional[bool] = False,
    ) -> Tuple[str, List[AllMessageValues], dict]

Import

from litellm.integrations.humanloop import HumanloopLogger, HumanLoopPromptManager

I/O Contract

Inputs

Parameter	Type	Required	Description
`model`	`str`	Yes	Model name. May be overridden by the Humanloop template model.
`messages`	`List[AllMessageValues]`	Yes	Original messages (replaced by compiled template).
`non_default_params`	`dict`	Yes	Non-default completion parameters.
`prompt_id`	`Optional[str]`	Yes	Humanloop prompt ID to fetch.
`prompt_variables`	`Optional[dict]`	No	Variables to substitute into the template.
`dynamic_callback_params`	`StandardCallbackDynamicParams`	Yes	May contain `humanloop_api_key`.

Key Methods

Method	Returns	Description
`HumanloopLogger.get_chat_completion_prompt(...)`	`Tuple[str, List, dict]`	Fetches template, compiles it, and returns updated model, messages, and params.
`HumanLoopPromptManager._get_prompt_from_id(prompt_id, api_key)`	`PromptManagementClient`	Fetches prompt from cache or API.
`HumanLoopPromptManager.compile_prompt(template, variables)`	`List[AllMessageValues]`	Substitutes variables into the template messages.

Outputs

Output	Type	Description
Return tuple	`Tuple[str, List[AllMessageValues], dict]`	Returns `(model, compiled_messages, merged_params)`.

Usage Examples

import litellm

# Register Humanloop as a callback
litellm.success_callback = ["humanloop"]

response = litellm.completion(
    model="humanloop/gpt-4",
    messages=[],  # Will be replaced by the Humanloop template
    prompt_id="pr_abc123",
    prompt_variables={"name": "Alice", "topic": "Python"},
    metadata={"humanloop_api_key": "hl_api_key_here"},
)

# Direct use of the prompt manager
from litellm.integrations.humanloop import prompt_manager

template = prompt_manager._get_prompt_from_id(
    humanloop_prompt_id="pr_abc123",
    humanloop_api_key="hl_api_key_here",
)

compiled = prompt_manager.compile_prompt(
    prompt_template=template["prompt_template"],
    prompt_variables={"name": "Bob"},
)

Related Pages

BerriAI_Litellm_Anthropic_Cache_Control_Hook - another prompt management hook
BerriAI_Litellm_Dual_Cache - the base class for HumanLoopPromptManager

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment