Implementation:BerriAI Litellm Prometheus Services

Attribute	Value
Sources	`litellm/integrations/prometheus_services.py`
Domains	Monitoring, Prometheus, Metrics, Service Health, Integrations
Last Updated	2026-02-15 16:00 GMT

Overview

The PrometheusServicesLogger monitors the health of LiteLLM's internal and adjacent services (Redis, PostgreSQL, LLM API providers) by exposing Prometheus metrics on the /metrics endpoint.

Description

PrometheusServicesLogger creates and manages Prometheus Histogram, Counter, and Gauge metrics for each defined ServiceType (e.g., Redis, PostgreSQL, LLM providers). On initialization, it iterates over all ServiceTypes and creates the configured metrics (histograms for latency, counters for total/failed requests, gauges for size). The class provides sync and async hooks for success and failure events via ServiceLoggerPayload objects. On success, it observes latency histograms, increments total request counters, and optionally updates gauges. On failure, it increments both failed and total request counters, with additional labels for error class and function name to aid debugging. Metric names follow the pattern litellm_{service}_{type}. The class checks the Prometheus registry to avoid duplicate metric registration and supports a mock testing mode.

Usage

Import and instantiate PrometheusServicesLogger when running the LiteLLM proxy server with Prometheus monitoring. Requires the prometheus_client package.

Code Reference

Source Location

litellm/integrations/prometheus_services.py

Signature

class PrometheusServicesLogger:
    def __init__(
        self,
        mock_testing: bool = False,
        **kwargs,
    )

Import

from litellm.integrations.prometheus_services import PrometheusServicesLogger

I/O Contract

Inputs

Parameter	Type	Required	Description
`mock_testing`	`bool`	No	Enable mock testing mode (counts calls without Prometheus). Default `False`.
`kwargs`	`dict`	No	Additional keyword arguments (currently unused).

Key Methods

Method	Returns	Description
`service_success_hook(payload: ServiceLoggerPayload)`	`None`	Sync: observes latency histogram and increments total requests counter.
`service_failure_hook(payload: ServiceLoggerPayload)`	`None`	Sync: increments both failed and total request counters.
`async_service_success_hook(payload: ServiceLoggerPayload)`	`None`	Async: observes latency, increments total, optionally updates gauge.
`async_service_failure_hook(payload: ServiceLoggerPayload, error)`	`None`	Async: increments failed/total counters with error class and function name labels.
`create_histogram(service, type_of_request)`	`Histogram`	Creates or retrieves a Prometheus Histogram for a service.
`create_counter(service, type_of_request, additional_labels=None)`	`Counter`	Creates or retrieves a Prometheus Counter.
`create_gauge(service, type_of_request)`	`Gauge`	Creates or retrieves a Prometheus Gauge.

Outputs

Output	Type	Description
Side effect	Prometheus metrics	Updates Prometheus Histogram, Counter, and Gauge metrics in the default registry.

Metric naming pattern:

litellm_{service}_latency (Histogram)
litellm_{service}_total_requests (Counter)
litellm_{service}_failed_requests (Counter, with labels: error_class, function_name)
litellm_{service}_size (Gauge)

Usage Examples

from litellm.integrations.prometheus_services import PrometheusServicesLogger
from litellm.types.services import ServiceLoggerPayload, ServiceTypes

logger = PrometheusServicesLogger()

# Log a successful Redis operation
payload = ServiceLoggerPayload(
    service=ServiceTypes.REDIS,
    duration=0.025,
    call_type="get",
)
logger.service_success_hook(payload)

# Async failure logging
payload = ServiceLoggerPayload(
    service=ServiceTypes.POSTGRES,
    duration=1.5,
    call_type="insert_data",
)
await logger.async_service_failure_hook(
    payload=payload,
    error=ConnectionError("Connection refused"),
)
# Increments litellm_postgres_failed_requests with labels:
#   error_class="ConnectionError", function_name="insert_data"

Related Pages

BerriAI_Litellm_OpenMeter_Logger - another metering integration
BerriAI_Litellm_PostHog_Logger - another analytics integration

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment