Implementation:BerriAI Litellm Service Types

Attribute	Value
Sources	litellm/types/services.py
Domains	Service Monitoring, Metrics, Observability, Health Checks
Last Updated	2026-02-15 16:00 GMT

Overview

Type definitions for LiteLLM's service monitoring system, defining service types, metric categories, configuration mappings, and the logging payload structure for tracking service health.

Description

This module provides the type hierarchy for LiteLLM's internal service monitoring and observability system. It defines how the proxy tracks the health, latency, and error rates of internal services and adjacent infrastructure. Key types include:

• ServiceMetrics -- Enum of metric types: COUNTER, HISTOGRAM, GAUGE.
• ServiceTypes -- Enum of all monitored services including core infrastructure (REDIS, DB, LITELLM, ROUTER, AUTH, PROXY_PRE_CALL) and operational queues (spend update queues for daily spend tracking across keys, users, teams, agents, tags, and organizations).
• ServiceConfig -- TypedDict specifying which metrics a service supports.
• DEFAULT_SERVICE_CONFIGS -- A pre-built dictionary mapping each ServiceType to its supported metrics (Counter+Histogram for core services, Gauge for operational queues and lock managers).
• ServiceEventMetadata -- TypedDict for dynamic gauge labels and values attached to service events.
• ServiceLoggerPayload -- Pydantic model encapsulating a service event with error status, duration, service type, call type, and metadata.

Usage

Import from this module when:

• Implementing custom service monitoring hooks (service_success_hook, service_failure_hook).
• Extending the service monitoring system with new service types.
• Building dashboards or alerting systems that consume service metrics.
• Working with the proxy's internal health check and observability features.

Code Reference

Source Location

litellm/types/services.py (138 lines)

Key Types

Type Name	Kind	Description
ServiceMetrics	Enum	Metric types: COUNTER, HISTOGRAM, GAUGE
ServiceTypes	str, Enum	All monitored service identifiers
ServiceConfig	TypedDict	Metric configuration for a service
DEFAULT_SERVICE_CONFIGS	Dict	Default metric configs per service type
ServiceEventMetadata	TypedDict	Dynamic gauge labels and values for events
ServiceLoggerPayload	BaseModel	Full service event logging payload

ServiceTypes Members

Member	Value	Metrics	Description
REDIS	"redis"	Counter, Histogram	Redis cache service
DB	"postgres"	Counter, Histogram	PostgreSQL database
BATCH_WRITE_TO_DB	"batch_write_to_db"	Counter, Histogram	Batch database write operations
RESET_BUDGET_JOB	"reset_budget_job"	Counter, Histogram	Budget reset cron job
LITELLM	"self"	Counter, Histogram	LiteLLM core service
ROUTER	"router"	Counter, Histogram	Router/load balancer
AUTH	"auth"	Counter, Histogram	Authentication service
PROXY_PRE_CALL	"proxy_pre_call"	Counter, Histogram	Proxy pre-call processing
POD_LOCK_MANAGER	"pod_lock_manager"	Gauge	Distributed pod lock manager
IN_MEMORY_DAILY_SPEND_UPDATE_QUEUE	"in_memory_daily_spend_update_queue"	Gauge	In-memory daily spend queue
REDIS_DAILY_SPEND_UPDATE_QUEUE	"redis_daily_spend_update_queue"	Gauge	Redis daily spend queue
REDIS_DAILY_END_USER_SPEND_UPDATE_QUEUE	"redis_daily_end_user_spend_update_queue"	Gauge	Redis end-user daily spend queue
REDIS_DAILY_ORG_SPEND_UPDATE_QUEUE	"redis_daily_org_spend_update_queue"	Gauge	Redis org daily spend queue
REDIS_DAILY_TEAM_SPEND_UPDATE_QUEUE	"redis_daily_team_spend_update_queue"	Gauge	Redis team daily spend queue
REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE	"redis_daily_agent_spend_update_queue"	Gauge	Redis agent daily spend queue
REDIS_DAILY_TAG_SPEND_UPDATE_QUEUE	"redis_daily_tag_spend_update_queue"	Gauge	Redis tag daily spend queue
IN_MEMORY_SPEND_UPDATE_QUEUE	"in_memory_spend_update_queue"	Gauge	In-memory current spend queue
REDIS_SPEND_UPDATE_QUEUE	"redis_spend_update_queue"	Gauge	Redis current spend queue

Signature: ServiceLoggerPayload

class ServiceLoggerPayload(BaseModel):
    is_error: bool = Field(description="did an error occur")
    error: Optional[str] = Field(None, description="what was the error")
    service: ServiceTypes = Field(description="who is this for? - postgres/redis")
    duration: float = Field(description="How long did the request take?")
    call_type: str = Field(description="The call of the service, being made")
    event_metadata: Optional[dict] = Field(
        description="The metadata logged during service success/failure"
    )

Import

from litellm.types.services import (
    ServiceMetrics,
    ServiceTypes,
    ServiceConfig,
    DEFAULT_SERVICE_CONFIGS,
    ServiceEventMetadata,
    ServiceLoggerPayload,
)

I/O Contract

ServiceLoggerPayload (Service Event)

Field	Type	Default	Description
is_error	bool	(required)	Whether an error occurred
error	Optional[str]	None	Error message if is_error is True
service	ServiceTypes	(required)	The service that generated this event
duration	float	(required)	Duration of the service call in seconds
call_type	str	(required)	The specific operation performed
event_metadata	Optional[dict]	None	Additional metadata (gauge_labels, gauge_value)

ServiceEventMetadata

Field	Type	Description
gauge_labels	Optional[str]	Labels for gauge metric
gauge_value	Optional[float]	Value for gauge metric

Usage Examples

Logging a successful service call

from litellm.types.services import ServiceLoggerPayload, ServiceTypes

payload = ServiceLoggerPayload(
    is_error=False,
    error=None,
    service=ServiceTypes.REDIS,
    duration=0.015,
    call_type="get_cache",
    event_metadata=None,
)

Logging a failed service call

from litellm.types.services import ServiceLoggerPayload, ServiceTypes

payload = ServiceLoggerPayload(
    is_error=True,
    error="Connection refused",
    service=ServiceTypes.DB,
    duration=5.0,
    call_type="batch_write",
    event_metadata=None,
)

Using gauge metrics for queue monitoring

from litellm.types.services import ServiceLoggerPayload, ServiceTypes, ServiceEventMetadata

metadata: ServiceEventMetadata = {
    "gauge_labels": "redis_daily_spend_queue_size",
    "gauge_value": 42.0,
}

payload = ServiceLoggerPayload(
    is_error=False,
    service=ServiceTypes.REDIS_DAILY_SPEND_UPDATE_QUEUE,
    duration=0.0,
    call_type="queue_size_check",
    event_metadata=metadata,
)

Checking default metric config for a service

from litellm.types.services import DEFAULT_SERVICE_CONFIGS, ServiceTypes, ServiceMetrics

config = DEFAULT_SERVICE_CONFIGS[ServiceTypes.REDIS.value]
# {'metrics': [ServiceMetrics.COUNTER, ServiceMetrics.HISTOGRAM]}

supports_gauge = ServiceMetrics.GAUGE in config["metrics"]
# False -- REDIS uses Counter and Histogram, not Gauge

Related Pages

• Guardrail Types -- Guardrail logging integrates with the service monitoring system.
• Proxy Server -- The proxy server that initializes and manages service monitoring.
• Tag Management Types -- Tag-based spend tracking feeds into service queue metrics.