Workflow:BerriAI Litellm Observability Integration

Knowledge Sources	LiteLLM LiteLLM Docs
Domains	LLM_Ops, Observability, Monitoring
Last Updated	2026-02-15 16:00 GMT

Overview

End-to-end process for integrating observability, logging, and monitoring into LLM API calls using LiteLLM's callback system.

Description

This workflow covers the setup and use of LiteLLM's callback and logging system to capture telemetry data from every LLM API call. The system supports 40+ external integrations (Langfuse, Datadog, Prometheus, OpenTelemetry, Weights & Biases, etc.) and a custom logger base class for building proprietary integrations. Every call generates a StandardLoggingPayload containing request details, response data, usage statistics, cost, latency, and error information.

Key outputs:

• Structured logs for every LLM API call with request/response details
• Real-time metrics (latency, tokens, cost) exported to monitoring systems
• Error tracking and alerting for failed LLM calls
• Audit trail for compliance and debugging

Usage

Execute this workflow when you need visibility into LLM API usage for debugging, cost tracking, performance monitoring, or compliance. This is essential for production deployments where understanding call patterns, tracking costs, and detecting failures is required.

Execution Steps

Step 1: Integration Selection

Choose which observability integrations to enable based on your monitoring stack. LiteLLM supports built-in integrations by string name (e.g., "langfuse", "datadog", "prometheus", "otel", "s3", "wandb") and custom logger classes for proprietary systems. Multiple integrations can be active simultaneously.

Key considerations:

• Built-in integrations are referenced by string names in callback lists
• Custom loggers extend the CustomLogger base class
• Batch loggers extend CustomBatchLogger for buffered sending
• Each integration may require its own API key or connection configuration

Step 2: Callback Registration

Register callbacks by adding them to litellm.success_callback and litellm.failure_callback lists (SDK mode) or in the proxy config YAML under litellm_settings.success_callback. Success callbacks fire after every successful LLM call; failure callbacks fire on errors.

Key considerations:

• Success and failure callbacks are separate lists and can contain different integrations
• Callbacks are invoked asynchronously to minimize latency impact on the main request
• The proxy config supports callback-specific environment variable configuration
• Dynamic callbacks can be added per-request via metadata

Step 3: Logging Payload Construction

On every LLM call, the logging system constructs a StandardLoggingPayload containing the model, provider, messages (optionally redacted), response, token usage, cost, latency breakdown (queue time, API time, total time), and metadata. This payload is passed to all registered callbacks.

What happens:

• Pre-call hooks fire before the API request with request details
• Post-call hooks fire after the response with full payload
• Failure hooks fire on exceptions with error details
• The payload includes both raw data and computed fields (cost, tokens)

Step 4: Integration Data Export

Each registered callback processes the logging payload and exports data to its target system. Batch loggers accumulate payloads in memory and flush periodically. Direct loggers send data immediately. The logging worker handles async dispatch to prevent blocking the main request.

Key considerations:

• Batch loggers use configurable flush intervals and batch sizes
• OpenTelemetry exports spans with LLM-specific attributes
• Prometheus exports counters and histograms for metrics dashboards
• Langfuse and similar tools create trace hierarchies for LLM chains

Step 5: Custom Logger Development

For proprietary monitoring systems, create a custom logger by extending the CustomLogger base class and implementing the callback methods: log_success_event(), log_failure_event(), async_log_success_event(), and async_log_failure_event(). Register the custom logger instance in the callback lists.

Key considerations:

• The CustomLogger base class provides no-op defaults for all methods
• Both sync and async variants are supported; async is preferred for non-blocking operation
• Custom loggers can access the full StandardLoggingPayload and response objects
• Guardrail loggers extend CustomGuardrail for pre-call content filtering

Step 6: Message Redaction and Privacy

Configure message redaction to strip sensitive content from logs before it reaches external systems. LiteLLM supports global redaction (replacing all message content with placeholder text) and per-callback redaction settings. This is critical for compliance with data privacy regulations.

Key considerations:

• litellm.turn_off_message_logging = True enables global redaction
• Per-callback redaction can be configured in proxy settings
• Redaction replaces message content while preserving metadata (tokens, cost, latency)
• Custom redaction logic can be implemented in custom loggers

GitHub URL

Workflow Repository