Principle:BerriAI Litellm Integration Selection
| Knowledge Sources | Domains | Last Updated |
|---|---|---|
| [[1]] | Observability, Integration Management | 2026-02-15 |
Overview
Integration selection is the process of choosing and configuring which observability platforms receive telemetry data from an LLM gateway, guided by a declarative registry of supported backends.
Description
When an application proxies requests to many LLM providers, operators need visibility into cost, latency, error rates, and content. Rather than hard-coding every monitoring platform, a well-designed system maintains a registry -- a structured catalog of all supported observability integrations -- along with the metadata needed to configure each one at runtime.
The registry approach solves several problems:
- Discoverability -- Operators can programmatically query which integrations are available and what credentials each one requires.
- Uniform configuration -- Every integration entry follows the same schema (identifier, display name, required parameters, optional parameters), so the UI, CLI, and API layer can all render configuration forms without special-casing.
- Key/team-scoped logging -- Some integrations support per-API-key or per-team credential overrides, enabling multi-tenant logging where different teams route telemetry to different backends.
- Parameter typing -- Each dynamic parameter carries a type (password, text, number) so configuration interfaces can present appropriate input widgets and validation.
Usage
Apply integration selection when:
- Setting up a new LLM proxy deployment and deciding which observability platforms to enable.
- Building an admin UI that needs to enumerate available logging backends and their configuration requirements.
- Implementing multi-tenant logging where different teams or API keys route to different monitoring platforms.
- Adding a new observability backend to the system and needing a standardized way to register it.
Theoretical Basis
Registry Pattern
The integration registry follows the Service Locator pattern combined with a declarative configuration schema. Each entry in the registry is a self-describing record:
REGISTRY_ENTRY := {
id: unique string identifier
displayName: human-readable label
logo: visual asset reference
supports_key_team_logging: boolean -- can credentials differ per key/team?
dynamic_params: map of PARAM_NAME -> PARAM_DESCRIPTOR
description: freeform text
}
PARAM_DESCRIPTOR := {
type: "password" | "text" | "number"
ui_name: label for display
description: help text
required: boolean
}
Selection Algorithm (Pseudocode)
function select_integration(registry, integration_id, user_params):
entry = registry.find(id == integration_id)
if entry is None:
raise UnknownIntegrationError
for param_name, descriptor in entry.dynamic_params:
if descriptor.required and param_name not in user_params:
raise MissingParameterError(param_name)
if descriptor.type == "password":
validate_secret(user_params[param_name])
if entry.supports_key_team_logging:
allow per-key credential overrides
return instantiate_handler(entry, user_params)
Multi-Tenant Logging
When supports_key_team_logging is true, the system allows different API keys or teams to supply their own credentials for the same integration type. This means a single proxy deployment can simultaneously send:
- Team A's logs to Langfuse instance X
- Team B's logs to Langfuse instance Y
The credential resolution follows a priority chain: per-request dynamic params, then per-key settings, then per-team settings, then global environment variables.