Principle:BerriAI Litellm Database Setup
| Knowledge Sources | Domains | Last Updated |
|---|---|---|
| BerriAI/litellm repository | Database Migrations, Schema Management, Persistent Storage | 2026-02-15 |
Overview
Setting up persistent storage for API keys, spend tracking, and team management via schema migrations in an LLM proxy gateway.
Description
Database setup is the process of establishing and maintaining the persistent storage layer that underpins the operational features of an LLM proxy gateway. Without a database, the proxy server can only operate in stateless mode -- forwarding requests without tracking usage, enforcing budgets, or managing API keys. Introducing a database enables:
- API key management -- Generating, storing, validating, and revoking API keys with associated metadata (budget limits, team associations, expiration dates).
- Spend tracking -- Recording token usage and cost data per key, team, and user for billing and accountability.
- Team and organization management -- Defining organizational hierarchies with teams, members, and role-based access control.
- Model management -- Storing model deployment configurations in the database for dynamic updates without config file changes.
- Rate limiting state -- Persisting rate limit counters and budget reset timestamps.
The database schema defines tables for verification tokens (API keys), users, teams, organizations, spend logs, and configuration state. Schema migrations ensure that the database structure evolves safely across software versions, applying incremental changes without data loss.
Usage
Use database setup when:
- Deploying the proxy in production where API key authentication and spend tracking are required.
- Managing multiple teams or users with individual budget limits and model access controls.
- Requiring persistent audit trails of API usage and cost attribution.
- Running the proxy in a multi-instance (horizontally scaled) deployment where shared state is necessary.
- Upgrading the proxy to a new version that introduces schema changes.
Theoretical Basis
Database migration follows the incremental schema evolution pattern, where each migration is a versioned, ordered transformation applied to the database schema. The system tracks which migrations have been applied and ensures that each migration runs exactly once, in order.
FUNCTION setup_database(database_url, schema_definition):
-- Phase 1: Generate the Prisma client from schema
RUN_COMMAND("prisma generate")
IF exit_code != 0 THEN
LOG_ERROR("Client generation failed")
ABORT
-- Phase 2: Apply pending migrations
RUN_COMMAND("prisma migrate deploy")
IF exit_code != 0 THEN
LOG_ERROR("Migration deployment failed")
ABORT
-- Phase 3: Connect the client
client = NEW PrismaClient(database_url)
AWAIT client.connect()
RETURN client
Key concepts:
- Schema-first design -- The database schema is defined declaratively (e.g., in a Prisma schema file), and client code and migrations are generated from this definition.
- Forward-only migrations -- Production deployments use
migrate deploy, which applies pending migrations without creating new ones, ensuring deterministic schema state. - Client generation -- After schema changes, a client library is regenerated to provide type-safe database access matching the current schema.
- Idempotent connectivity -- The proxy server checks for database connectivity at startup and gracefully degrades to stateless mode if the database is unavailable.
Typical schema structure for an LLM proxy:
TABLE verification_tokens:
token STRING (PRIMARY KEY, hashed)
key_name STRING
key_alias STRING
team_id STRING (FOREIGN KEY -> teams)
user_id STRING
max_budget FLOAT
spend FLOAT
expires DATETIME
models LIST[STRING]
rpm_limit INTEGER
tpm_limit INTEGER
metadata JSON
created_at DATETIME
updated_at DATETIME
TABLE teams:
team_id STRING (PRIMARY KEY)
team_alias STRING
members LIST[MEMBER]
max_budget FLOAT
spend FLOAT
models LIST[STRING]
TABLE spend_logs:
request_id STRING (PRIMARY KEY)
api_key STRING
model STRING
tokens INTEGER
spend FLOAT
timestamp DATETIME