Principle:OpenHands OpenHands LiteLLM Provisioning
| Knowledge Sources | |
|---|---|
| Domains | Organization_Management, Multi_Tenancy |
| Last Updated | 2026-02-11 21:00 GMT |
Overview
Provisioning LLM proxy resources (team and API key) for a new organizational entity ensures the organization can access language model services immediately upon creation.
Description
The LiteLLM Provisioning principle implements a resource provisioning pattern in which dependent external resources are created before the primary entity is persisted. During organization onboarding, the system must create a LiteLLM team (representing the organization's usage boundary and budget) and generate an API key bound to the creating user. These external resources are provisioned first so that:
- The organization entity can be constructed with its LiteLLM settings already populated.
- If the external service is unavailable or rejects the request, the workflow fails early without leaving orphaned database records.
- Budget limits and access controls are established atomically with the organization's identity.
This pattern is characteristic of systems that depend on external service integrations where the external state must exist before the local entity references it.
Usage
Apply this principle when an entity's creation depends on successfully provisioning resources in an external system. Common scenarios include:
- Creating LLM proxy teams and API keys during organization setup
- Provisioning cloud infrastructure resources before recording them in a local database
- Setting up third-party billing accounts before creating the corresponding local user or organization
Theoretical Basis
The resource provisioning pattern follows an allocate-then-create sequence:
# Pseudocode for resource provisioning before entity creation
def provision_external_resources(entity_id, user_id):
# Step 1: Create the external team/group
team = external_service.create_team(
team_alias=entity_id,
max_budget=default_budget
)
# Step 2: Generate credentials for the user within the team
api_key = external_service.generate_key(
user_id=user_id,
team_id=team.id,
key_alias=f"{entity_id}_key"
)
# Step 3: Return provisioned settings for local entity construction
return {"team_id": team.id, "api_key": api_key}
Key considerations:
- Ordering: External resources are provisioned before the local entity is created, ensuring no database row references nonexistent external state.
- Compensation: If the local entity creation subsequently fails, the provisioned external resources must be cleaned up (see the Compensating Transaction principle).
- Idempotency: The provisioning calls should be designed so that retries do not create duplicate teams or keys in the external system.