Principle:OpenHands OpenHands Compensating Transaction
| Knowledge Sources | |
|---|---|
| Domains | Organization_Management, Multi_Tenancy |
| Last Updated | 2026-02-11 21:00 GMT |
Overview
Persisting multi-resource state with compensation (rollback) on failure ensures eventual consistency across distributed resources when a local transaction cannot be completed.
Description
The Compensating Transaction principle implements a saga pattern for managing operations that span multiple systems — in this case, a local database and an external LiteLLM proxy service. When the onboarding workflow has provisioned external resources (LiteLLM team and API key) and constructed local entities (Org and OrgMember), the final step attempts to persist everything to the database in a single transaction. If this persistence fails, the system must compensate by cleaning up the already-provisioned external resources to avoid leaving orphaned state.
This pattern differs from traditional ACID transactions because the external LiteLLM service does not participate in the database's transaction boundary. Instead, the workflow uses a try/except structure:
- Try: Commit the Org and OrgMember entities to the database.
- Except: On any database error, invoke a cleanup routine that deletes the LiteLLM team and API key, then re-raise the original error.
Usage
Apply this principle whenever a workflow creates resources in multiple systems that cannot share a single transaction boundary. Common scenarios include:
- Persisting organization records after provisioning external LLM proxy resources
- Saving billing subscriptions after creating payment processor customers
- Any distributed operation where partial completion leaves the system in an inconsistent state
Theoretical Basis
The compensating transaction pattern follows a commit-or-compensate structure:
# Pseudocode for compensating transaction
def persist_with_compensation(local_entities, external_resource_ids):
try:
# Attempt to persist all local entities atomically
database.add_all(local_entities)
database.commit()
return local_entities
except DatabaseError as e:
# Database commit failed — compensate by cleaning up external resources
database.rollback()
cleanup_error = cleanup_external_resources(external_resource_ids)
if cleanup_error:
log.error("Compensation also failed", error=cleanup_error)
raise e
Key considerations:
- Best-effort compensation: The cleanup of external resources is best-effort. If the compensation itself fails, the error is logged but the original exception is still raised. A background reconciliation process may be needed.
- Idempotent cleanup: The cleanup routine should be idempotent so that retries do not cause additional errors if the resources were already partially deleted.
- Ordering: External resources are provisioned first and cleaned up last. Local entities are persisted last and rolled back first. This ordering minimizes the window of inconsistency.
- Logging: Both the original failure and any compensation failures should be logged with full context for debugging and manual intervention.