Principle:PrefectHQ Prefect Batched Flow Run Deletion
| Metadata | |
|---|---|
| Sources | Prefect |
| Domains | Orchestration, Database_Maintenance |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A pattern for safely deleting flow run records in rate-limited batches with error handling to prevent database overload during maintenance operations.
Description
Batched Flow Run Deletion implements safe bulk deletion of flow run records from the Prefect server. Instead of deleting all records at once (which could overload the database), records are processed in configurable batches with rate limiting (asyncio.sleep between deletions). Each deletion handles ObjectNotFound gracefully (concurrent cleanup may have already removed the run). This provides:
- Configurable batch sizes
- Rate limiting to prevent database overload
- Graceful error handling per record
- Accurate deletion counts
Usage
Use this pattern for database maintenance operations that need to delete large numbers of flow run records. Always pair with an approval gate (human or AI) and a preview step.
Theoretical Basis
Batched processing with rate limiting. Large bulk operations are split into fixed-size batches with delays between operations to prevent resource exhaustion. The Bulkhead pattern ensures individual failures don't cascade.