Principle:Promptfoo Promptfoo Data Migration
| Knowledge Sources | |
|---|---|
| Domains | Database, Migration, Caching |
| Last Updated | 2026-02-14 07:45 GMT |
Overview
Principle governing the safe migration of persistent data (database schemas and cache formats) across version upgrades while preserving data integrity.
Description
Data Migration defines how promptfoo handles schema evolution and data format changes across versions. This covers both (1) SQLite database schema migrations via Drizzle ORM (adding/modifying tables as the application evolves) and (2) cache format migrations (converting from legacy cache-manager v4 format to the new keyv-file format). The principle mandates that migrations are idempotent, that failed migrations are recoverable, and that temporary transition periods include sunset dates after which legacy support is dropped.
Usage
Apply this principle when making changes to the database schema (create a Drizzle migration) or when changing persistent storage formats (implement a migration with a sunset date).
Theoretical Basis
The migration approach follows two complementary patterns:
- Incremental Schema Migration (database): SQL migration files are applied sequentially, each tracked by a metadata table to ensure idempotency. Drizzle ORM generates migration files from schema diff.
- Format Migration with Sunset (cache): A one-time migration converts old format data to the new format, with a sunset date after which migration code is removed and users start fresh.
Both patterns ensure forward-only migration (no rollback support) under the assumption that new format is always preferred.