Heuristic:Promptfoo Promptfoo Cache Configuration Tips
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Development_Workflow |
| Last Updated | 2026-02-14 08:00 GMT |
Overview
Cache management best practices: use `--no-cache` during development to ensure fresh results, default 14-day TTL for production, and awareness that latency assertions fail on cached results.
Description
Promptfoo caches LLM API responses to avoid redundant API calls during iterative development. The cache uses a disk-based JSON store at `~/.promptfoo/cache` with a 14-day TTL by default. While caching speeds up repeated evaluations, it can mask changes during development. Additionally, certain assertions (like latency measurement) are incompatible with cached results and will throw explicit errors.
The cache system also includes a migration layer for upgrading from older cache formats, with automatic backup creation and a sunset date for the old format.
Usage
Apply these tips when running promptfoo evaluations during development. Always use `--no-cache` when iterating on prompts or provider configurations. Let the cache work normally for regression testing and CI pipelines.
The Insight (Rule of Thumb)
- Action 1: Always use `--no-cache` during active development to ensure fresh results.
- Action 2: Keep the default 14-day TTL for CI/CD pipelines where prompt/config stability is expected.
- Action 3: Use `PROMPTFOO_CACHE_TYPE=memory` for ephemeral test environments.
- Action 4: Never use latency assertions without `--no-cache`.
- Value: Default TTL is 14 days (1,209,600 seconds). Configurable via `PROMPTFOO_CACHE_TTL`.
- Trade-off: Caching saves API costs but can mask prompt/config changes. Development speed vs correctness.
Reasoning
From `src/cache.ts:24-25`:
/** Default cache TTL: 14 days in seconds */
const DEFAULT_CACHE_TTL_SECONDS = 60 * 60 * 24 * 14;
Latency assertion incompatibility from `src/assertions/latency.ts:7-11`:
if (latencyMs === undefined) {
throw new Error(
'Latency assertion does not support cached results. Rerun the eval with --no-cache'
);
}
Cache configuration from `src/cache.ts:19-22`:
let enabled = getEnvBool('PROMPTFOO_CACHE_ENABLED', true);
const cacheType =
getEnvString('PROMPTFOO_CACHE_TYPE') ||
(getEnvString('NODE_ENV') === 'test' ? 'memory' : 'disk');
The 14-day TTL balances two concerns: long enough that regression tests don't needlessly re-call APIs, but short enough that model behavior changes are eventually captured. Cache invalidation by content hash means identical inputs always hit cache regardless of TTL.