Principle:Langfuse Langfuse Prompt Cache Invalidation

Knowledge Sources	Langfuse
Domains	Prompt Management, Caching, Distributed Systems
Last Updated	2026-02-14 00:00 GMT

Overview

Prompt Cache Invalidation is a Redis-based strategy that ensures prompt mutations are atomically reflected to all consumers by combining short-lived distributed locks with bulk key deletion, preventing stale reads during the critical window of a write operation.

Description

When prompts are served to SDKs and applications at high throughput, a caching layer (Redis) is essential to avoid hitting the database on every request. However, caching introduces the classic distributed systems challenge: when a prompt is created, updated, or has its labels moved, all cached entries for that prompt name must be invalidated so that subsequent reads see the fresh data.

The naive approach of deleting cache keys after the write creates a race condition: a concurrent read between the write and the delete could re-populate the cache with stale data. Langfuse solves this with a lock-invalidate-unlock protocol:

1. Lock: Before mutating data, acquire a short-lived lock (TTL: 30 seconds) scoped to the project ID. While the lock is held, all cache reads for that project are bypassed, forcing consumers to read from the database.
2. Invalidate: Delete all cached prompt entries for the project from Redis using a key index (a Redis set that tracks which cache keys exist for a given project).
3. Write: Perform the database mutation inside a transaction.
4. Unlock: Release the lock, allowing subsequent reads to repopulate the cache from the now-current database state.

This ensures there is no window where stale cached data can be served after a mutation.

Usage

Use Prompt Cache Invalidation whenever:

• A prompt version is created, labels are moved, or tags are updated.
• The system must guarantee that SDK consumers receive up-to-date prompt content after a mutation.
• High-throughput prompt retrieval requires caching, but correctness must not be sacrificed.
• Prompt composability (dependency graphs) means a single prompt change could affect cached resolved graphs across the entire project.

Theoretical Basis

Cache Architecture

The caching system uses three types of Redis keys:

Cache Key:       prompt:{projectId}:{promptName}:{version|label}
Key Index:       prompt_key_index:{projectId}
Lock Key:        LOCK:prompt:{projectId}

• Cache keys store the serialized PromptResult (including resolved dependency content) with a configurable TTL (LANGFUSE_CACHE_PROMPT_TTL_SECONDS).
• Key index is a Redis Set that tracks all cache keys for a given project. This enables efficient bulk deletion without scanning the keyspace.
• Lock keys use the LOCK: prefix to ensure they are never accidentally deleted by the invalidation process (which deletes keys matching the prompt: prefix).

Lock-Invalidate-Unlock Protocol

FUNCTION mutatePrompt(projectId, promptName, mutationFn):
    // Phase 1: Lock
    lockKey = "LOCK:prompt:" + projectId
    REDIS.SETEX(lockKey, 30, "locked")   // 30-second TTL auto-releases on failure

    // Phase 2: Invalidate
    keyIndexKey = "prompt_key_index:" + projectId
    allKeys = REDIS.SMEMBERS(keyIndexKey)

    // Also handle legacy key index format for backwards compatibility
    legacyKeyIndexKey = keyIndexKey + ":" + promptName
    legacyKeys = REDIS.SMEMBERS(legacyKeyIndexKey)

    keysToDelete = UNION(allKeys, {keyIndexKey}, legacyKeys, {legacyKeyIndexKey})
    safeMultiDel(keysToDelete)            // Batched deletion via pipeline

    // Phase 3: Database mutation (transactional)
    result = TRANSACTION(mutationFn)

    // Phase 4: Unlock
    REDIS.DEL(lockKey)

    RETURN result

Read Path with Lock Awareness

FUNCTION getPrompt(projectId, promptName, version_or_label):
    // Check if cache should be used
    IF NOT cacheEnabled:
        RETURN fetchFromDatabase(...)

    lockKey = "LOCK:prompt:" + projectId
    IF REDIS.EXISTS(lockKey):
        // Cache is locked: bypass and read from database directly
        RETURN fetchFromDatabase(...)

    // Try cache
    cacheKey = "prompt:" + projectId + ":" + promptName + ":" + version_or_label
    cached = REDIS.GETEX(cacheKey, "EX", ttlSeconds)  // Read and refresh TTL

    IF cached IS NOT NULL:
        RETURN deserialize(cached)

    // Cache miss: fetch from DB and populate cache
    result = fetchFromDatabase(...)
    IF result IS NOT NULL:
        REDIS.SADD(keyIndexKey, cacheKey)    // Register in key index
        REDIS.SET(cacheKey, serialize(result), "EX", ttlSeconds)

    RETURN result

Key properties:

1. Lock TTL as safety net: The 30-second TTL on the lock key ensures that even if the unlock step fails (due to a crash or network issue), the lock will automatically expire and normal caching resumes. The TTL is short enough to minimize impact but long enough to cover typical mutation durations.
2. Project-wide invalidation: Because prompt composability means a change to one prompt can affect resolved content of other prompts (via dependencies), the invalidation operates at the project level rather than per-prompt. This is a deliberate trade-off: slightly more cache churn in exchange for guaranteed correctness.
3. Legacy compatibility: The system handles migration from an older key index format (per-prompt-name) to the current format (per-project) by checking and deleting both key index variants during invalidation.
4. Safe multi-delete: The safeMultiDel utility batches deletion operations via Redis pipelines to avoid blocking the Redis event loop with large DELETE commands.

Related Pages

Implemented By

• Implementation:Langfuse_Langfuse_PromptService_Cache