Implementation:LMCache LMCache LRU Cache Policy

Knowledge Sources	LMCache
Domains	Caching, Eviction Policy
Last Updated	2026-02-09 00:00 GMT

Overview

LRUCachePolicy implements a Least Recently Used eviction strategy for the LMCache storage backend.

Description

The LRUCachePolicy class extends BaseCachePolicy to provide LRU-based cache eviction. It uses Python's OrderedDict as the underlying mutable mapping, where recently accessed keys are moved to the end of the dictionary on cache hits. Eviction candidates are selected from the front (oldest-accessed entries), respecting the can_evict flag on each cache entry. The policy also tracks chunk hash timestamps for observability, reporting reuse intervals to the LMCStatsMonitor, and caps the internal timestamp dictionary at 12.5 million entries before clearing.

Usage

Use this policy when the cache storage backend should evict the least recently used entries first. It is the default eviction strategy suitable for workloads with temporal locality where recently accessed KV cache chunks are likely to be accessed again.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/storage_backend/cache_policy/lru.py
• Lines: 1-81

Signature

class LRUCachePolicy(BaseCachePolicy[KeyType, OrderedDict[KeyType, Any]]):
    def __init__(self) -> None: ...
    def init_mutable_mapping(self) -> OrderedDict[KeyType, Any]: ...
    def update_chunk_hash_dict(self, key: KeyType) -> None: ...
    def update_on_hit(self, key: KeyType, cache_dict: OrderedDict[KeyType, Any]) -> None: ...
    def update_on_put(self, key: KeyType) -> None: ...
    def update_on_force_evict(self, key: KeyType) -> None: ...
    def get_evict_candidates(self, cache_dict: OrderedDict[KeyType, Any], num_candidates: int = 1) -> list[KeyType]: ...

Import

from lmcache.v1.storage_backend.cache_policy.lru import LRUCachePolicy

I/O Contract

Inputs

Name	Type	Required	Description
key	KeyType	Yes	The cache key being accessed, stored, or evicted
cache_dict	OrderedDict[KeyType, Any]	Yes (for update_on_hit, get_evict_candidates)	The ordered dictionary maintaining cache entries in access order
num_candidates	int	No (default 1)	Number of eviction candidates to return

Outputs

Name	Type	Description
init_mutable_mapping return	OrderedDict[KeyType, Any]	A new empty OrderedDict for cache storage
get_evict_candidates return	list[KeyType]	List of keys to evict (from front of OrderedDict, skipping non-evictable entries)

Usage Examples

from lmcache.v1.storage_backend.cache_policy.lru import LRUCachePolicy

# Create the LRU cache policy
policy = LRUCachePolicy()

# Initialize the cache storage
cache_dict = policy.init_mutable_mapping()

# On a cache put (new entry stored)
policy.update_on_put(key)

# On a cache hit (existing entry accessed)
policy.update_on_hit(key, cache_dict)

# When storage is full, get eviction candidates
candidates = policy.get_evict_candidates(cache_dict, num_candidates=3)