Implementation:LMCache LMCache MRU Cache Policy

Knowledge Sources	LMCache
Domains	Caching, Eviction Policy
Last Updated	2026-02-09 00:00 GMT

Overview

MRUCachePolicy implements a Most Recently Used eviction strategy for the LMCache storage backend.

Description

The MRUCachePolicy class extends BaseCachePolicy to provide MRU-based cache eviction. Like the LRU policy, it uses an OrderedDict as its underlying mapping and moves accessed keys to the end on cache hits. However, eviction candidates are selected by iterating in reverse order (from the back), targeting the most recently used entries first. Entries that have their can_evict flag set to false are skipped during candidate selection, so the actual number of returned candidates may be fewer than requested.

Usage

Use this policy when the workload benefits from evicting recently used cache entries. MRU is effective in scenarios where recently accessed KV cache chunks are unlikely to be needed again soon, such as sequential scan patterns where each chunk is processed once and then not revisited.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/storage_backend/cache_policy/mru.py
• Lines: 1-61

Signature

class MRUCachePolicy(BaseCachePolicy[KeyType, OrderedDict[KeyType, Any]]):
    def __init__(self) -> None: ...
    def init_mutable_mapping(self) -> OrderedDict[KeyType, Any]: ...
    def update_on_hit(self, key: KeyType, cache_dict: OrderedDict[KeyType, Any]) -> None: ...
    def update_on_put(self, key: KeyType) -> None: ...
    def update_on_force_evict(self, key: KeyType) -> None: ...
    def get_evict_candidates(self, cache_dict: OrderedDict[KeyType, Any], num_candidates: int = 1) -> list[KeyType]: ...

Import

from lmcache.v1.storage_backend.cache_policy.mru import MRUCachePolicy

I/O Contract

Inputs

Name	Type	Required	Description
key	KeyType	Yes	The cache key being accessed, stored, or evicted
cache_dict	OrderedDict[KeyType, Any]	Yes (for update_on_hit, get_evict_candidates)	The ordered dictionary maintaining cache entries in access order
num_candidates	int	No (default 1)	Number of eviction candidates to return

Outputs

Name	Type	Description
init_mutable_mapping return	OrderedDict[KeyType, Any]	A new empty OrderedDict for cache storage
get_evict_candidates return	list[KeyType]	List of keys to evict (from back of OrderedDict, skipping non-evictable entries)

Usage Examples

from lmcache.v1.storage_backend.cache_policy.mru import MRUCachePolicy

# Create the MRU cache policy
policy = MRUCachePolicy()

# Initialize the cache storage
cache_dict = policy.init_mutable_mapping()

# On a cache hit, the key moves to the end (most recently used position)
policy.update_on_hit(key, cache_dict)

# Eviction candidates come from the back (most recently used first)
candidates = policy.get_evict_candidates(cache_dict, num_candidates=2)