Implementation:LMCache LMCache LRU Evictor

Knowledge Sources	LMCache
Domains	Cache Eviction, Storage Backend
Last Updated	2026-02-09 00:00 GMT

Overview

Implements a Least Recently Used (LRU) cache eviction strategy with a configurable maximum cache size.

Description

The LRUEvictor extends BaseEvictor to provide LRU eviction semantics. On cache access (update_on_get), the accessed key is moved to the end of the OrderedDict to mark it as most recently used. On cache insertion (update_on_put), if the new entry would exceed the maximum cache size, the evictor iterates from the front of the ordered dictionary (least recently used) and collects keys to evict until sufficient space is freed. If a single entry is larger than the total maximum cache size, it is rejected as ILLEGAL. The maximum cache size is specified in gigabytes and internally converted to bytes.

Usage

Use LRUEvictor as the eviction policy for LMCache storage backends that need bounded memory usage with LRU replacement. Pass the desired maximum size in gigabytes at construction time.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/storage_backend/evictor/lru_evictor.py
• Lines: 1-77

Signature

class LRUEvictor(BaseEvictor):
    def __init__(self, max_cache_size: float = 10.0): ...
    def update_on_get(self, key: Union[CacheEngineKey, str], cache_dict: OrderedDict) -> None: ...
    def update_on_put(self, cache_dict: OrderedDict, cache_size: int) -> Tuple[List, PutStatus]: ...

Import

from lmcache.storage_backend.evictor.lru_evictor import LRUEvictor

I/O Contract

Inputs

Name	Type	Required	Description
max_cache_size	float	No (default 10.0)	Maximum cache size in gigabytes
key	Union[CacheEngineKey, str]	Yes (update_on_get)	The cache key being accessed
cache_dict	OrderedDict	Yes	The current ordered cache dictionary
cache_size	int	Yes (update_on_put)	Size of the new entry in bytes

Outputs

Name	Type	Description
evict_keys	List[Union[CacheEngineKey, str]]	Keys of entries to evict (oldest first)
status	PutStatus	LEGAL if insertion is allowed, ILLEGAL if entry exceeds total capacity

Usage Examples

from collections import OrderedDict
from lmcache.storage_backend.evictor.lru_evictor import LRUEvictor

# Create an LRU evictor with 2 GB maximum
evictor = LRUEvictor(max_cache_size=2.0)

cache = OrderedDict()

# Simulate a put: check which keys need eviction
evict_keys, status = evictor.update_on_put(cache, new_entry_bytes)
for k in evict_keys:
    del cache[k]

# Simulate a get: update recency
evictor.update_on_get("my_key", cache)