Implementation:LMCache LMCache Base Evictor

Knowledge Sources	LMCache
Domains	Cache Eviction, Storage Backend
Last Updated	2026-02-09 00:00 GMT

Overview

Defines the abstract base class for cache evictors and a no-op DummyEvictor implementation.

Description

The BaseEvictor abstract class establishes the interface that all cache eviction strategies must implement. It requires two abstract methods: update_on_get for updating eviction state when a cache entry is accessed, and update_on_put for determining which entries to evict when storage is full. The PutStatus enum distinguishes between legal (cacheable) and illegal (too large) put operations. A concrete get_size helper method computes the byte size of KV cache objects, supporting torch.Tensor, bytearray, and DiskCacheMetadata types. The DummyEvictor class provides a no-eviction implementation that accepts all puts without evicting anything.

Usage

Subclass BaseEvictor to implement custom eviction policies (e.g., LRU, LFU). Use DummyEvictor when eviction is not needed, such as during testing or when storage is unlimited.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/storage_backend/evictor/base_evictor.py
• Lines: 1-96

Signature

class PutStatus(Enum):
    LEGAL = auto()
    ILLEGAL = auto()

class BaseEvictor(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def update_on_get(self, key: Union[CacheEngineKey, str], cache_dict: OrderedDict) -> None: ...
    @abc.abstractmethod
    def update_on_put(self, cache_dict: OrderedDict, cache_size: int) -> Tuple[List[Union[CacheEngineKey, str]], PutStatus]: ...
    def get_size(self, kv_obj: Union[torch.Tensor, bytes]) -> int: ...

class DummyEvictor(BaseEvictor):
    def update_on_get(self, key, cache_dict) -> None: ...
    def update_on_put(self, cache_dict, cache_size): ...

Import

from lmcache.storage_backend.evictor.base_evictor import BaseEvictor, PutStatus, DummyEvictor

I/O Contract

Inputs

Name	Type	Required	Description
key	Union[CacheEngineKey, str]	Yes (update_on_get)	Cache key of the accessed entry
cache_dict	OrderedDict	Yes	Current cache dictionary mapping keys to KV objects
cache_size	int	Yes (update_on_put)	Size in bytes of the new cache entry to be inserted
kv_obj	Union[torch.Tensor, bytearray, DiskCacheMetadata]	Yes (get_size)	KV cache object to compute size for

Outputs

Name	Type	Description
evict_keys	List[Union[CacheEngineKey, str]]	List of keys to evict from the cache
status	PutStatus	LEGAL if the entry can be cached, ILLEGAL if it exceeds capacity
size_in_bytes	int	Byte size of the given KV object (from get_size)

Usage Examples

from lmcache.storage_backend.evictor.base_evictor import DummyEvictor, PutStatus

evictor = DummyEvictor()

# On cache access
evictor.update_on_get("key1", cache_dict)

# On cache insertion
evict_keys, status = evictor.update_on_put(cache_dict, new_entry_size)
if status == PutStatus.LEGAL:
    for key in evict_keys:
        del cache_dict[key]
    cache_dict[new_key] = new_value