Implementation:LMCache LMCache Base Cache Policy

Knowledge Sources	LMCache
Domains	Cache Eviction, Storage Backend
Last Updated	2026-02-09 00:00 GMT

Overview

BaseCachePolicy is an abstract generic interface that defines the contract for cache eviction policies in LMCache's storage backends.

Description

This abstract base class is parameterized by KeyType and MapType (a MutableMapping bound type), allowing concrete policies to work with different key types and storage structures. It declares four abstract methods: init_mutable_mapping for creating the storage container, update_on_hit for updating internal state when a cache entry is accessed, update_on_put for updating state when a new entry is stored, update_on_force_evict for cleaning up state when an entry is forcibly evicted, and get_evict_candidates for selecting entries to evict when storage is full. This design cleanly separates the eviction policy logic from the storage backend itself.

Usage

Use this as the base class when implementing new cache eviction policies (e.g., FIFO, LRU, LFU). Storage backends use the policy interface to manage eviction without being coupled to a specific algorithm.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/storage_backend/cache_policy/base_policy.py
• Lines: 1-87

Signature

class BaseCachePolicy(Generic[KeyType, MapType], metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def init_mutable_mapping(self) -> MapType: ...

    @abc.abstractmethod
    def update_on_hit(self, key: KeyType, cache_dict: MapType) -> None: ...

    @abc.abstractmethod
    def update_on_put(self, key: KeyType) -> None: ...

    @abc.abstractmethod
    def update_on_force_evict(self, key: KeyType) -> None: ...

    @abc.abstractmethod
    def get_evict_candidates(self, cache_dict: MapType, num_candidates: int = 1) -> list[KeyType]: ...

Import

from lmcache.v1.storage_backend.cache_policy.base_policy import BaseCachePolicy, KeyType, MapType

I/O Contract

Inputs

Name	Type	Required	Description
key	KeyType	Yes	Cache entry key for update operations
cache_dict	MapType	Yes	The mutable mapping containing current cache entries
num_candidates	int	No	Number of eviction candidates to return (default 1)

Outputs

Name	Type	Description
init_mutable_mapping	MapType	A new mutable mapping instance for cache storage
get_evict_candidates	list[KeyType]	List of keys recommended for eviction (may be fewer than num_candidates)

Usage Examples

from lmcache.v1.storage_backend.cache_policy.base_policy import BaseCachePolicy

# Implement a custom cache policy
class MyCachePolicy(BaseCachePolicy[str, dict]):
    def init_mutable_mapping(self) -> dict:
        return {}

    def update_on_hit(self, key, cache_dict):
        pass  # Custom hit tracking

    def update_on_put(self, key):
        pass  # Custom put tracking

    def update_on_force_evict(self, key):
        pass  # Custom eviction cleanup

    def get_evict_candidates(self, cache_dict, num_candidates=1):
        return list(cache_dict.keys())[:num_candidates]