Implementation:LMCache LMCache FIFO Cache Policy

Knowledge Sources	LMCache
Domains	Cache Eviction, Storage Backend
Last Updated	2026-02-09 00:00 GMT

Overview

FIFOCachePolicy implements a First-In-First-Out cache eviction strategy for LMCache storage backends.

Description

This class extends BaseCachePolicy and leverages the insertion-order preservation of Python dictionaries to implement FIFO semantics. The update_on_hit, update_on_put, and update_on_force_evict methods are all no-ops since FIFO does not need to track access frequency or recency. The get_evict_candidates method iterates through the cache dictionary in insertion order and selects the first num_candidates entries whose can_evict property is True, performing a best-effort eviction where fewer entries may be returned than requested if some are pinned.

Usage

Use this policy when a simple FIFO eviction strategy is desired. It is the simplest eviction policy available and incurs no additional bookkeeping overhead.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/storage_backend/cache_policy/fifo.py
• Lines: 1-58

Signature

class FIFOCachePolicy(BaseCachePolicy[KeyType, dict[KeyType, Any]]):
    def __init__(self): ...

    def init_mutable_mapping(self) -> dict[KeyType, Any]: ...
    def update_on_hit(self, key: KeyType, cache_dict: dict[KeyType, Any]) -> None: ...
    def update_on_put(self, key: KeyType) -> None: ...
    def update_on_force_evict(self, key: KeyType) -> None: ...
    def get_evict_candidates(self, cache_dict: dict[KeyType, Any], num_candidates: int = 1) -> list[KeyType]: ...

Import

from lmcache.v1.storage_backend.cache_policy.fifo import FIFOCachePolicy

I/O Contract

Inputs

Name	Type	Required	Description
key	KeyType	Yes	Cache entry key (unused in update methods for FIFO)
cache_dict	dict[KeyType, Any]	Yes	The cache dictionary with entries maintaining insertion order
num_candidates	int	No	Maximum number of eviction candidates to return (default 1)

Outputs

Name	Type	Description
init_mutable_mapping	dict	An empty Python dictionary preserving insertion order
get_evict_candidates	list[KeyType]	Up to num_candidates keys eligible for eviction, in insertion order

Usage Examples

from lmcache.v1.storage_backend.cache_policy.fifo import FIFOCachePolicy

policy = FIFOCachePolicy()
cache = policy.init_mutable_mapping()

# Store entries
cache["key1"] = some_value
policy.update_on_put("key1")

cache["key2"] = some_value
policy.update_on_put("key2")

# Get eviction candidates (oldest first)
candidates = policy.get_evict_candidates(cache, num_candidates=1)
# candidates == ["key1"] (first inserted)