Implementation:LMCache LMCache TTL List Cache

Knowledge Sources	LMCache
Domains	Caching, Concurrency
Last Updated	2026-02-09 00:00 GMT

Overview

TTLListCache is a generic, thread-safe list cache with configurable time-to-live expiration.

Description

The TTLListCache class is a generic container (parameterized over type T) that caches a list of items and automatically refreshes the data when the TTL expires. It uses a double-checked locking pattern with threading.Lock for thread safety: a fast path checks expiration without locking, and only acquires the lock on the slow path to refresh stale data. The cache supports timeout overrides per call (including zero for always-fresh data), provides cache age reporting, and includes utility methods for clearing, length checking, and string representation. The refresh function is provided as a callback to get_cached, enabling lazy and pluggable data sourcing.

Usage

Use TTLListCache when you need to periodically refresh a list of items (such as remote key listings) without querying the source on every access. It is useful for caching results of expensive operations that can tolerate bounded staleness, such as listing keys in a remote storage backend.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/utils/cache_utils.py
• Lines: 1-125

Signature

class TTLListCache(Generic[T]):
    def __init__(self, timeout_seconds: float = 30.0) -> None: ...
    def get_cached(self, get_fresh_data: Callable[[], list[T]], timeout_override: Optional[float] = None) -> list[T]: ...
    def clear(self) -> None: ...
    def is_expired(self, timeout_override: Optional[float] = None, current_time: Optional[float] = None) -> bool: ...
    def get_cache_age(self) -> float: ...
    def __len__(self) -> int: ...
    def __repr__(self) -> str: ...

Import

from lmcache.v1.utils.cache_utils import TTLListCache

I/O Contract

Inputs

Name	Type	Required	Description
timeout_seconds	float	No (default 30.0)	Default TTL in seconds for cached data
get_fresh_data	Callable[[], list[T]]	Yes (for get_cached)	Callback function that returns fresh data when the cache is expired
timeout_override	Optional[float]	No	Per-call TTL override; 0 means always refresh, None uses the default

Outputs

Name	Type	Description
get_cached return	list[T]	The cached list of items, either from cache or freshly fetched
is_expired return	bool	True if cache is expired or uninitialized
get_cache_age return	float	Age of the cache in seconds; float('inf') if uninitialized

Usage Examples

from lmcache.v1.utils.cache_utils import TTLListCache

# Create a cache with 30-second TTL
cache = TTLListCache[str](timeout_seconds=30.0)

# Define a data-fetching function
def fetch_remote_keys() -> list[str]:
    return ["key1", "key2", "key3"]

# Get data (fetches on first call, then returns cached until TTL expires)
keys = cache.get_cached(fetch_remote_keys)

# Force a fresh fetch regardless of TTL
keys = cache.get_cached(fetch_remote_keys, timeout_override=0)

# Check cache state
print(f"Cache age: {cache.get_cache_age():.1f}s")
print(f"Expired: {cache.is_expired()}")
print(f"Items: {len(cache)}")

# Clear the cache
cache.clear()