Implementation:LMCache LMCache Config

Knowledge Sources	LMCache
Domains	Configuration, KV Cache
Last Updated	2026-02-09 00:00 GMT

Overview

Defines the LMCacheMemPoolMetadata dataclass and the default blend separator constant used for memory pool initialization.

Description

This module provides a lightweight configuration dataclass, LMCacheMemPoolMetadata, that captures the shape, dtype, and maximum size parameters required to initialize an LMCache memory pool. The kv_shape tuple has five dimensions representing the structure of KV cache tensors. The module also defines blend_default_separator, a string constant used as the default separator token in LMCache's blend (prefix sharing) functionality.

Usage

Use LMCacheMemPoolMetadata when initializing memory pool components that need to know the KV cache tensor layout and size constraints. Use blend_default_separator when constructing or parsing blended cache keys.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/config.py
• Lines: 1-24

Signature

@dataclass
class LMCacheMemPoolMetadata:
    kv_shape: Tuple[int, int, int, int, int]
    kv_dtype: torch.dtype
    max_local_cache_size: int

blend_default_separator = "[BLEND_SEP]"

Import

from lmcache.config import LMCacheMemPoolMetadata, blend_default_separator

I/O Contract

Inputs

Name	Type	Required	Description
kv_shape	Tuple[int, int, int, int, int]	Yes	Five-dimensional shape of KV cache tensors for the memory pool
kv_dtype	torch.dtype	Yes	Data type of the KV cache tensors (e.g., torch.float16)
max_local_cache_size	int	Yes	Maximum number of entries in the local cache

Outputs

Name	Type	Description
LMCacheMemPoolMetadata	dataclass	Immutable metadata object for memory pool initialization
blend_default_separator	str	Default separator string "[BLEND_SEP]" for blended cache keys

Usage Examples

import torch
from lmcache.config import LMCacheMemPoolMetadata

metadata = LMCacheMemPoolMetadata(
    kv_shape=(32, 2, 128, 8, 128),
    kv_dtype=torch.float16,
    max_local_cache_size=1000,
)