Implementation:LMCache LMCache Standalone Manager

Knowledge Sources	LMCache
Domains	KV Cache, Standalone Deployment
Last Updated	2026-02-09 00:00 GMT

Overview

StandaloneLMCacheManager is a specialized manager for running LMCache in standalone mode without vLLM dependencies.

Description

This class extends LMCacheManager to provide a simplified initialization path for standalone deployments. It accepts pre-constructed LMCacheMetadata, a GPU connector, and broadcast functions as constructor arguments rather than extracting them from a vLLM configuration. The _init_components method creates the LMCache engine via LMCacheEngineBuilder.get_or_create and starts an internal API server. The post_init method handles engine post-initialization and health monitor setup, with proper error handling for initialization failures. Currently only the "worker" role is supported; the "scheduler" role is not yet implemented for standalone mode.

Usage

Use this manager when deploying LMCache independently of vLLM, such as in custom inference pipelines or testing scenarios. It provides the same cache management interface as the standard manager but without requiring a vLLM configuration.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/standalone/manager.py
• Lines: 1-118

Signature

class StandaloneLMCacheManager(LMCacheManager):
    def __init__(
        self,
        config: Any,
        metadata: LMCacheMetadata,
        gpu_connector: Any,
        broadcast_fn: Callable,
        broadcast_object_fn: Callable,
        connector: Optional[Any] = None,
    ): ...

    def _init_components(self) -> None: ...
    def post_init(self) -> None: ...
    def stop_services(self) -> None: ...

Import

from lmcache.v1.standalone.manager import StandaloneLMCacheManager

I/O Contract

Inputs

Name	Type	Required	Description
config	Any	Yes	LMCache engine configuration
metadata	LMCacheMetadata	Yes	Pre-constructed metadata with model and deployment info
gpu_connector	Any	Yes	GPU connector instance for KV cache GPU operations
broadcast_fn	Callable	Yes	Broadcast function for tensor parallel communication
broadcast_object_fn	Callable	Yes	Broadcast function for arbitrary Python objects
connector	Optional[Any]	No	Reference to LMCacheConnectorV1Impl for internal API server

Outputs

Name	Type	Description
_lmcache_engine	LMCacheEngine	The initialized cache engine instance
_api_server	InternalAPIServer	The internal API server for management operations

Usage Examples

from lmcache.v1.standalone.manager import StandaloneLMCacheManager
from lmcache.v1.metadata import LMCacheMetadata

# Create metadata for standalone mode
metadata = LMCacheMetadata(
    model_name="llama-7b",
    world_size=1,
    local_world_size=1,
    worker_id=0,
    local_worker_id=0,
    kv_dtype=torch.float16,
    kv_shape=(32, 2, 256, 32, 128),
)

# Initialize the standalone manager
manager = StandaloneLMCacheManager(
    config=config,
    metadata=metadata,
    gpu_connector=gpu_connector,
    broadcast_fn=broadcast_fn,
    broadcast_object_fn=broadcast_object_fn,
)
manager.post_init()

# Use the manager...

# Shutdown
manager.stop_services()