Principle:LMCache LMCache Cache Controller Architecture
| Knowledge Sources | |
|---|---|
| Domains | Distributed_Systems, Cluster_Management |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A centralized controller pattern that coordinates peer-to-peer KV cache sharing across multiple vLLM instances through worker registration, health monitoring, and cache operation dispatch.
Description
The Cache Controller Architecture provides a centralized coordination point for a cluster of LMCache-enabled vLLM instances. The controller runs as a standalone FastAPI service that maintains a registry of connected workers, tracks which KV cache chunks exist on which instances, and dispatches operations (lookup, move, pin, clear, compress) to workers via ZMQ messaging.
The architecture separates concerns:
- LMCacheControllerManager: Top-level orchestrator managing registration, health checks, and message routing
- KVController: Handles KV-specific operations (lookup, move, pin, clear, compress)
- RegistrationController: Manages worker registration and discovery
- LMCacheClusterExecutor: Dispatches operations to workers via ZMQ sockets
Usage
Deploy the controller as the first component in a P2P sharing cluster. Workers register automatically on startup when enable_controller=True in their config.
Theoretical Basis
The controller implements a hub-and-spoke coordination model:
- Hub: Controller with global view of all workers and their cache contents
- Spokes: LMCacheWorker agents running alongside each vLLM instance
- Data plane: Direct NIXL P2P transfers between workers (no data through controller)
- Control plane: ZMQ messages for registration, heartbeats, and operation dispatch