Implementation:LMCache LMCache LMC Server Connector

Knowledge Sources	LMCache
Domains	Storage Backend, Network Communication
Last Updated	2026-02-09 00:00 GMT

Overview

LMCServerConnector provides TCP socket-based communication with the LMCache dedicated server for remote KV cache storage.

Description

The LMCServerConnector class extends RemoteConnector to communicate with an LMCache server using raw TCP sockets. It uses a custom binary protocol with ClientMetaMessage and ServerMetaMessage for serialization of commands (GET, PUT, EXIST). The connector deliberately uses blocking socket.recv_into() for receive operations to avoid memory copies, while sends use the event loop's sock_sendall. An asyncio.Lock ensures thread-safe access to the shared socket. Memory allocation for received data is handled by the LocalCPUBackend, and the connector supports both async and synchronous existence checks.

Usage

Use this connector when deploying a dedicated LMCache server (accessed via lm://host:port URLs). It is the native LMCache protocol connector designed for direct server-to-server KV cache sharing with minimal overhead.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/storage_backend/connector/lm_connector.py
• Lines: 1-176

Signature

class LMCServerConnector(RemoteConnector):
    def __init__(
        self,
        host: str,
        port: int,
        loop: asyncio.AbstractEventLoop,
        local_cpu_backend: LocalCPUBackend,
    ) -> None: ...
    def receive_all(self, meta: ServerMetaMessage) -> Optional[MemoryObj]: ...
    async def exists(self, key: CacheEngineKey) -> bool: ...
    def exists_sync(self, key: CacheEngineKey) -> bool: ...
    async def put(self, key: CacheEngineKey, memory_obj: MemoryObj) -> None: ...
    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]: ...
    async def list(self) -> List[str]: ...
    async def close(self) -> None: ...

Import

from lmcache.v1.storage_backend.connector.lm_connector import LMCServerConnector

I/O Contract

Inputs

Name	Type	Required	Description
host	str	Yes	LMCache server hostname or IP address
port	int	Yes	LMCache server port
loop	asyncio.AbstractEventLoop	Yes	Event loop for async socket operations
local_cpu_backend	LocalCPUBackend	Yes	Memory allocator for receive buffers
key	CacheEngineKey	Yes (for get/put/exists)	Cache key identifying the KV chunk
memory_obj	MemoryObj	Yes (for put)	Memory object containing the tensor data to send

Outputs

Name	Type	Description
exists return	bool	Whether the key exists on the LMCache server
get return	Optional[MemoryObj]	The retrieved memory object, or None if key not found or allocation failure
put return	None	Data is sent to the server asynchronously

Usage Examples

from lmcache.v1.storage_backend.connector.lm_connector import LMCServerConnector

# Create a connector to the LMCache server
connector = LMCServerConnector(
    host="localhost",
    port=65432,
    loop=asyncio.get_event_loop(),
    local_cpu_backend=cpu_backend,
)

# Store a KV cache chunk
await connector.put(cache_key, memory_obj)

# Check if a key exists
exists = await connector.exists(cache_key)

# Retrieve the KV cache chunk
result = await connector.get(cache_key)

# Close the connection
await connector.close()