Implementation:LMCache LMCache V1 Server

Knowledge Sources	LMCache
Domains	KV Cache, Network Server
Last Updated	2026-02-09 00:00 GMT

Overview

LMCacheServer is a standalone TCP server that accepts cache PUT, GET, EXIST, and HEALTH requests from remote LMCache clients and stores KV cache data in a configurable storage backend.

Description

The server binds to a host and port, listens for client connections, and dispatches each connection to a dedicated handler thread. Each client handler runs a loop that reads a fixed-size ClientMetaMessage header, deserializes the command, and executes the corresponding operation. PUT commands read the data payload and store it via the storage backend. GET commands retrieve data and send back a ServerMetaMessage header followed by the data. EXIST commands check key presence and return a success/fail status. HEALTH commands return a success response for liveness checks. The storage backend is created via CreateStorageBackend with a configurable device (CPU by default). The module includes a main() entry point that accepts host, port, and optional storage device from command-line arguments.

Usage

Use this server as a standalone remote KV cache store that LMCache clients connect to over TCP. Start it via python -m lmcache.v1.server host port [device].

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/server/__main__.py
• Lines: 1-170

Signature

class LMCacheServer:
    def __init__(self, host: str, port: int, device: str): ...

    def receive_all(self, client_socket, n) -> Optional[bytearray]: ...
    def handle_client(self, client_socket): ...
    def run(self): ...

def main(): ...

Import

from lmcache.v1.server.__main__ import LMCacheServer

I/O Contract

Inputs

Name	Type	Required	Description
host	str	Yes	Hostname or IP address to bind the server to
port	int	Yes	Port number to listen on
device	str	No	Storage backend device, defaults to "cpu"

Outputs

Name	Type	Description
ServerMetaMessage	bytes	Serialized response header sent to client for each command
data	bytes	KV cache data payload sent for GET responses

Supported Commands

Command	Description	Response
PUT	Store KV cache data with metadata	(none, data is stored)
GET	Retrieve KV cache data by key	ServerMetaMessage + data bytes
EXIST	Check if a key exists	ServerMetaMessage with SUCCESS or FAIL
HEALTH	Server liveness check	ServerMetaMessage with SUCCESS

Usage Examples

# Start from command line:
# python -m lmcache.v1.server 0.0.0.0 8080 cpu

# Or programmatically:
from lmcache.v1.server.__main__ import LMCacheServer

server = LMCacheServer(host="0.0.0.0", port=8080, device="cpu")
server.run()  # Blocks, accepting client connections