Implementation:LMCache LMCache Server

Knowledge Sources	LMCache
Domains	Networking, Storage, Server
Last Updated	2026-02-09 00:00 GMT

Overview

Implements the LMCache TCP server that accepts client connections and handles KV cache PUT, GET, EXIST, and LIST operations.

Description

The LMCacheServer class creates a TCP socket server that listens for incoming connections and spawns a new thread per client via threading.Thread. Each client connection is handled in a loop that reads fixed-size ClientMetaMessage headers, dispatches to the appropriate handler based on the command type, and sends back ServerMetaMessage responses. The server uses a pluggable storage backend created by CreateStorageBackend(device), supporting CPU or GPU storage. The main() function provides a command-line entry point accepting host, port, and optional storage device arguments.

Usage

Run this module as python -m lmcache.server <host> <port> [device] to start a standalone KV cache server. Clients connect via TCP and use the protocol defined in lmcache.protocol to store and retrieve serialized KV cache data.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/server/__main__.py
• Lines: 1-140

Signature

class LMCacheServer:
    def __init__(self, host: str, port: int, device: str): ...
    def receive_all(self, client_socket, n: int) -> Optional[bytearray]: ...
    def handle_client(self, client_socket) -> None: ...
    def run(self) -> None: ...

def main() -> None: ...

Import

from lmcache.server.__main__ import LMCacheServer

I/O Contract

Inputs

Name	Type	Required	Description
host	str	Yes	IP address or hostname to bind the server to
port	int	Yes	TCP port number to listen on
device	str	No (default "cpu")	Storage device for the backend ("cpu" or a GPU device string)
client_socket	socket	Yes (handle_client)	Connected client socket
n	int	Yes (receive_all)	Number of bytes to read from the socket

Outputs

Name	Type	Description
data	bytearray or None	Received bytes from receive_all, or None if connection closed
(server responses)	bytes	ServerMetaMessage headers followed by optional data payloads sent over TCP

Usage Examples

# Start the server programmatically
from lmcache.server.__main__ import LMCacheServer

server = LMCacheServer("0.0.0.0", 8080, "cpu")
server.run()  # Blocks, listening for connections

# Or from the command line:
# python -m lmcache.server 0.0.0.0 8080 cpu