Implementation:LMCache LMCache ZMQ Offload Server

Knowledge Sources	LMCache
Domains	KV Cache, Offloading, Messaging
Last Updated	2026-02-09 00:00 GMT

Overview

ZMQOffloadServer is a ZeroMQ-based implementation of the offload server that receives offload requests over IPC sockets and delegates them to the LMCache engine.

Description

This class implements OffloadServerInterface using a ZMQ REP socket bound to an IPC path derived from the engine ID, offload RPC port, and tensor-parallel rank. It runs a background daemon thread that continuously receives OffloadMsg messages via msgspec deserialization, calls the offload method to store the data through the LMCache engine, and sends back an OffloadRetMsg response. The offload method delegates directly to LMCacheEngine.store(). Graceful shutdown is handled by setting a running flag, closing the socket to unblock recv(), and joining the background thread with a timeout.

Usage

Use this server when KV cache offload requests originate from a different process (e.g., a vLLM worker) and need to be communicated via ZMQ IPC. The server is typically started alongside the cache engine on each tensor-parallel rank.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/offload_server/zmq_server.py
• Lines: 1-122

Signature

class ZMQOffloadServer(OffloadServerInterface):
    def __init__(
        self,
        lmcache_engine: LMCacheEngine,
        tp_rank: int,
    ): ...

    def offload(
        self,
        hashes: List[int],
        slot_mapping: List[int],
        offsets: List[int],
    ) -> bool: ...

    def close(self) -> None: ...

Import

from lmcache.v1.offload_server.zmq_server import ZMQOffloadServer

I/O Contract

Inputs

Name	Type	Required	Description
lmcache_engine	LMCacheEngine	Yes	The cache engine instance used to store offloaded data
tp_rank	int	Yes	Tensor-parallel rank for constructing the IPC socket path
hashes	List[int]	Yes	Chunk hashes identifying the data to offload
slot_mapping	List[int]	Yes	Slot IDs mapping GPU memory locations
offsets	List[int]	Yes	Number of tokens in each block

Outputs

Name	Type	Description
offload result	bool	Always True on successful store delegation to the engine

Usage Examples

from lmcache.v1.offload_server.zmq_server import ZMQOffloadServer

# Initialize with an existing cache engine
server = ZMQOffloadServer(lmcache_engine=engine, tp_rank=0)

# The server runs a background thread listening for offload requests.
# Offload can also be called directly:
success = server.offload(hashes=[111, 222], slot_mapping=[0, 1], offsets=[256, 256])

# Shutdown gracefully
server.close()