Implementation:LMCache LMCache Lookup Client

Knowledge Sources	LMCache
Domains	Cache Lookup, Inter-Process Communication, Synchronous Processing
Last Updated	2026-02-09 00:00 GMT

Overview

LMCacheLookupClient and LMCacheLookupServer provide a ZMQ-based synchronous (blocking) lookup mechanism for KV cache hit detection across worker processes using REQ/ROUTER socket patterns.

Description

The LMCacheLookupClient implements the LookupClientInterface and communicates with LMCacheLookupServer instances via ZMQ REQ sockets with configurable send/receive timeouts. The client computes token hashes using a TokenDatabase, sends multipart messages containing hashes, offsets, lookup ID, and request configs to all worker servers, and waits synchronously for responses. It handles timeout errors by recreating all sockets and supports both chunked and blending (segment) token database modes. The server uses a ROUTER socket pattern, processes requests in a background thread, and dispatches lookups to LMCacheEngine.lookup. When results differ across workers, the minimum hit count is used for safety in tensor-parallel setups.

Usage

Use this module for synchronous cache lookups in environments where blocking behavior is acceptable or preferred over the async variant. The client runs in the scheduler process and the servers run in each worker process.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/lookup_client/lmcache_lookup_client.py
• Lines: 1-392

Signature

class LMCacheLookupClient(LookupClientInterface):
    def __init__(self, config: LMCacheEngineConfig, metadata: LMCacheMetadata) -> None: ...
    def lookup_cache(self, lookup_id: str) -> Optional[int]: ...
    def lookup(self, token_ids: Union[torch.Tensor, list[int]], lookup_id: str,
               request_configs: Optional[dict] = None) -> Optional[int]: ...
    def clear_lookup_status(self, lookup_id: str) -> None: ...
    def supports_producer_reuse(self) -> bool: ...
    def close(self) -> None: ...

class LMCacheLookupServer:
    def __init__(self, lmcache_engine: LMCacheEngine, metadata: LMCacheMetadata) -> None: ...
    def close(self) -> None: ...

Import

from lmcache.v1.lookup_client.lmcache_lookup_client import (
    LMCacheLookupClient,
    LMCacheLookupServer,
)

I/O Contract

Inputs

Name	Type	Required	Description
config	LMCacheEngineConfig	Yes	Engine configuration including lookup timeout, blending mode, and extra config
metadata	LMCacheMetadata	Yes	Metadata including engine_id, world_size, worker_id, use_mla flag, and RPC port
token_ids	Union[torch.Tensor, list[int]]	Yes (for lookup)	Token IDs to hash and look up in the cache
lookup_id	str	Yes	Unique identifier for the lookup request
request_configs	Optional[dict]	No	Optional per-request configuration parameters

Outputs

Name	Type	Description
lookup_cache()	Optional[int]	-1 if not found, int >= 0 for number of hit tokens (None is not returned for sync client)
lookup()	Optional[int]	Number of hit tokens (int >= 0), or 0 on timeout/error
supports_producer_reuse()	bool	Always returns True

Usage Examples

from lmcache.v1.lookup_client.lmcache_lookup_client import (
    LMCacheLookupClient,
    LMCacheLookupServer,
)

# Client side (scheduler process)
client = LMCacheLookupClient(config, metadata)
num_hit_tokens = client.lookup(token_ids, lookup_id="req-456")
print(f"Cache hit: {num_hit_tokens} tokens")

# Check cached result
cached = client.lookup_cache("req-456")
client.clear_lookup_status("req-456")
client.close()

# Server side (worker process) - uses context manager
with LMCacheLookupServer(lmcache_engine, metadata) as server:
    # Server processes requests in background thread
    pass