Implementation:LMCache LMCache Lookup Client Interface

Knowledge Sources	LMCache
Domains	Lookup Client, Abstract Interface
Last Updated	2026-02-09 00:00 GMT

Overview

Defines the abstract interface that all LMCache lookup clients must implement, specifying the contract for cache lookup, resource cleanup, and optional features.

Description

LookupClientInterface is an abstract base class (using abc.ABCMeta) that establishes the contract for all lookup client implementations in LMCache. It declares two abstract methods: lookup (which performs the initial token ID lookup and pinning, returning the number of cached tokens or None if in progress) and close (for resource cleanup). It also provides default implementations for lookup_cache (subsequent lookups by ID, returning hit count or None), supports_producer_reuse (returns False by default), and clear_lookup_status (no-op by default). The interface separates the initial lookup (which involves token processing) from subsequent cache checks (which only need the lookup ID).

Usage

Implement this interface to create a new lookup client variant. The scheduler calls lookup for the first token lookup of a request, lookup_cache for subsequent checks, and clear_lookup_status when a request completes. Check supports_producer_reuse to determine if producer KV cache reuse optimization is available.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/v1/lookup_client/abstract_client.py
• Lines: 1-77

Signature

class LookupClientInterface(metaclass=abc.ABCMeta):
    def lookup_cache(self, lookup_id: str) -> Optional[int]: ...
    @abc.abstractmethod
    def lookup(self, token_ids: Union[torch.Tensor, list[int]],
               lookup_id: str,
               request_configs: Optional[dict] = None) -> Optional[int]: ...
    @abc.abstractmethod
    def close(self) -> None: ...
    def supports_producer_reuse(self) -> bool: ...
    def clear_lookup_status(self, lookup_id: str) -> None: ...

Import

from lmcache.v1.lookup_client.abstract_client import LookupClientInterface

I/O Contract

Inputs

Name	Type	Required	Description
token_ids	Union[torch.Tensor, list[int]]	Yes	Token IDs to look up in the cache
lookup_id	str	Yes	Unique identifier to associate with the lookup request
request_configs	Optional[dict]	No	Request-specific configuration (tags and other configs)

Outputs

Name	Type	Description
hit_tokens	Optional[int]	Number of tokens found in cache; -1 means not found; None means lookup is in progress
supports_reuse	bool	Whether producer KV cache reuse is supported (default: False)

Usage Examples

from lmcache.v1.lookup_client.abstract_client import LookupClientInterface

class MyLookupClient(LookupClientInterface):
    def lookup(self, token_ids, lookup_id, request_configs=None):
        # Perform cache lookup logic
        hit_count = self._check_cache(token_ids)
        return hit_count

    def close(self):
        # Clean up resources
        pass

# Usage
client = MyLookupClient()
result = client.lookup(token_ids=[1, 2, 3, 4], lookup_id="req-001")
if result is not None and result >= 0:
    print(f"Cache hit: {result} tokens")

# Subsequent lookups by ID
cached = client.lookup_cache("req-001")

# Cleanup
client.clear_lookup_status("req-001")
client.close()