Implementation:LMCache LMCache Connector V1 085

Knowledge Sources	LMCache
Domains	Integration, vLLM, KV Cache
Last Updated	2026-02-09 00:00 GMT

Overview

Implements the vLLM v1 KV connector interface for LMCache, targeting vLLM version 0.8.5 with a simplified API surface.

Description

The LMCacheConnectorV1Dynamic class in this file is a compatibility variant for vLLM 0.8.5. Unlike the newer connector, this version has a simpler constructor that takes only vllm_config and role parameters, does not include register_kv_caches, get_finished, get_block_ids_with_load_errors, or request_finished methods, and returns a plain int from get_num_new_matched_tokens rather than a tuple. It otherwise follows the same delegation pattern, routing all calls to an internal LMCacheConnectorV1Impl instance.

Usage

Use this connector when running vLLM version 0.8.5 with LMCache. The appropriate connector version is typically selected automatically based on the installed vLLM version.

Code Reference

Source Location

• Repository: LMCache
• File: lmcache/integration/vllm/lmcache_connector_v1_085.py
• Lines: 1-150

Signature

class LMCacheConnectorV1Dynamic(KVConnectorBase_V1):
    def __init__(self, vllm_config: "VllmConfig", role: KVConnectorRole): ...
    def start_load_kv(self, forward_context: "ForwardContext", **kwargs) -> None: ...
    def wait_for_layer_load(self, layer_name: str) -> None: ...
    def save_kv_layer(self, layer_name: str, kv_layer: torch.Tensor, attn_metadata: "AttentionMetadata", **kwargs) -> None: ...
    def wait_for_save(self): ...
    def shutdown(self): ...
    def get_num_new_matched_tokens(self, request: "Request", num_computed_tokens: int) -> int: ...
    def update_state_after_alloc(self, request: "Request", num_external_tokens: int): ...
    def build_connector_meta(self, scheduler_output: SchedulerOutput) -> KVConnectorMetadata: ...

Import

from lmcache.integration.vllm.lmcache_connector_v1_085 import LMCacheConnectorV1Dynamic

I/O Contract

Inputs

Name	Type	Required	Description
vllm_config	VllmConfig	Yes	The vLLM 0.8.5 configuration object
role	KVConnectorRole	Yes	Whether this connector runs as scheduler or worker
forward_context	ForwardContext	Yes (start_load_kv)	Context with KV caches and layer names for the forward pass
layer_name	str	Yes (wait_for_layer_load, save_kv_layer)	Transformer layer name
kv_layer	torch.Tensor	Yes (save_kv_layer)	Paged KV buffer for the layer
attn_metadata	AttentionMetadata	Yes (save_kv_layer)	Attention metadata
request	Request	Yes (scheduler methods)	The vLLM request object
num_computed_tokens	int	Yes (get_num_new_matched_tokens)	Locally computed token count

Outputs

Name	Type	Description
num_new_matched_tokens	int	Number of externally available tokens beyond what is computed
connector_meta	KVConnectorMetadata	Connector metadata for the scheduling step

Usage Examples

# Automatically instantiated by vLLM 0.8.5's KV connector framework:
from lmcache.integration.vllm.lmcache_connector_v1_085 import LMCacheConnectorV1Dynamic

connector = LMCacheConnectorV1Dynamic(
    vllm_config=vllm_config,
    role=KVConnectorRole.WORKER,
)