Implementation:LMCache LMCache P2PBackend Batched Get Non Blocking

Knowledge Sources	LMCache
Domains	Caching, High_Performance_Computing
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for retrieving KV cache chunks from peer instances via NIXL RDMA reads, provided by the P2PBackend class.

Description

The P2PBackend.batched_get_non_blocking method handles cross-instance KV cache retrieval. It first checks local existence via batched_async_contains (which queries the controller), then for chunks found on peers, initiates NIXL async_batched_read operations to fetch the data. The method returns a list of MemoryObj that will be populated when the RDMA transfers complete.

Usage

Called internally by the storage manager during the retrieve operation when P2P mode is enabled. Users do not call this directly.

Code Reference

Source Location

Repository: LMCache
File: lmcache/v1/storage_backend/p2p_backend.py
Lines: L550-L649

Signature

def batched_get_non_blocking(
    self,
    lookup_id: str,
    keys: list[CacheEngineKey],
    transfer_spec: Any = None,
) -> list[MemoryObj]:
    """Retrieve KV cache chunks from peer instances via NIXL.

    Args:
        lookup_id: Unique identifier for this lookup operation
        keys: Cache chunk keys to retrieve
        transfer_spec: NIXL transfer metadata with peer info

    Returns:
        List of MemoryObj populated with KV data from remote peers
    """

Import

from lmcache.v1.storage_backend.p2p_backend import P2PBackend

I/O Contract

Inputs

Name	Type	Required	Description
lookup_id	str	Yes	Unique lookup identifier
keys	list[CacheEngineKey]	Yes	Cache chunk keys to retrieve
transfer_spec	Any	No	NIXL transfer metadata

Outputs

Name	Type	Description
return	list[MemoryObj]	Memory objects populated via NIXL RDMA reads from peer

Related Pages

Implements Principle

Principle:LMCache_LMCache_P2P_KV_Transfer

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment