Principle:LMCache LMCache P2P KV Transfer
| Knowledge Sources | |
|---|---|
| Domains | Caching, High_Performance_Computing |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
A cross-instance KV cache retrieval mechanism that fetches cached KV tensors from peer vLLM instances via NIXL RDMA transfers, coordinated by the centralized controller.
Description
P2P KV Transfer enables one vLLM instance to retrieve KV cache chunks from another instance that has already computed them. When a local cache miss occurs, the P2PBackend queries the controller to find which peer holds the needed chunks, establishes a NIXL connection to that peer, and performs an async RDMA read to fetch the KV data.
Usage
This operates transparently when enable_p2p=True. On cache miss, the P2PBackend's batched_get_non_blocking method triggers a controller lookup and initiates NIXL reads from the peer.
Theoretical Basis
P2P transfer follows a lookup-connect-read pattern:
- Lookup: Controller's KVController.batched_p2p_lookup identifies which peer has the chunks
- Connect: NixlChannel establishes RDMA connection to peer (lazy initialization)
- Read: NixlChannel.async_batched_read performs one-sided RDMA reads into local buffer
Related Pages
Implemented By
Page Connections
Double-click a node to navigate. Hold to expand connections.
Principle
Implementation
Heuristic
Environment