Implementation:LMCache LMCache PDBackend Batched Submit Put Task

Knowledge Sources	LMCache
Domains	Networking, High_Performance_Computing
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for transferring KV cache chunks from prefiller to decoder via NIXL RDMA writes, provided by the PDBackend class.

Description

The PDBackend.batched_submit_put_task method orchestrates the full sender-side KV transfer. It ensures a NIXL peer connection exists, sends an AllocRequest to the receiver to allocate buffer space, filters out already-sent chunks, performs batched NIXL writes, and sends a ProxyNotif via ZMQ to signal completion.

Usage

Called internally by the StorageManager during the store operation when PD mode is enabled. Users do not call this directly.

Code Reference

Source Location

Repository: LMCache
File: lmcache/v1/storage_backend/pd_backend.py
Lines: L383-L462

Signature

def batched_submit_put_task(
    self,
    keys: Sequence[CacheEngineKey],
    memory_objs: List[MemoryObj],
    transfer_spec: Any = None,
    on_complete_callback: Optional[Callable[[CacheEngineKey], None]] = None,
) -> None:
    """Submit batched put tasks to transfer KV caches to peer.

    Args:
        keys: Cache engine keys for each chunk
        memory_objs: Memory objects containing KV data
        transfer_spec: Contains receiver endpoint info (host, init_port, alloc_port)
        on_complete_callback: Optional callback per key after transfer completes
    """

Import

from lmcache.v1.storage_backend.pd_backend import PDBackend

I/O Contract

Inputs

Name	Type	Required	Description
keys	Sequence[CacheEngineKey]	Yes	Cache chunk keys
memory_objs	List[MemoryObj]	Yes	KV cache data in memory objects
transfer_spec	Any	Yes	Contains receiver_host, receiver_init_port, receiver_alloc_port, req_id, is_last_prefill

Outputs

Name	Type	Description
(none)	None	KV data written to receiver via NIXL; ProxyNotif sent if is_last_prefill

Related Pages

Implements Principle

Principle:LMCache_LMCache_NIXL_KV_Transfer

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment