Environment:LMCache LMCache NIXL Transfer Library

Knowledge Sources	LMCache NIXL
Domains	Infrastructure, RDMA, Distributed_Systems
Last Updated	2026-02-09 00:00 GMT

Overview

NIXL (NVIDIA Inference eXtension Library) runtime for RDMA-based KV cache transfer between GPU instances, required for Disaggregated Prefill and P2P KV Cache Sharing.

Description

NIXL provides the high-performance RDMA transfer channel used by LMCache for cross-instance KV cache movement. It is a mandatory dependency for the Disaggregated Prefill workflow (prefill-to-decode KV transfer) and the P2P KV Cache Sharing workflow (cross-instance cache sharing). NIXL requires Python < 3.13 due to its dependency on numba, which constrains numpy to <= 2.2.6. The NIXL channel (`NixlChannel`) wraps the `nixl_agent` API for buffer registration, peer connection, and RDMA data transfer operations.

Usage

Use this environment when deploying Disaggregated Prefill (via `PDBackend`) or P2P KV Cache Sharing (via `P2PBackend` and `NixlStorageBackend`). NIXL is not required for local-only KV cache offloading or CacheBlend operations that do not cross instance boundaries. The NIXL channel is created via the `CreateTransferChannel` factory with `channel_type="nixl"`.

System Requirements

Category	Requirement	Notes
Hardware	NVIDIA GPU with RDMA support	InfiniBand or RoCE networking recommended
Python	>= 3.10, < 3.13	NIXL uses numba which requires Python < 3.13
Networking	High-bandwidth interconnect	RDMA (InfiniBand/RoCE) for optimal performance
OS	Linux	RDMA stack required

Dependencies

System Packages

RDMA drivers (InfiniBand verbs or RoCE)
NVIDIA GPU drivers with CUDA 12.x

Python Packages

`nixl` (conditional: `python_version < "3.13"`)
`numpy` <= 2.2.6 (constrained by numba)
`torch` (for GPU buffer management)

Credentials

No credentials required. Network configuration is handled via LMCache config parameters:

`pd_peer_host`: Hostname of peer instance for PD transfers.
`pd_peer_init_port`: Port for peer initialization.
`pd_peer_alloc_port`: Port for peer allocation.
`nixl_receiver_port`: Port for NIXL receiver (replaces deprecated `nixl_peer_port`).

Quick Install

# NIXL is installed automatically with lmcache on Python < 3.13
pip install lmcache

# Verify NIXL is available
python -c "from nixl._api import nixl_agent; print('NIXL available')"

Code Evidence

NIXL import guard from `lmcache/v1/transfer_channel/nixl_channel.py:603-608`:

try:
    from nixl._api import nixl_agent
except ImportError as err:
    raise RuntimeError("NIXL is not available") from err

Conditional NIXL dependency from `requirements/common.txt:10`:

# if nixl decides to support >=3.13 in the future, we can remove this constraint
nixl; python_version < "3.13"

Transfer channel factory from `lmcache/v1/transfer_channel/__init__.py:47-57`:

if channel_type == "nixl":
    from lmcache.v1.transfer_channel.nixl_channel import NixlChannel
    assert "backends" in kwargs, (
        "`backends` must be provided to create nixl transfer channel."
    )
    transfer_channel = NixlChannel(
        async_mode=async_mode, role=role, buffer_ptr=buffer_ptr,
        buffer_size=buffer_size, align_bytes=align_bytes,
        tp_rank=tp_rank, peer_init_url=peer_init_url,
        device=device, **kwargs,
    )

Deprecated NIXL config aliases from `lmcache/v1/config.py:37-43`:

_CONFIG_ALIASES = {
    "nixl_peer_host": "pd_peer_host",
    "nixl_peer_init_port": "pd_peer_init_port",
    "nixl_peer_alloc_port": "pd_peer_alloc_port",
    "nixl_proxy_host": "pd_proxy_host",
    "nixl_proxy_port": "pd_proxy_port",
    "nixl_buffer_size": "pd_buffer_size",
    "nixl_role": "pd_role",
}

Common Errors

Error Message	Cause	Solution
`RuntimeError: NIXL is not available`	NIXL not installed or Python >= 3.13	Use Python < 3.13 and ensure `nixl` is installed
`ImportError: nixl._api`	NIXL package installed but native library missing	Rebuild nixl with RDMA support
`AssertionError: backends must be provided`	Missing `nixl_backends` in config	Configure `nixl_backends` in the YAML config for NIXL storage
NIXL connection timeout	Network connectivity issue between instances	Verify RDMA/InfiniBand connectivity and firewall rules

Compatibility Notes

Python 3.13+: NIXL is unavailable. Use shared storage backends (Redis, filesystem) instead for cross-instance KV sharing.
Non-RDMA networks: NIXL can fall back to TCP, but performance will be significantly reduced.
Mock channel: For testing without NIXL, use `channel_type="mock_memory"` which provides an in-memory mock implementation.
Deprecated configs: All `nixl_*` config prefixes are deprecated in favor of `pd_*` prefixes (e.g., `nixl_peer_host` is now `pd_peer_host`).

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment