Implementation:Bitsandbytes foundation Bitsandbytes XPU Backend Ops
| Knowledge Sources | |
|---|---|
| Domains | XPU_Backend, Quantization, Kernel_Dispatch |
| Last Updated | 2026-02-07 13:31 GMT |
Overview
Intel XPU backend kernel registration module that dispatches quantization and dequantization operations to SYCL native libraries, Triton kernels, or PyTorch fallbacks.
Description
This module implements the bitsandbytes operation dispatch for Intel XPU (GPU) devices. It uses a three-tier fallback strategy: (1) SYCL native library — preferred path using compiled C++ SYCL kernels via the bitsandbytes native library for dequantize_4bit, dequantize_blockwise, and gemv_4bit; (2) Triton kernels — used for quantize_blockwise, quantize_4bit, and optimizer operations even when SYCL is available, and as complete fallback when SYCL is not; (3) PyTorch default — logged warning fallback when neither SYCL nor Triton is available. The module also registers int8_linear_matmul using torch._int_mm for PyTorch >= 2.9.
Usage
This module is automatically loaded when an XPU device is detected. Users interact with it through the standard bitsandbytes functional API (e.g., bitsandbytes.functional.dequantize_4bit). The dispatch to XPU happens transparently.
Code Reference
Source Location
- Repository: bitsandbytes
- File: bitsandbytes/backends/xpu/ops.py
- Lines: 1-242
Signature
# Key registered kernels:
@register_kernel("bitsandbytes::int8_linear_matmul", "xpu")
def _(A: torch.Tensor, B: torch.Tensor) -> torch.Tensor: ...
@register_kernel("bitsandbytes::dequantize_4bit", "xpu")
def _(A, absmax, blocksize, quant_type, shape, dtype) -> torch.Tensor: ...
@register_kernel("bitsandbytes::dequantize_blockwise", "xpu")
def _(A, absmax, code, blocksize, dtype) -> torch.Tensor: ...
@register_kernel("bitsandbytes::gemv_4bit", "xpu")
def _(A, B, shapeB, absmax, code, blocksize) -> torch.Tensor: ...
Import
# Auto-registered on import when XPU device is present.
# Used via the standard functional API:
import bitsandbytes.functional as F
F.dequantize_4bit(tensor, quant_state) # dispatches to XPU kernel
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| A | torch.Tensor | Yes | Quantized tensor (uint8 for 4-bit/8-bit) |
| absmax | torch.Tensor | Yes | Per-block scaling factors |
| blocksize | int | Yes | Quantization block size |
| quant_type | str | Yes | "nf4" or "fp4" |
| shape | Sequence[int] | Yes | Target output shape (for 4-bit) |
| dtype | torch.dtype | Yes | Target output dtype (float16, bfloat16, float32) |
Outputs
| Name | Type | Description |
|---|---|---|
| output | torch.Tensor | Dequantized tensor on XPU device |
Usage Examples
XPU 4-bit Inference
import torch
import bitsandbytes as bnb
# With Intel XPU available, operations dispatch automatically
model = AutoModelForCausalLM.from_pretrained(
"model_name",
quantization_config=bnb.BitsAndBytesConfig(load_in_4bit=True),
device_map="xpu",
)
# All dequantize_4bit calls use the XPU SYCL kernels internally