Environment:Vllm project Vllm CUDA Runtime
| Knowledge Sources | |
|---|---|
| Domains | GPU_Computing, CUDA |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
NVIDIA CUDA runtime library environment for vLLM's next-generation SM100+ (Blackwell) kernels, providing the CUDA runtime API, CUTLASS 3.x template library, and Blackwell-specific features required by fused multi-head attention (FMHA) MLA reduction and tile scheduling kernels.
Description
This environment defines the CUDA runtime libraries and compilation requirements for vLLM's Blackwell-generation GPU kernels. SM100 (Blackwell, B100/B200/GB200) introduces fifth-generation Tensor Cores with native support for larger tile sizes, enhanced TMA (Tensor Memory Accelerator) with multicast capabilities, and the ability to issue asynchronous warpgroup-level MMA instructions with improved scheduling. vLLM's SM100-specific kernels include the FMHA (Fused Multi-Head Attention) MLA (Multi-Head Latent Attention) reduction kernel, which performs the final reduction step of multi-head latent attention across split-K partitions, and the MLA tile scheduler, which dynamically distributes attention tiles across thread blocks for load-balanced execution. These kernels are built using CUTLASS 3.x, NVIDIA's template library for high-performance linear algebra, which provides composable building blocks for TMA-based data movement, warpgroup MMA, and epilogue fusion.
Usage
These kernels are automatically compiled and selected when vLLM detects SM 10.0 (Blackwell) hardware at runtime. The CUTLASS 3.x headers must be available at build time (typically included as a submodule or downloaded during the build process). The CUDA toolkit version must support SM 10.0 target architectures. At runtime, the kernel selection logic in vLLM's attention backend automatically dispatches to the SM100-optimized path when a Blackwell GPU is detected, providing improved throughput for MLA-based models (e.g., DeepSeek-V2/V3) compared to the generic FlashAttention path.
Requirements
| Requirement | Value |
|---|---|
| GPU Hardware | NVIDIA Blackwell: B100, B200, GB200 |
| Compute Capability | SM 10.0+ |
| CUDA Toolkit | 12.8+ (with SM 10.0 support) |
| CUDA Runtime API | cudaRT for kernel launch, memory management, stream synchronization |
| CUTLASS | 3.x (template library for TMA, warpgroup MMA, epilogue fusion) |
| GPU Memory | 80+ GB HBM3e |
| NVLink | NVLink 5.0 (for multi-GPU Blackwell configurations) |
| Host Compiler | GCC >= 11 or Clang >= 14 (required by CUTLASS 3.x C++17 features) |
| CMake | >= 3.26.1 |