Environment:Vllm project Vllm NVIDIA CUDA
| Knowledge Sources | |
|---|---|
| Domains | GPU_Computing, NVIDIA_CUDA, CI_CD |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
NVIDIA CUDA environment specifically configured for vLLM's CI/CD test pipelines, providing NVIDIA GPU hardware, CUDA toolkit, nvidia-docker, and the test execution infrastructure required to validate vLLM's CUDA backend across multiple GPU architectures on every pull request and release.
Description
This environment defines the NVIDIA CUDA testing infrastructure used in vLLM's Buildkite CI pipelines. The test pipeline validates vLLM's correctness and performance across NVIDIA GPU generations (Ampere, Ada Lovelace, Hopper, and Blackwell) by running the full test suite on physical GPU hardware. The pipeline configuration specifies Docker images preconfigured with the CUDA toolkit, PyTorch, FlashInfer, and all vLLM dependencies. Tests are executed in nvidia-docker containers with GPU passthrough, ensuring that CUDA kernels, attention backends, quantization methods, and model architectures produce correct results on real hardware. The test matrix covers multiple dimensions: GPU architecture, dtype (FP16, BF16, FP8), attention backend (FlashAttention, FlashInfer, xFormers), quantization format (AWQ, GPTQ, Marlin, FP8), and model family (LLaMA, Mistral, GPT-NeoX, etc.). Pipeline results are reported as GitHub commit status checks required for merge.
Usage
The NVIDIA CUDA test pipeline is triggered on every pull request and on merges to the main branch. The pipeline configuration (.buildkite/test-pipeline.yaml or equivalent) defines build steps with Docker images, test commands, GPU agent queue selectors, parallelism settings, and retry policies. Buildkite agents equipped with NVIDIA GPUs execute test steps in isolated Docker containers. The nvidia-docker runtime provides GPU device passthrough, CUDA library mounting, and GPU memory isolation between concurrent pipeline runs.
Requirements
| Requirement | Value |
|---|---|
| NVIDIA GPU | A100, H100, L40, or equivalent test hardware |
| CUDA Toolkit | 12.x (matching vLLM's VLLM_MAIN_CUDA_VERSION) |
| NVIDIA Driver | >= 535 |
| nvidia-docker | NVIDIA Container Toolkit (nvidia-docker2) for GPU passthrough |
| Docker | Docker Engine for containerized test execution |
| Buildkite Agent | Self-hosted agent on NVIDIA GPU node |
| Test Docker Image | CUDA-based image with vLLM and all test dependencies |
| Operating System | Linux (Ubuntu 20.04+ with NVIDIA kernel modules) |
| CI Platform | Buildkite with GitHub webhook integration |