Environment:Ggml org Ggml Vulkan GPU Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, GPU_Computing |
| Last Updated | 2026-02-10 07:40 GMT |
Overview
Cross-platform Vulkan GPU environment requiring the Vulkan SDK with glslc shader compiler, supporting cooperative matrix and integer dot product extensions for accelerated inference.
Description
This environment provides GPU acceleration via the Vulkan graphics/compute API. It is the most portable GPU backend, supporting NVIDIA, AMD, Intel, and Apple GPUs across Linux, Windows, and macOS. The backend compiles GLSL compute shaders to SPIR-V at build time via a standalone shader generator tool. Optional Vulkan extensions provide additional performance for matrix operations.
Usage
Use this environment for cross-platform GPU acceleration when CUDA (NVIDIA-only) or Metal (Apple-only) is not available. It is particularly useful for AMD GPUs on Linux/Windows and as an alternative GPU backend on macOS alongside Metal.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux, Windows, or macOS | Cross-platform |
| GPU | Any Vulkan-capable GPU | NVIDIA, AMD, Intel, or Apple |
| Build System | CMake 3.19+ | Required by Vulkan backend CMake |
| SDK | Vulkan SDK with glslc | Shader compiler required at build time |
Dependencies
System Packages
- Vulkan SDK (with development headers and `glslc`)
- Vulkan-capable GPU driver
Optional Vulkan Extensions
- `GL_KHR_cooperative_matrix` (faster matrix multiplication)
- `GL_NV_cooperative_matrix2` (NVIDIA cooperative matrix v2)
- `GL_EXT_integer_dot_product` (integer dot product acceleration)
- `GL_EXT_bfloat16` (BF16 data type support)
Credentials
No credentials are required.
Quick Install
# Install Vulkan SDK (Ubuntu)
sudo apt-get install vulkan-tools libvulkan-dev glslc
# Build GGML with Vulkan backend
cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release
Code Evidence
Vulkan SDK detection from `src/ggml-vulkan/CMakeLists.txt:1-9`:
cmake_minimum_required(VERSION 3.19)
find_package(Vulkan COMPONENTS glslc REQUIRED)
Extension detection at build time from `src/ggml-vulkan/CMakeLists.txt:65-87`:
# GL_KHR_cooperative_matrix
execute_process(COMMAND ${Vulkan_GLSLC_EXECUTABLE}
--target-env=vulkan1.2 -o /dev/null ...
RESULT_VARIABLE VK_COOPMAT_RESULT)
# GL_NV_cooperative_matrix2
execute_process(COMMAND ${Vulkan_GLSLC_EXECUTABLE}
--target-env=vulkan1.2 -o /dev/null ...
RESULT_VARIABLE VK_COOPMAT2_RESULT)
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `Vulkan package not found` | Vulkan SDK not installed | Install Vulkan SDK from LunarG or system package manager |
| `glslc not found` | Shader compiler missing | Install `glslc` (part of Vulkan SDK or `shaderc` package) |
| `vkCreateDevice failed` | GPU driver does not support Vulkan | Update GPU driver to latest version with Vulkan support |
Compatibility Notes
- NVIDIA GPUs: Full support including cooperative matrix extensions for optimized matrix operations.
- AMD GPUs: Supported via RADV (open-source) or AMDVLK drivers on Linux.
- Intel GPUs: Supported via ANV driver on Linux.
- macOS: Supported via MoltenVK (Vulkan-over-Metal translation layer).
- COOPMAT2: Can be disabled at runtime with `GGML_VK_DISABLE_COOPMAT2=1` environment variable.