Implementation:Ggml org Ggml Vulkan backend api

Metadata

Field	Value
Page Type	Implementation (API Doc)
Knowledge Sources	GGML
Domains	ML_Infrastructure, Tensor_Computing, GPU_Computing
Last Updated	2025-05-15 12:00 GMT

Overview

Public C header declaring the Vulkan GPU backend interface for running tensor operations on GPUs that support the Vulkan graphics/compute API.

Description

ggml-vulkan.h declares the Vulkan backend's public API (29 lines). It provides:

1. Constants: GGML_VK_NAME = "Vulkan" and GGML_VK_MAX_DEVICES = 16.
2. Backend initialization: ggml_backend_vk_init(dev_num) creates a backend for a specific Vulkan device by index.
3. Type checking: ggml_backend_is_vk(backend) verifies if a backend is Vulkan-based.
4. Device enumeration:
   • ggml_backend_vk_get_device_count -- returns the number of Vulkan-capable devices
   • ggml_backend_vk_get_device_description -- gets a human-readable device name
   • ggml_backend_vk_get_device_memory -- queries free and total device memory
5. Buffer types:
   • ggml_backend_vk_buffer_type(dev_num) -- device-specific buffer type for GPU memory
   • ggml_backend_vk_host_buffer_type() -- pinned host buffer for faster CPU-GPU transfers
6. Registration: ggml_backend_vk_reg() returns the registration handle for auto-discovery.

All functions are marked with GGML_BACKEND_API and wrapped in extern "C".

Usage

Include this header to use the Vulkan backend for cross-platform GPU inference. The backend works on any Vulkan-capable GPU including NVIDIA, AMD, Intel, and Apple (via MoltenVK).

Code Reference

Source Location

GGML repo, file: include/ggml-vulkan.h (29 lines).

Signatures

#define GGML_VK_NAME "Vulkan"
#define GGML_VK_MAX_DEVICES 16

GGML_BACKEND_API ggml_backend_t ggml_backend_vk_init(size_t dev_num);
GGML_BACKEND_API bool ggml_backend_is_vk(ggml_backend_t backend);
GGML_BACKEND_API int  ggml_backend_vk_get_device_count(void);
GGML_BACKEND_API void ggml_backend_vk_get_device_description(int device, char * description, size_t description_size);
GGML_BACKEND_API void ggml_backend_vk_get_device_memory(int device, size_t * free, size_t * total);
GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_vk_buffer_type(size_t dev_num);
GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_vk_host_buffer_type(void);
GGML_BACKEND_API ggml_backend_reg_t ggml_backend_vk_reg(void);

Import

#include "ggml-vulkan.h"

I/O Contract

Inputs

Parameter	Type	Required	Description
dev_num	size_t	Yes	Vulkan device index (0-based) for initialization and buffer type queries.
device	int	Yes	Device index for description and memory queries.
backend	ggml_backend_t	Yes	Backend handle for type checking.
description	char *	Yes	Output buffer for device description string.
description_size	size_t	Yes	Size of the output description buffer.

Outputs

Output	Type	Description
Backend handle	ggml_backend_t	Initialized Vulkan backend for the specified device.
Type check	bool	true if the backend is Vulkan-based.
Device count	int	Number of Vulkan-capable devices in the system.
Buffer type	ggml_backend_buffer_type_t	Device or host-pinned buffer type handle.
Device memory	via output params	Free and total memory on the specified device.

Usage Examples

#include "ggml-vulkan.h"

// Enumerate Vulkan devices
int n_devices = ggml_backend_vk_get_device_count();

for (int i = 0; i < n_devices; i++) {
    char desc[256];
    ggml_backend_vk_get_device_description(i, desc, sizeof(desc));
    printf("Device %d: %s\n", i, desc);

    size_t free_mem, total_mem;
    ggml_backend_vk_get_device_memory(i, &free_mem, &total_mem);
    printf("  Memory: %zu / %zu MB\n", free_mem / (1024*1024), total_mem / (1024*1024));
}

// Initialize the first Vulkan device
ggml_backend_t backend = ggml_backend_vk_init(0);

Related Pages

• Principle:Ggml_org_Ggml_Backend_Interface
• Principle:Ggml_org_Ggml_Vulkan_GPU_Computation
• Implementation:Ggml_org_Ggml_Vulkan_backend
• Principle:Ggml_org_Ggml_Backend_Initialization