Principle:Ggml org Ggml Backend Interface
| Knowledge Sources | |
|---|---|
| Domains | Architecture, Plugin_System |
| Last Updated | 2026-02-10 |
Overview
The Backend Interface defines the abstract vtable-style contract that every hardware backend must implement to participate in GGML's pluggable computation system.
Description
GGML supports a wide variety of hardware accelerators -- discrete GPUs via CUDA, Vulkan, Metal, and SYCL, as well as CPU-only paths, NPUs, and DSP processors. To accommodate this diversity without coupling the core tensor library to any single vendor, GGML defines a Backend Interface: a set of C structs whose fields are function pointers (a vtable pattern). Any new backend implements this interface by filling in the function pointers and registering itself with the global backend registry. The core library then calls through these pointers without knowing which concrete backend is behind them.
The interface is layered into four levels:
ggml_backend_reg_i-- The registration interface. A backend plugin exports a registration function that returns aggml_backend_regcontaining an API version tag and pointers for enumerating devices and looking up extension functions viaget_proc_address.ggml_backend_device_i-- The device interface. Each physical or logical device reports its name, description, memory, type (CPU, GPU, iGPU, accelerator), and provides factory methods for creating backend streams and buffer types. It also exposessupports_opandsupports_buftso the scheduler can route operations.ggml_backend_i-- The stream (backend instance) interface. This is where computation happens. The backend providesgraph_compute, optional asynchronous tensor transfer (set_tensor_async,get_tensor_async), event recording, synchronization, and graph plan management.ggml_backend_buffer_type_i/ggml_backend_buffer_i-- The memory interface. Buffer types describe allocation alignment and sizing rules; buffers handle the actual memory operations:set_tensor,get_tensor,memset_tensor,cpy_tensor, andclear.
Each level is a plain C struct with an iface field of the corresponding _i struct type, a context void pointer for backend-private state, and any additional bookkeeping (e.g., api_version on ggml_backend_reg, guid on ggml_backend). This pattern is sometimes called a C vtable or manual polymorphism and is the standard approach for plugin systems in C libraries that cannot rely on C++ virtual dispatch.
Usage
Apply this principle when implementing a new hardware backend for GGML. The implementer fills in the function-pointer structs defined in ggml-backend-impl.h, writes the corresponding logic for buffer allocation, tensor data transfer, and graph computation on the target hardware, and then registers the backend with the GGML_BACKEND_DL_IMPL macro (for dynamic loading) or by calling ggml_backend_register at startup. This principle is also relevant when extending an existing backend with new capabilities such as event-based synchronization or custom extension functions exposed through get_proc_address.
Theoretical Basis
The Backend Interface follows the abstract interface / vtable pattern commonly used in C plugin architectures:
- Separation of Interface from Implementation -- The public header (
ggml-backend.h) defines opaque handle types (ggml_backend_t,ggml_backend_dev_t,ggml_backend_reg_t) and free functions that dispatch through the internal vtable. Consumers never see the_istructs; they call functions likeggml_backend_graph_computewhich internally dereferencebackend->iface.graph_compute. - Versioned API Contract -- The
ggml_backend_regstruct includes anapi_versionfield (currentlyGGML_BACKEND_API_VERSION 2). This allows the core library to detect and reject incompatible dynamically-loaded plugins, preventing undefined behavior from ABI mismatches. - Capability-Based Routing -- The device interface exposes
supports_op,supports_buft, andoffload_op. The backend scheduler uses these predicates to decide which device should execute each operation in a computation graph. This is a form of capability negotiation: the scheduler queries the interface rather than maintaining a static mapping of operations to backends. - Optional Function Pointers -- Many vtable slots are marked as optional (e.g.,
set_tensor_async,cpy_tensor,event_record). When a backend sets these toNULL, the core library falls back to synchronous or CPU-mediated paths. This keeps the minimum implementation burden low while allowing high-performance backends to opt into advanced features. - Dynamic Loading Support -- The macros
GGML_BACKEND_DL_IMPLandGGML_BACKEND_DL_SCORE_IMPLgenerate exported C entry points (ggml_backend_init,ggml_backend_score) that the dynamic loader uses to discover and rank backends at runtime.
This design is analogous to device-driver interfaces in operating system kernels, where each driver fills in a struct of function pointers and registers with a bus-specific subsystem.