Implementation:Tencent Ncnn GPU Instance And Device

Knowledge Sources	ncnn
Domains	GPU_Computing, Device_Management
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for initializing the Vulkan runtime, enumerating GPU devices, and querying hardware capabilities provided by the ncnn library.

Description

The ncnn Vulkan GPU management API consists of global functions and the GpuInfo class. create_gpu_instance initializes the Vulkan runtime (loading the driver, creating VkInstance, enumerating devices). get_gpu_count returns the number of available GPU devices. get_gpu_info returns a GpuInfo object with detailed hardware capabilities for a specific device index. Net::set_vulkan_device binds a network to a specific GPU.

The GpuInfo class exposes hardware properties including fp16 support, cooperative matrix (tensor core) support, subgroup operations, memory properties, and compute limits. These inform runtime decisions about precision and optimization strategies.

Usage

Call create_gpu_instance once at application startup. Query devices with get_gpu_count and get_gpu_info. Set the device on each Net instance with set_vulkan_device. Call destroy_gpu_instance at shutdown.

Code Reference

Source Location

• Repository: ncnn
• File: src/gpu.h (declarations), src/gpu.cpp (implementations)
• Lines: gpu.h:L21 (create_gpu_instance), gpu.h:L181-182 (get_gpu_count, get_default_gpu_index), gpu.h:L185-410 (GpuInfo class), gpu.cpp:L2523 (create_gpu_instance impl), gpu.cpp:L3141-3155 (get_gpu_count, get_default_gpu_index, get_gpu_info)

Signature

namespace ncnn {

// Initialize Vulkan runtime
// driver_path: optional custom Vulkan driver path (NULL for system default)
// return 0 on success
int create_gpu_instance(const char* driver_path = 0);

// Get global VkInstance (after create, before destroy)
VkInstance get_gpu_instance();

// Shutdown Vulkan runtime
void destroy_gpu_instance();

// Query available GPUs
int get_gpu_count();
int get_default_gpu_index();

// Get detailed GPU capabilities
const GpuInfo& get_gpu_info(int device_index);

// Get Vulkan device handle
const VulkanDevice* get_gpu_device(int device_index);

// GpuInfo provides hardware capability queries:
class GpuInfo
{
public:
    int device_index() const;
    VkPhysicalDevice physical_device() const;

    // Hardware capabilities
    bool support_fp16_packed() const;
    bool support_fp16_storage() const;
    bool support_fp16_arithmetic() const;
    bool support_int8_packed() const;
    bool support_int8_storage() const;
    bool support_int8_arithmetic() const;
    bool support_cooperative_matrix() const;
    bool support_cooperative_matrix_16_8_8() const;

    // Memory and compute limits
    uint32_t max_workgroup_count_x() const;
    uint32_t max_workgroup_count_y() const;
    uint32_t max_workgroup_count_z() const;
    uint32_t max_workgroup_size_x() const;
    // ...
};

// On Net: bind to specific GPU
void Net::set_vulkan_device(int device_index);
void Net::set_vulkan_device(const VulkanDevice* vkdev);

} // namespace ncnn

Import

#include "gpu.h"
#include "net.h"

I/O Contract

Inputs

Name	Type	Required	Description
driver_path	const char*	No	Custom Vulkan driver path (NULL for system default)
device_index	int	No	GPU index (0-based, default 0)

Outputs

Name	Type	Description
return (create_gpu_instance)	int	0 on success
get_gpu_count()	int	Number of available GPU devices
get_gpu_info()	const GpuInfo&	Hardware capabilities for a specific GPU

Usage Examples

Basic GPU Initialization

#include "gpu.h"
#include "net.h"

// Initialize Vulkan (once at startup)
ncnn::create_gpu_instance();

int gpu_count = ncnn::get_gpu_count();
if (gpu_count > 0)
{
    // Query GPU capabilities
    const ncnn::GpuInfo& info = ncnn::get_gpu_info(0);
    bool has_fp16 = info.support_fp16_arithmetic();

    // Set up network on GPU 0
    ncnn::Net net;
    net.opt.use_vulkan_compute = true;
    net.set_vulkan_device(0);

    // Load and run model...
}

// Cleanup (once at shutdown)
ncnn::destroy_gpu_instance();

Multi-GPU Selection

ncnn::create_gpu_instance();

int gpu_count = ncnn::get_gpu_count();
for (int i = 0; i < gpu_count; i++)
{
    const ncnn::GpuInfo& info = ncnn::get_gpu_info(i);
    fprintf(stderr, "GPU %d: %s\n", i, info.device_name());
}

// Use default GPU
int best_gpu = ncnn::get_default_gpu_index();
net.set_vulkan_device(best_gpu);

Related Pages

Implements Principle

• Principle:Tencent_Ncnn_Vulkan_GPU_Detection

Requires Environment

• Environment:Tencent_Ncnn_Vulkan_Environment