Implementation:NVIDIA DALI NVML Utilities Header

Knowledge Sources	NVIDIA_DALI
Domains	Utilities, GPU_Management
Last Updated	2026-02-08 16:00 GMT

Overview

Declares the NVML (NVIDIA Management Library) wrapper API for GPU device management, error handling, driver version queries, CPU affinity, and hardware decoder detection within DALI.

Description

The NVML utilities header in dali/util/nvml.h defines the C++ API used by DALI to interact with the NVIDIA Management Library. It provides error handling infrastructure through the NvmlError exception class, which translates NVML return codes into human-readable error messages covering over 20 distinct error conditions. The companion NvmlBadAlloc exception class handles memory allocation failures. A template specialization of cudaResultCheck for nvmlReturn_t maps NVML status codes to the appropriate exception types.

The dali::nvml namespace provides lifecycle management through Init(), Shutdown(), and the RAII-based NvmlInstance class, which initializes NVML on construction and shuts it down on destruction with proper move semantics. Thread safety is ensured via a shared mutex accessed through Mutex(). The header declares GetNVMLAffinityMask and SetCPUAffinity for managing CPU-GPU affinity, and provides GetDriverVersion() and GetCudaDriverVersion() as cached queries for driver version information.

For CUDA 11 and later, the header defines a DeviceProperties struct (containing brand type and compute capability) and functions for querying device info and hardware decoder availability. The HasHwDecoder functions check whether Tesla-class GPUs with compute capability 8.0 or 9.0 are present, which is required for hardware-accelerated video decoding. The isHWDecoderSupported function provides a convenient single-call check combining NVML initialization status and hardware decoder detection.

Usage

Include this header when you need to query GPU properties, manage NVML lifecycle, set CPU affinity for optimal GPU locality, check driver versions, or determine hardware decoder availability. Use NvmlInstance for RAII-based NVML lifecycle management in scoped contexts.

Code Reference

Source Location

• Repository: NVIDIA_DALI
• File: dali/util/nvml.h
• Lines: 1-305

Signature

class NvmlError : public std::runtime_error {
 public:
  explicit NvmlError(nvmlReturn_t result, const char *details = nullptr);
  static const char *ErrorString(nvmlReturn_t result);
  nvmlReturn_t result() const;
};

class NvmlBadAlloc : public CUDABadAlloc {};

namespace nvml {

inline std::mutex& Mutex();
inline void Init();
void GetNVMLAffinityMask(cpu_set_t *mask, size_t num_cpus);
void SetCPUAffinity(int core = -1);
inline void Shutdown();

class NvmlInstance {
 public:
  static NvmlInstance CreateNvmlInstance();
  explicit NvmlInstance(bool init = false);
  ~NvmlInstance();
};

inline bool HasCuda11NvmlFunctions();
inline float GetDriverVersion();
inline int GetCudaDriverVersion();

struct DeviceProperties {
  nvmlBrandType_t type;
  int cap_major;
  int cap_minor;
};

inline DeviceProperties GetDeviceInfo(int device_idx);
inline bool HasHwDecoder(int device_idx);
inline bool HasHwDecoder();
inline bool isHWDecoderSupported();

}  // namespace nvml

Import

#include "dali/util/nvml.h"

I/O Contract

Inputs

Name	Type	Required	Description
result	nvmlReturn_t	Yes (NvmlError)	NVML return code to translate into an error message
core	int	No (SetCPUAffinity)	CPU core index to pin to; -1 uses NVML-recommended affinity
device_idx	int	Yes (GetDeviceInfo, HasHwDecoder)	CUDA device index to query
mask	cpu_set_t*	Yes (GetNVMLAffinityMask)	Output CPU affinity bitmask
num_cpus	size_t	Yes (GetNVMLAffinityMask)	Total number of CPUs in the system

Outputs

Name	Type	Description
return value (GetDriverVersion)	float	NVIDIA driver version as a floating point number (cached)
return value (GetCudaDriverVersion)	int	CUDA driver version integer (cached)
return value (GetDeviceInfo)	DeviceProperties	Brand type and compute capability of the specified GPU
return value (HasHwDecoder)	bool	True if the device (or any device) has a hardware decoder
return value (isHWDecoderSupported)	bool	True if NVML is initialized and hardware decoding is available

Usage Examples

RAII NVML Lifecycle Management

#include "dali/util/nvml.h"

{
  auto nvml = dali::nvml::NvmlInstance::CreateNvmlInstance();
  // NVML is now initialized

  float driver_ver = dali::nvml::GetDriverVersion();
  int cuda_ver = dali::nvml::GetCudaDriverVersion();

  // NVML automatically shuts down when nvml goes out of scope
}

Checking Hardware Decoder Support

#include "dali/util/nvml.h"

dali::nvml::Init();

if (dali::nvml::isHWDecoderSupported()) {
  // Hardware video decoding is available
  auto props = dali::nvml::GetDeviceInfo(0);
  // props.type, props.cap_major, props.cap_minor
}

dali::nvml::Shutdown();

Setting CPU Affinity for GPU Locality

#include "dali/util/nvml.h"

dali::nvml::Init();

// Set CPU affinity based on NVML recommendation for current GPU
dali::nvml::SetCPUAffinity();

// Or pin to a specific core
dali::nvml::SetCPUAffinity(4);

dali::nvml::Shutdown();

Related Pages

• Environment:NVIDIA_DALI_CUDA_GPU_Environment