Implementation:NVIDIA TransformerEngine Userbuffers Header

Field	Value
Sources	TransformerEngine
Domains	Deep_Learning, Distributed_Computing
Last Updated	2026-02-07 14:00 GMT

Overview

Header file defining the communicator struct and API for the userbuffers inter-GPU communication system, which provides low-latency NVLink-based collective operations.

Description

userbuffers.h declares the core data structures and functions for TransformerEngine's custom high-performance communication layer that bypasses NCCL for intra-node NVLink transfers. The communicator struct holds per-rank state including GPU pointers, peer memory mappings, multicast handles, NVLink topology info, and operation counters.

Key definitions:

• communicator struct: Holds rank info (myrank, nranks, nvrank, nvsize), GPU memory pointers for up to NVTE_MAX_REGIONS (32) regions, peer pointer arrays, multicast handles, and launch mode configuration.
• ub_request struct: Tracks per-operation state for asynchronous communication including operation type, block size, offsets, and execution state counters.
• req_type enum: Enumerates supported operation types: sharp allreduce, send, non-sharp allreduce variants, and all-to-all.
• ExtAllgatherOp/ExtBarrierOp: Function types for external communication callbacks when not using MPI.

Usage

Include this header when implementing or extending the userbuffers communication layer. It is used internally by CommOverlapCore and related classes.

Code Reference

Source Location

Repository

NVIDIA/TransformerEngine

File

transformer_engine/common/comm_gemm_overlap/userbuffers/userbuffers.h

Lines

1--315

Signature

struct communicator {
  int myrank, nranks;
  int nvrank, nvsize;
  int free_region;
  int launch_mode;
  void *gpu_ptrs;
  int sms, threads;
  void *mem_ptr[NVTE_MAX_REGIONS];
  void **peer_ptr[NVTE_MAX_REGIONS];
  // ... additional fields for multicast, NVLink topology, etc.
};

using ExtAllgatherOp = std::function<void(void*, size_t, void*, size_t, ExtComm)>;
using ExtBarrierOp = std::function<void(ExtComm)>;

Import

#include "userbuffers.h"

I/O Contract

Inputs

Name	Type	Required	Description
N/A	N/A	N/A	This is a header file defining types and function declarations

Outputs

Name	Type	Description
N/A	N/A	Provides type definitions and function declarations

Usage Examples

#include "userbuffers.h"

// Check if built with MPI support
#ifdef NVTE_UB_WITH_MPI
  // MPI-based communicator creation
  communicator *comm;
  create_communicator_grouped2_mpi(&comm, 1, 1, tp_size, 1);
#else
  // External callback-based communicator creation
  communicator *comm;
  create_communicator_grouped2(&comm, rank, nranks, local_rank, local_size,
                               node_id, num_nodes, allgather_fn, barrier_fn,
                               1, 1, tp_size, 1);
#endif

Related Pages

• Environment:NVIDIA_TransformerEngine_CUDA_Toolkit_Requirements