Implementation:NVIDIA TransformerEngine Userbuffers Host

Field	Value
Sources	TransformerEngine
Domains	Deep_Learning, Distributed_Computing
Last Updated	2026-02-07 14:00 GMT

Overview

Implements the host-side setup and management of userbuffers communicators, including GPU memory allocation, peer-to-peer memory registration, and multi-node NVLink (MNNVL) fabric detection.

Description

userbuffers-host.cpp is the bootstrapping code for the userbuffers inter-GPU communication system. It creates communicator objects with MPI or external allgather/barrier callbacks, allocates contiguous CUDA memory using cuMemCreate for IPC sharing, exchanges memory handles via IPC sockets, and sets up multicast groups when NVLink fabric is available.

Key capabilities:

• Communicator creation: create_communicator_grouped2 and create_communicator_grouped2_mpi set up communicator state with rank topology information.
• MNNVL fabric detection: has_mnnvl_fabric checks runtime CUDA version and device attributes for multi-node NVLink support.
• GPU memory management: Allocates contiguous CUDA memory using cuMemCreate with IPC-shareable handles for peer-to-peer access.
• IPC socket exchange: Uses Unix domain sockets to exchange memory handles between ranks for shared memory setup.
• CPU affinity: Manages CPU affinity pinning per GPU rank for optimal NUMA locality.

Usage

This code is called internally during CommOverlapCore initialization to set up the low-latency communication infrastructure that underlies TransformerEngine's comm+GEMM overlap optimizations.

Code Reference

Source Location

Repository

NVIDIA/TransformerEngine

File

transformer_engine/common/comm_gemm_overlap/userbuffers/userbuffers-host.cpp

Lines

1--724

Signature

void create_communicator_grouped2(
    communicator **comm, int myrank, int numranks,
    int mylocal, int numlocal, int mynode, int numnodes,
    ExtAllgatherOp allgather, ExtBarrierOp barrier,
    int pipegpus, int pipenodes, int tensorgpus, int tensornodes);

void create_communicator_grouped2_mpi(
    communicator **comm, int pipegpus, int pipenodes,
    int tensorgpus, int tensornodes);

bool has_mnnvl_fabric(int device_id);

Import

#include "userbuffers.h"

I/O Contract

Inputs

Name	Type	Required	Description
myrank	int	Yes	Global rank of the current process
numranks	int	Yes	Total number of ranks
mylocal	int	Yes	Local rank within the node
numlocal	int	Yes	Number of local ranks per node
allgather	ExtAllgatherOp	Yes (non-MPI)	External allgather callback function
barrier	ExtBarrierOp	Yes (non-MPI)	External barrier callback function

Outputs

Name	Type	Description
comm	communicator**	Pointer to newly created communicator struct

Usage Examples

#include "userbuffers.h"

// With external callbacks (non-MPI path)
communicator *comm;
create_communicator_grouped2(&comm, rank, world_size,
                             local_rank, local_size, node_id, num_nodes,
                             allgather_fn, barrier_fn,
                             1, 1, tp_size, 1);

Related Pages

• Environment:NVIDIA_TransformerEngine_CUDA_Toolkit_Requirements