Implementation:NVIDIA TransformerEngine Cast Transpose Noop API

Field	Value
Sources	TransformerEngine
Domains	Deep_Learning, Optimization
Last Updated	2026-02-07 14:00 GMT

Overview

Declares C API functions for transpose and cast-transpose operations that support a no-op flag, allowing kernels to be conditionally skipped at runtime without host-side branching.

Description

cast_transpose_noop.h exposes two extern "C" functions:

• nvte_transpose_with_noop: Transposes the input tensor unless the noop single-element tensor has a non-zero value, in which case the kernel exits immediately.
• nvte_cast_transpose_with_noop: Casts and transposes the input tensor with the same noop behavior.

This no-op mechanism enables efficient conditional execution within CUDA graph captures and optimizer pipelines where certain operations may need to be dynamically skipped (e.g., gradient overflow detection) without breaking graph structure or requiring CPU-GPU synchronization.

Usage

Use when implementing gradient scaling pipelines where the cast-transpose may need to be conditionally skipped based on a gradient overflow flag that resides in device memory.

Code Reference

Source Location

Repository

NVIDIA/TransformerEngine

File

transformer_engine/common/include/transformer_engine/cast_transpose_noop.h

Lines

1--44

Signature

void nvte_transpose_with_noop(const NVTETensor input, const NVTETensor noop,
                              NVTETensor output, cudaStream_t stream);

void nvte_cast_transpose_with_noop(const NVTETensor input, const NVTETensor noop,
                                   NVTETensor output, cudaStream_t stream);

Import

#include "transformer_engine/cast_transpose_noop.h"

I/O Contract

Inputs

Name	Type	Required	Description
input	NVTETensor	Yes	Input tensor to be cast/transposed
noop	NVTETensor	Yes	Single-element tensor; non-zero value skips the kernel
stream	cudaStream_t	Yes	CUDA stream for the operation

Outputs

Name	Type	Description
output	NVTETensor	Transposed (and optionally cast) output, or unchanged if noop is set

Usage Examples

#include "transformer_engine/cast_transpose_noop.h"

// Conditionally cast-transpose based on overflow detection
// noop_flag is set to non-zero by the grad scaler on overflow
nvte_cast_transpose_with_noop(grad_input, noop_flag, fp8_output, stream);

Related Pages

• Environment:NVIDIA_TransformerEngine_CUDA_Toolkit_Requirements