Implementation:NVIDIA TransformerEngine Recipe C API

Field	Value
Sources	TransformerEngine
Domains	Deep_Learning, Optimization
Last Updated	2026-02-07 14:00 GMT

Overview

Declares the C API for FP8 scaling factor management, including delayed scaling recipe updates, amax computation, scale-from-amax derivation, and partial amax/cast operations for blockwise and MXFP8 scaling modes.

Description

recipe.h provides the core infrastructure for FP8 training recipes that manage dynamic scaling factors to keep FP8 tensors within representable range:

• Delayed scaling: nvte_delayed_scaling_recipe_amax_and_scale_update rotates amax history, reduces it via "max" or "most_recent" algorithms, and computes new scales targeting 2^(-margin) * max_fp8.
• Bulk update: nvte_delayed_scaling_recipe_amax_and_scale_update_after_reduction handles post-allreduce updates across multiple tensors.
• Current scaling: nvte_compute_amax and nvte_compute_scale_from_amax provide online amax-to-scale conversion.
• Partial operations: nvte_fp8_block_scaling_compute_partial_amax and nvte_mxfp8_compute_partial_amax support incremental blockwise quantization.

Usage

Use in the training loop to update FP8 scaling factors between iterations. Both delayed scaling (history-based) and current scaling (online) workflows are supported.

Code Reference

Source Location

Repository

NVIDIA/TransformerEngine

File

transformer_engine/common/include/transformer_engine/recipe.h

Lines

1--316

Signature

void nvte_delayed_scaling_recipe_amax_and_scale_update(
    const NVTETensor amax_history, const NVTETensor scale,
    NVTETensor updated_amax_history, NVTETensor updated_scale,
    const char* amax_compute_algo, NVTEDType fp8_dtype,
    float margin, cudaStream_t stream);

void nvte_compute_amax(const NVTETensor input, NVTETensor output,
                       cudaStream_t stream);

void nvte_compute_scale_from_amax(NVTETensor output,
                                  const NVTEQuantizationConfig config,
                                  cudaStream_t stream);

Import

#include "transformer_engine/recipe.h"

I/O Contract

Inputs

Name	Type	Required	Description
amax_history	NVTETensor	Yes	History of max absolute values [history_length, num_scales]
scale	NVTETensor	Yes	Current scaling factors [num_scales]
amax_compute_algo	const char*	Yes	"max" or "most_recent"
fp8_dtype	NVTEDType	Yes	Target FP8 data type
margin	float	Yes	Scaling factor margin

Outputs

Name	Type	Description
updated_amax_history	NVTETensor	Rotated and updated amax history
updated_scale	NVTETensor	Updated scaling factors

Usage Examples

#include "transformer_engine/recipe.h"

// Update FP8 scaling factors using delayed scaling recipe
nvte_delayed_scaling_recipe_amax_and_scale_update(
    amax_history, scale,
    updated_amax_history, updated_scale,
    "max", kNVTEFloat8E4M3, /*margin=*/0.0f, stream);

Related Pages

• Environment:NVIDIA_TransformerEngine_CUDA_Toolkit_Requirements