Implementation:NVIDIA TransformerEngine Debug Log FP8 Stats
| Field | Value |
|---|---|
| Sources | TransformerEngine |
| Domains | Deep_Learning, PyTorch, Debug, Quantization |
| Last Updated | 2026-02-07 14:00 GMT |
Overview
Logs statistics of FP8 quantized tensors, supporting cross-recipe comparison (delayed scaling, current scaling, MXFP8, block scaling) within the Transformer Engine debug framework.
Description
The LogFp8TensorStats feature collects quantization statistics on FP8 tensors during training. It supports computing statistics for the current recipe, but also allows hypothetical "what-if" comparisons by casting tensors to different FP8 recipes. Available statistics include underflow percentage, overflow percentage (delayed scaling only), scale inverse min/max, and mean squared error (MSE). The feature is micro-batch aware, accumulates statistics between debug_api.step() calls, and supports distributed reduction across ranks.
Usage
Enable via YAML configuration under the transformer_engine.LogFp8TensorStats section. Use freq greater than 1 to reduce overhead from extra casts. Statistics are flushed at each debug_api.step() call.
Code Reference
Source Location
- Repository
NVIDIA/TransformerEngine- File
transformer_engine/debug/features/log_fp8_tensor_stats.py- Lines
- 1--367
Signature
@Registry.register_feature(namespace="transformer_engine")
class LogFp8TensorStats(BaseLogTensorStats):
def check_if_stat_is_supported(self, stat: str, current_recipe: str) -> bool: ...
def get_recipe_from_stat(self, stat: str, default_recipe: str = "") -> Tuple[str, bool]: ...
def inspect_tensor_enabled(self, config, layer_name, tensor_name, iteration) -> Tuple[bool, Optional[int]]: ...
def inspect_tensor(self, config, layer_name, tensor_name, iteration, tp_group, tensor, rowwise_quantized_tensor=None, columnwise_quantized_tensor=None, quantizer=None) -> None: ...
Import
from transformer_engine.debug.features.log_fp8_tensor_stats import LogFp8TensorStats
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| config | Dict | Yes | Must contain stats list; optionally freq, start_step, end_step, start_end_list
|
| layer_name | str | Yes | Name of the TE layer |
| tensor_name | str | Yes | One of activation, weight, gradient
|
| tensor | torch.Tensor | Yes | Original high-precision tensor |
| quantizer | Quantizer | Yes | The quantizer used for this tensor (must not be None) |
| rowwise_quantized_tensor | QuantizedTensor | Yes | The quantized tensor (must be a QuantizedTensor) |
Outputs
| Name | Type | Description |
|---|---|---|
| (none) | None | Statistics are buffered and logged at the next debug_api.step()
|
Usage Examples
# YAML configuration:
# example_fp8_tensor_stat_collection:
# enabled: True
# layers:
# layer_types: [layernorm_linear]
# transformer_engine:
# LogFp8TensorStats:
# enabled: True
# tensors_struct:
# - tensor: activation
# stats: [mxfp8_underflows%]
# freq: 1
# - tensor: gradient
# stats: [underflows%]
# freq: 5
# start_step: 0
# end_step: 80