Implementation:Microsoft Onnxruntime CPU DropoutGrad
| Knowledge Sources | |
|---|---|
| Domains | Training, CPU_Kernels |
| Last Updated | 2026-02-10 04:00 GMT |
Overview
Concrete tool for computing dropout gradient on CPU in the ONNX Runtime training framework.
Description
This file implements the DropoutGrad kernel and the TrainableDropoutGrad kernel. Both kernels compute the backward pass for dropout by applying the saved boolean mask from the forward pass to the upstream gradient. If an element was kept during the forward pass (mask is true), the gradient flows through scaled by 1/(1-ratio); otherwise it is zeroed out. The optional ratio input is read as a scalar tensor. If no mask is provided, the gradient passes through unchanged. The DropoutGrad kernel supports float and double types. TrainableDropoutGrad provides the same functionality for the opset 7 TrainableDropout variant.
Usage
This kernel is invoked during the backward pass when a Dropout node was present in the forward training graph. It uses the boolean mask produced by the forward Dropout to zero out gradients for dropped elements.
Code Reference
Source Location
- Repository: Microsoft_Onnxruntime
- File: orttraining/orttraining/training_ops/cpu/nn/dropout_op.cc
- Lines: 1-101
Signature
template <typename T>
Status DropoutGrad<T>::Compute(OpKernelContext* context) const;
template <typename T>
Status TrainableDropoutGrad<T>::Compute(OpKernelContext* context) const;
Import
#include "orttraining/orttraining/training_ops/cpu/nn/dropout_op.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| dY | Tensor(T) | Yes | Upstream gradient |
| mask | Tensor(bool) | No | Boolean dropout mask from forward pass |
| ratio | Tensor(float) | No | Dropout ratio (scalar) |
Outputs
| Name | Type | Description |
|---|---|---|
| dX | Tensor(T) | Gradient w.r.t. input, with dropped elements zeroed |
Usage Examples
ONNX_OPERATOR_TYPED_KERNEL_EX(
DropoutGrad, kMSDomain, 1, float, kCpuExecutionProvider,
KernelDefBuilder().TypeConstraint("T", DataTypeImpl::GetTensorType<float>()),
DropoutGrad<float>);
ONNX_OPERATOR_TYPED_KERNEL_EX(
TrainableDropoutGrad, kMSDomain, 1, float, kCpuExecutionProvider,
KernelDefBuilder().TypeConstraint("T", DataTypeImpl::GetTensorType<float>()),
TrainableDropoutGrad<float>);