Implementation:Microsoft Onnxruntime CUDA ConvTransposeGrad
| Knowledge Sources | |
|---|---|
| Domains | Training, CUDA_Kernels |
| Last Updated | 2026-02-10 04:00 GMT |
Overview
Concrete tool for computing gradients of transposed convolution (deconvolution) operations in the ONNX Runtime CUDA training framework.
Description
Implements the ConvTransposeGrad operator for CUDA that computes gradients for the transposed convolution backward pass. The operator computes three optional gradient outputs: (1) input gradient dX using forward convolution via cuDNN (ComputeInputGradient), (2) weight gradient dW using backward filter convolution via cuDNN (ComputeWeightGradient), and (3) bias gradient dB via cuDNN backward bias (ComputeBiasGradient). The implementation uses AlgoIterator with cuDNN algorithm performance profiling to select optimal convolution algorithms. It delegates to shared convolution utilities (PrepareConvForwardArgs, PrepareConvBackwardFilterArgs) for descriptor setup. Registered for float, double, and MLFloat16 in kMSDomain version 1.
Usage
Invoked during the backward pass of training when the model contains transposed convolution (deconvolution) layers, commonly used in generative models and segmentation networks.
Code Reference
Source Location
- Repository: Microsoft_Onnxruntime
- File: orttraining/orttraining/training_ops/cuda/nn/conv_transpose_grad.cc
- Lines: 1-308
Signature
template <typename T>
class ConvTransposeGrad : public CudaKernel {
Status ComputeInternal(OpKernelContext* context) const;
Status ComputeInputGradient(onnxruntime::Stream* stream, const ConvArgs& args) const;
Status ComputeWeightGradient(onnxruntime::Stream* stream, const ConvArgs& args) const;
Status ComputeBiasGradient(const ConvArgs& args) const;
};
Import
#include "orttraining/training_ops/cuda/nn/conv_transpose_grad.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| dY | Tensor(T) | Yes | Gradient of loss with respect to convolution output |
| X | Tensor(T) | Yes | Original input tensor from forward pass |
| W | Tensor(T) | Yes | Convolution weight tensor from forward pass |
Outputs
| Name | Type | Description |
|---|---|---|
| dX | Tensor(T) | Gradient with respect to input (optional) |
| dW | Tensor(T) | Gradient with respect to weights (optional) |
| dB | Tensor(T) | Gradient with respect to bias (optional) |
Usage Examples
// Registration for float type
ONNX_OPERATOR_TYPED_KERNEL_EX(ConvTransposeGrad, kMSDomain, 1, float,
kCudaExecutionProvider,
(*KernelDefBuilder::Create())
.TypeConstraint("T", DataTypeImpl::GetTensorType<float>()),
ConvTransposeGrad<float>);