Implementation:Microsoft Onnxruntime CPU ConvGrad
| Knowledge Sources | |
|---|---|
| Domains | Training, CPU_Kernels |
| Last Updated | 2026-02-10 04:00 GMT |
Overview
Concrete tool for computing convolution gradients on CPU in the ONNX Runtime training framework.
Description
This file implements the ConvGrad kernel, which computes gradients for convolution operations during the backward pass. It supports 1D, 2D, and 3D convolutions. The implementation uses im2col/col2im transformations along with GEMM operations for efficient gradient computation. For the weight gradient (dW), it performs im2col on the input and multiplies by the upstream gradient. For the input gradient (dX), it multiplies the transposed weights by the upstream gradient and then applies col2im. The bias gradient (dBias) is computed as the sum of the upstream gradient over batch and spatial dimensions. The kernel handles padding, strides, and dilation parameters. Group convolution is supported by splitting channels into groups.
Usage
This kernel is invoked during the backward pass when a convolution layer is present in the training graph. It computes gradients with respect to the input tensor, the convolution filters, and optionally the bias.
Code Reference
Source Location
- Repository: Microsoft_Onnxruntime
- File: orttraining/orttraining/training_ops/cpu/nn/conv_grad.cc
- Lines: 1-262
Signature
template <typename T>
Status ConvGrad<T>::Compute(OpKernelContext* context) const;
Import
#include "orttraining/orttraining/training_ops/cpu/nn/conv_grad.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| dY | Tensor(float) | Yes | Upstream gradient [N, C_out, spatial...] |
| X | Tensor(float) | Yes | Input tensor from forward pass [N, C_in, spatial...] |
| W | Tensor(float) | Yes | Convolution weights [C_out, C_in/group, kernel...] |
Outputs
| Name | Type | Description |
|---|---|---|
| dX | Tensor(float) | Gradient w.r.t. input (optional) |
| dW | Tensor(float) | Gradient w.r.t. weights (optional) |
| dBias | Tensor(float) | Gradient w.r.t. bias (optional) [C_out] |
Usage Examples
ONNX_OPERATOR_KERNEL_EX(
ConvGrad,
kMSDomain,
1,
kCpuExecutionProvider,
KernelDefBuilder().TypeConstraint("T", DataTypeImpl::GetTensorType<float>()),
ConvGrad<float>);