Implementation:Deepspeedai DeepSpeed Conversion Utils

Knowledge Sources	DeepSpeed
Domains	Type_Conversion, CUDA_Kernels, Precision_Handling
Last Updated	2026-02-09 00:00 GMT

Overview

Templated type conversion utilities providing efficient device-side conversions between floating-point and integer types with CUDA intrinsics.

Description

The conversion_utils.h header provides a comprehensive type conversion system using C++ template specialization to handle conversions between various numeric types including float, double, __half, __nv_bfloat16, and signed/unsigned integers (8/16/32/64-bit). The implementation prioritizes CUDA intrinsics when available (controlled by PTX_AVAILABLE flag) for optimal performance, with fallback to standard C++ conversions. The system supports identity conversions for all types, enabling generic template code where the destination type might match the source type. Special attention is given to precision-aware conversions, particularly for FP16/BF16 types common in deep learning workloads.

Usage

Use these utilities when implementing CUDA kernels that require type conversions between different numeric precisions, especially when working with mixed-precision training or quantization operations. The conversion::to<TO, FROM>(val) function provides a unified interface for all conversions, enabling generic code that works across different data types without explicit casting logic.

Code Reference

Source Location

• Repository: DeepSpeed
• File: csrc/includes/conversion_utils.h

Signature

namespace conversion {
    // Main conversion template
    template <typename TO, typename FROM>
    DS_D_INLINE TO to(FROM val);

    // Examples:
    // Float conversions
    template <> DS_D_INLINE float to(__half val);
    template <> DS_D_INLINE __half to(float val);

    // Integer conversions
    template <> DS_D_INLINE int32_t to(float val);
    template <> DS_D_INLINE float to(int32_t val);

    // Vector conversions
    template <> DS_D_INLINE float2 to(__half2 val);
    template <> DS_D_INLINE __half2 to(float2 val);

    #ifdef BF16_AVAILABLE
    template <> DS_D_INLINE __nv_bfloat16 to(float val);
    #endif
}

Import

#include "csrc/includes/conversion_utils.h"

I/O Contract

Input	Type	Description
val	FROM (template)	Value to convert (any supported numeric type)

Output	Type	Description
result	TO (template)	Converted value in target type

Usage Examples

Mixed-Precision Computation:

__global__ void mixed_precision_kernel(__half* input, float* output, int n) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < n) {
        __half h_val = input[idx];
        // Convert to float for computation
        float f_val = conversion::to<float>(h_val);
        float result = f_val * 2.0f;
        output[idx] = result;
    }
}

Generic Quantization:

template <typename T>
__device__ void quantize_value(T mem_value, int8_t* output) {
    // Always compute in float regardless of input type
    float compute_value = conversion::to<float>(mem_value);
    float scaled = compute_value * scale;
    int32_t quantized = conversion::to<int32_t>(scaled);
    *output = (int8_t)clamp(quantized, -128, 127);
}

BFloat16 Processing:

#ifdef BF16_AVAILABLE
__device__ void process_bf16(__nv_bfloat16* data, int idx) {
    __nv_bfloat16 bf16_val = data[idx];
    // Convert to float for precise computation
    float f_val = conversion::to<float>(bf16_val);
    f_val = fmaxf(f_val, 0.0f);  // ReLU
    // Convert back to bf16
    data[idx] = conversion::to<__nv_bfloat16>(f_val);
}
#endif

Related Pages

• Quantization Utils - Uses conversion utilities for quantization operations
• Dequantization Utils - Applies conversions during dequantization
• Memory Access Utils - Often used together for data type transformations