Implementation:FMInference FlexLLMGen DeepSpeed Runtime Constants
| Field | Value |
|---|---|
| Sources | Repo: FlexLLMGen, Upstream: DeepSpeed |
| Domains | Configuration_Management, Runtime_Infrastructure |
| Last Updated | 2026-02-09 00:00 GMT |
Overview
Vendored DeepSpeed module that defines all configuration key names, default values, and format strings used across the DeepSpeed runtime configuration system.
Description
The constants.py file (421 lines) is a vendored copy of DeepSpeed's runtime constants module. It serves as the single source of truth for all configuration key strings and their default values, ensuring consistency between configuration parsing, validation, and documentation.
Key constant categories include:
- Routes -- ROUTE_TRAIN, ROUTE_EVAL, ROUTE_PREDICT, ROUTE_ENCODE for controlling execution paths.
- Batch size -- TRAIN_BATCH_SIZE, TRAIN_MICRO_BATCH_SIZE_PER_GPU, GRADIENT_ACCUMULATION_STEPS with their default values and format documentation strings.
- Sparse attention -- Constants for multiple attention sparsity modes (dense, fixed, variable, bigbird, bslongformer) with block sizes, local/global block counts, and layout parameters.
- Optimizer and scheduler -- OPTIMIZER, SCHEDULER, MAX_GRAD_NORM, type strings, and parameter group names.
- Precision -- BFLOAT16, FP16, AMP section keys with enabled flags and format documentation.
- ZeRO -- ZERO_ALLOW_UNTESTED_OPTIMIZER and related flags.
- Gradient handling -- SPARSE_GRADIENTS, gradient accumulation format strings.
Each constant follows a naming convention: FEATURE_NAME for the JSON key string, FEATURE_NAME_DEFAULT for the default value, and optionally FEATURE_FORMAT for a documentation/error string describing the expected JSON format.
Usage
These constants are imported by config.py and other runtime modules to ensure key string consistency. They are not used directly by end users. This module is part of the vendored benchmark dependencies in FlexLLMGen.
Code Reference
| Field | Value |
|---|---|
| Repository | FlexLLMGen |
| File | benchmark/third_party/DeepSpeed/deepspeed/runtime/constants.py |
| Lines | 1-421 |
| Type | AUTO_KEEP (vendored dependency) |
Example constant definitions:
ROUTE_TRAIN = "train"
ROUTE_EVAL = "eval"
ROUTE_PREDICT = "predict"
TRAIN_BATCH_SIZE = "train_batch_size"
TRAIN_BATCH_SIZE_DEFAULT = None
TRAIN_MICRO_BATCH_SIZE_PER_GPU = "train_micro_batch_size_per_gpu"
TRAIN_MICRO_BATCH_SIZE_PER_GPU_DEFAULT = None
GRADIENT_ACCUMULATION_STEPS = "gradient_accumulation_steps"
GRADIENT_ACCUMULATION_STEPS_DEFAULT = None
SPARSE_ATTENTION = "sparse_attention"
SPARSE_FIXED_MODE = "fixed"
SPARSE_BLOCK_DEFAULT = 16
I/O Contract
Inputs
This module defines only constants and does not accept inputs.
Outputs
| Output | Type | Description |
|---|---|---|
| Key strings | str | JSON configuration key names (e.g., "train_batch_size") |
| Default values | various | Default values for optional configuration parameters |
| Format strings | str | Documentation strings describing expected configuration formats |