Implementation:OpenRLHF OpenRLHF Get strategy

Knowledge Sources	OpenRLHF DeepSpeed Documentation
Domains	Distributed_Computing, Training_Infrastructure
Last Updated	2026-02-07 00:00 GMT

Overview

Concrete tool for creating a distributed training strategy provided by OpenRLHF.

Description

The get_strategy function is a factory that creates a DeepspeedStrategy instance from parsed CLI arguments. It extracts seed, determinism flags, gradient clipping norms, batch sizes, and ZeRO stage from the argument namespace, then constructs the strategy object. This function is called at the beginning of every training script (SFT, DPO, RM, KD, PPO).

Usage

Import and call this function immediately after parsing CLI arguments and before any model or data operations. It is the mandatory first step in all OpenRLHF training workflows.

Code Reference

Source Location

• Repository: OpenRLHF
• File: openrlhf/utils/utils.py
• Lines: L25-37

Signature

def get_strategy(args):
    """
    Factory function to create a DeepspeedStrategy from CLI args.

    Args:
        args: argparse.Namespace containing:
            - seed (int): Random seed (default 42)
            - full_determinism (bool): Enable full determinism (default False)
            - max_norm (float): Max gradient norm for clipping (default 1.0)
            - micro_train_batch_size (int): Per-GPU batch size (default 1)
            - train_batch_size (int): Global batch size (default 128)
            - zero_stage (int): DeepSpeed ZeRO stage (0, 1, 2, or 3)

    Returns:
        DeepspeedStrategy: Initialized strategy object
    """

Import

from openrlhf.utils.utils import get_strategy

I/O Contract

Inputs

Name	Type	Required	Description
args	argparse.Namespace	Yes	Parsed CLI arguments containing training configuration

Outputs

Name	Type	Description
strategy	DeepspeedStrategy	Initialized distributed training strategy

Usage Examples

Standard Usage

import argparse
from openrlhf.utils.utils import get_strategy

# Parse CLI args (simplified)
parser = argparse.ArgumentParser()
parser.add_argument("--zero_stage", type=int, default=2)
parser.add_argument("--micro_train_batch_size", type=int, default=4)
parser.add_argument("--train_batch_size", type=int, default=128)
args = parser.parse_args()

# Create strategy
strategy = get_strategy(args)

# Initialize distributed backend
strategy.setup_distributed()

Related Pages

Implements Principle

• Principle:OpenRLHF_OpenRLHF_Distributed_Strategy_Configuration

Requires Environment

• Environment:OpenRLHF_OpenRLHF_CUDA_GPU_Environment