Implementation:OpenRLHF OpenRLHF Interactive Chat

Knowledge Sources	OpenRLHF
Domains	Inference, CLI, Generation
Last Updated	2026-02-07 10:40 GMT

Overview

Concrete tool for interactive text generation with a trained language model via a command-line REPL.

Description

The generate function loads a pretrained language model using OpenRLHF's Actor class and provides a read-eval-print loop (REPL) for interactive text generation. It supports chat template formatting, greedy and sampling-based decoding, conditional SFT prompts, 4-bit quantization via QLoRA, and configurable repetition penalty and temperature. The conversation history is maintained across turns when apply_chat_template is enabled.

Usage

Use this CLI tool for manually testing and evaluating a trained model's text generation quality. It is useful for debugging model outputs after SFT, DPO, KTO, or RLHF training, and supports both single-turn and multi-turn conversation modes.

Code Reference

Source Location

• Repository: OpenRLHF
• File: openrlhf/cli/interactive_chat.py
• Lines: 1-149

Signature

def generate(args) -> None:
    """
    Interactive text generation REPL.

    Args:
        args: Namespace with pretrain, attn_implementation, param_dtype,
              load_in_4bit, max_len, greedy_sampling, top_p, temperature,
              repetition_penalty, input_template, apply_chat_template,
              ta_prompt, enable_csft, csft_prompt, disable_fast_tokenizer
    """

Import

from openrlhf.cli.interactive_chat import generate

I/O Contract

Inputs

Name	Type	Required	Description
args.pretrain	str	Yes	HuggingFace model name or path
args.max_len	int	No	Maximum generation length (default: 4096)
args.greedy_sampling	bool	No	Use greedy decoding (default: False)
args.top_p	float	No	Nucleus sampling threshold (default: 0.9)
args.temperature	float	No	Sampling temperature (default: 0.2)
args.apply_chat_template	bool	No	Use HF chat template for multi-turn (default: False)
args.load_in_4bit	bool	No	Enable QLoRA 4-bit quantization (default: False)

Outputs

Name	Type	Description
stdout	str	Generated model responses printed to console

Usage Examples

Basic Interactive Chat

python -m openrlhf.cli.interactive_chat \
    --pretrain meta-llama/Llama-3-8b-chat \
    --apply_chat_template \
    --max_len 4096 \
    --temperature 0.7

# Then interactively type prompts:
# > Please enter a prompt (or type 'exit' to quit): What is RLHF?
# [Model generates response]
# > Please enter a prompt (or type 'exit' to quit): exit

With QLoRA and Greedy Decoding

python -m openrlhf.cli.interactive_chat \
    --pretrain ./ckpt/my_sft_model \
    --load_in_4bit \
    --greedy_sampling \
    --max_len 2048

Related Pages

• Environment:OpenRLHF_OpenRLHF_CUDA_GPU_Environment