Implementation:Isaac sim IsaacGymEnvs Rl Games Runner Integration

Knowledge Sources	IsaacGymEnvs rl_games
Domains	Training, Integration
Last Updated	2026-02-15 00:00 GMT

Overview

Wrapper documentation for integrating IsaacGymEnvs with the rl_games training framework through environment, agent, model, and observer registration.

Description

The integration is split across two files. train.py contains the build_runner() function that orchestrates all registrations and builds the Runner. rlgames_utils.py provides the wrapper classes: RLGPUEnv (vectorized environment adapter), RLGPUAlgoObserver (metrics observer), and MultiObserver (observer aggregator). Together, these components plug IsaacGymEnvs' GPU-accelerated environments into rl_games' factory system.

Type

Wrapper Doc

Usage

Use this implementation when setting up the rl_games Runner for training on any IsaacGymEnvs task. The registration flow is invoked once at the start of every training run.

Code Reference

Source Location

• Repository: IsaacGymEnvs
• File: isaacgymenvs/train.py (Lines 186-215)
• File: isaacgymenvs/utils/rlgames_utils.py (Lines 130-296)

Signature

# From rl_games.torch_runner
class Runner:
    def __init__(self, algo_observer=None):
        ...
    def run(self, args):
        ...

# From isaacgymenvs/utils/rlgames_utils.py
class RLGPUEnv(vecenv.IVecEnv):
    """Vectorized environment wrapper for GPU-accelerated Isaac Gym tasks."""
    def __init__(self, config_name, num_actors, **kwargs):  # L242-296
        ...

class RLGPUAlgoObserver(AlgoObserver):
    """Observer that collects per-episode metrics from GPU env infos."""
    def __init__(self):  # L130
        ...
    def after_init(self, algo):  # L135
        ...
    def process_infos(self, infos, done_indices):  # L143
        ...
    def after_print_stats(self, frame, epoch_num, total_time):  # L178
        ...

class MultiObserver(AlgoObserver):
    """Aggregates multiple AlgoObserver instances."""
    def __init__(self, observers_):  # L212-239
        ...

# From isaacgymenvs/utils/rlgames_utils.py
def get_rlgames_env_creator(
    seed,
    task_config,
    task_name,
    sim_device,
    rl_device,
    graphics_device_id,
    headless,
    multi_gpu,
    post_create_hook,
    virtual_screen_capture,
    force_render,
):
    ...

# From isaacgymenvs/train.py (nested function)
def build_runner(algo_observer):
    ...

Import

from rl_games.torch_runner import Runner
from rl_games.common import vecenv, env_configurations
from rl_games.common.algo_observer import AlgoObserver
from isaacgymenvs.utils.rlgames_utils import RLGPUEnv, RLGPUAlgoObserver, MultiObserver, get_rlgames_env_creator

I/O Contract

Inputs

Name	Type	Required	Description
algo_observer	AlgoObserver	Yes	Observer instance (typically RLGPUAlgoObserver or MultiObserver) attached to the Runner
config_name	str	Yes	Configuration key used by rl_games to look up the environment creator
num_actors	int	Yes	Number of parallel environment instances (matches num_envs in task config)
seed	int	Yes	Random seed for environment reproducibility
task_config	dict	Yes	Hydra-composed task configuration dictionary
task_name	str	Yes	Name of the Isaac Gym task (e.g., 'Ant', 'Humanoid')
sim_device	str	Yes	Device for physics simulation (e.g., 'cuda:0')
rl_device	str	Yes	Device for RL computations (e.g., 'cuda:0')
graphics_device_id	int	Yes	GPU device ID for rendering (-1 for headless)
headless	bool	Yes	Whether to run without a display

Outputs

Name	Type	Description
runner	Runner	Fully initialized rl_games Runner with agent built, network compiled, and optimizer set up
RLGPUEnv instance	RLGPUEnv	Vectorized environment wrapping the Isaac Gym task, created on-demand by the Runner

Registration Flow

The following sequence shows the complete registration process executed inside build_runner():

# 1. Register the RLGPU vectorized environment type
vecenv.register(
    'RLGPU',
    lambda config_name, num_actors, **kwargs: RLGPUEnv(config_name, num_actors, **kwargs)
)

# 2. Register environment configuration with the env creator
env_configurations.register(
    'rlgpu',
    {
        'vecenv_type': 'RLGPU',
        'env_creator': lambda **kwargs: create_rlgpu_env(**kwargs),
    }
)

# 3. Build the Runner with the algo observer
runner = Runner(algo_observer=algo_observer)

# 4. Register custom agents for AMP tasks
runner.algo_factory.register_builder(
    'amp_continuous',
    lambda **kwargs: AMPAgent(**kwargs)
)

# 5. Register custom models and networks
from rl_games.common import model_builder
model_builder.register_model('continuous_amp', AMPModelBuilder)
model_builder.register_network('amp', AMPNetworkBuilder)

# 6. Set the configuration and run
runner.load(rlg_config_dict)
runner.run({'train': True, 'play': False})

Usage Examples

Standard Training Setup

from isaacgymenvs.utils.rlgames_utils import RLGPUAlgoObserver, MultiObserver
from isaacgymenvs.utils.wandb_utils import WandbAlgoObserver
from rl_games.torch_runner import Runner

# Build observer stack
gpu_observer = RLGPUAlgoObserver()
wandb_observer = WandbAlgoObserver(cfg)
observer = MultiObserver([gpu_observer, wandb_observer])

# Build and run
runner = build_runner(algo_observer=observer)
runner.run({'train': True})

Related Pages

• Isaac_sim_IsaacGymEnvs_RL_Agent_Initialization - implements - Principle describing the registration pattern.
• Isaac_sim_IsaacGymEnvs_CommonAgent_Train - next step - The training loop that executes after initialization.
• Isaac_sim_IsaacGymEnvs_WandbAlgoObserver_Logging - related - Observer logging implementation registered during this flow.

Principle:Isaac_sim_IsaacGymEnvs_RL_Agent_Initialization

• Environment:Isaac_sim_IsaacGymEnvs_Pip_Dependencies