Implementation:Farama Foundation Gymnasium Performance Benchmark
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, Performance_Testing |
| Last Updated | 2026-02-15 03:00 GMT |
Overview
A collection of runtime performance benchmarking functions for measuring environment step throughput, initialization speed, and render performance.
Description
The performance module provides three benchmarking functions designed to measure different performance characteristics of Gymnasium environments:
benchmark_step measures the throughput of the environment's step() method by repeatedly sampling random actions and stepping the environment for a target duration (default 5 seconds). It handles episode resets automatically and returns the average steps per second. This is the primary metric for evaluating environment simulation speed.
benchmark_init measures how quickly an environment can be initialized and reset by repeatedly calling an environment factory function and performing the first reset. It returns the average initializations per second, which is useful for evaluating startup costs in vectorized environments.
benchmark_render measures the render throughput by repeatedly calling env.render() for a target duration and returning renders per second. It does not work with render_mode='human' since human rendering is typically frame-rate limited.
All three functions use a wall-clock time loop that runs for slightly over the specified target_duration seconds.
Usage
Use these functions when profiling environment performance, comparing implementations (e.g., original vs. JAX-accelerated), or diagnosing performance bottlenecks in training pipelines.
Code Reference
Source Location
- Repository: Farama_Foundation_Gymnasium
- File:
gymnasium/utils/performance.py
Signature
def benchmark_step(env: gymnasium.Env, target_duration: int = 5, seed=None) -> float
def benchmark_init(env_lambda: Callable[[], gymnasium.Env], target_duration: int = 5, seed=None) -> float
def benchmark_render(env: gymnasium.Env, target_duration: int = 5) -> float
Import
from gymnasium.utils.performance import benchmark_step, benchmark_init, benchmark_render
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| env | gymnasium.Env | Yes (step/render) | The environment to benchmark |
| env_lambda | Callable | Yes (init) | Factory function that creates and returns an environment |
| target_duration | int | No | Benchmark duration in seconds (default 5) |
| seed | int or None | No | Seed for env.reset() and action space (default None) |
Outputs
| Name | Type | Description |
|---|---|---|
| throughput | float | Average operations per second (steps/s, inits/s, or renders/s) |
Usage Examples
import gymnasium as gym
from gymnasium.utils.performance import benchmark_step, benchmark_init, benchmark_render
# Measure step throughput
env = gym.make("CartPole-v1")
steps_per_sec = benchmark_step(env, target_duration=5, seed=42)
print(f"Step throughput: {steps_per_sec:.0f} steps/s")
# Measure initialization speed
inits_per_sec = benchmark_init(lambda: gym.make("CartPole-v1"), target_duration=5)
print(f"Init throughput: {inits_per_sec:.1f} inits/s")
# Compare old vs new implementation
env_old = gym.make("CartPole-v0")
old_throughput = benchmark_step(env_old)
env_new = gym.make("CartPole-v1")
new_throughput = benchmark_step(env_new)
speedup = new_throughput / old_throughput
print(f"Speedup: {speedup:.2f}x")