Heuristic:Farama Foundation Gymnasium Sync Vs Async VectorEnv Selection

Knowledge Sources	Gymnasium Gymnasium Vector Docs
Domains	Reinforcement_Learning, Optimization
Last Updated	2026-02-15 03:00 GMT

Overview

Decision framework for choosing between `SyncVectorEnv` (sequential, simple) and `AsyncVectorEnv` (multiprocessing, parallel) based on environment step cost.

Description

Gymnasium provides two vectorized environment implementations: `SyncVectorEnv` runs sub-environments sequentially in a for-loop within the main process, while `AsyncVectorEnv` runs each sub-environment in a separate process using Python multiprocessing. The choice between them significantly impacts training throughput. `SyncVectorEnv` has zero inter-process communication overhead but blocks on each environment step. `AsyncVectorEnv` pays a serialization/IPC cost per step but runs all environments truly in parallel, which dominates when individual `step()` calls are expensive.

Usage

Use this heuristic when setting up vectorized training and choosing between `gymnasium.vector.SyncVectorEnv` and `gymnasium.vector.AsyncVectorEnv`, or when passing `vectorization_mode` to `gymnasium.make_vec()`.

The Insight (Rule of Thumb)

Action: Use `SyncVectorEnv` for lightweight environments (Classic Control, Toy Text) and `AsyncVectorEnv` for heavyweight environments (MuJoCo, Box2D, custom environments with complex physics).
Value: Default vectorization mode is `"sync"`. Switch to `"async"` when individual `env.step()` takes more than ~1ms.
Trade-off: `AsyncVectorEnv` introduces multiprocessing overhead (pickling, IPC pipes, process spawn). For fast environments, this overhead outweighs the parallelism benefit.
Advanced option: `AsyncVectorEnv` supports a `worker` parameter for custom worker processes but the docstring warns: "This is an advanced mode option providing high flexibility but high chance to shoot yourself in the foot."

Reasoning

`SyncVectorEnv` executes environments in a Python for-loop, which means N environments take N times the single-environment step time. `AsyncVectorEnv` distributes environments across OS processes, so step time is approximately max(single_env_step) + IPC_overhead. When single-step time is very low (< 1ms), the IPC overhead of serializing observations through multiprocessing pipes dominates, making `SyncVectorEnv` faster. When single-step time is high (> 1ms), true parallelism wins.

The default in `gymnasium.make_vec()` is `"sync"` (line 849 of `registration.py`), which is the safer choice. Users should benchmark with `gymnasium.utils.performance.benchmark_step()` to make data-driven decisions.

Code Evidence

Default vectorization mode from `gymnasium/envs/registration.py:849`:

vectorization_mode = vectorization_mode or VectorizeMode.SYNC

SyncVectorEnv sequential execution from `gymnasium/vector/sync_vector_env.py:26-27`:

class SyncVectorEnv(VectorEnv):
    """Vectorized environment that serially runs multiple environments."""

AsyncVectorEnv worker warning from `gymnasium/vector/async_vector_env.py:127-130`:

# worker parameter docstring:
# "This is an advanced mode option providing high flexibility but
#  high chance to shoot yourself in the foot."

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment