Principle:Farama Foundation Gymnasium Asynchronous Vector Execution
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, Parallelism |
| Last Updated | 2026-02-15 03:00 GMT |
Overview
A vectorization strategy that runs multiple environment instances in parallel via multiprocessing, maximizing throughput for computationally expensive environments.
Description
Asynchronous Vector Execution spawns each sub-environment in a separate worker process, communicating via pipes. This enables true parallel execution, where environment step calls happen simultaneously across CPU cores.
Key features:
- Shared memory: Observations are written directly to shared memory buffers, avoiding serialization overhead for large observations (e.g., images)
- True parallelism: Bypasses Python's GIL via multiprocessing
- Configurable context: Supports "fork", "spawn", or "forkserver" process creation contexts
The speedup is significant when T_env >> T_communication, making it ideal for physics simulations (MuJoCo, Box2D) and rendering-heavy environments.
Usage
Use asynchronous execution when environments have significant per-step computation cost (physics simulation, rendering). For lightweight tabular environments, the communication overhead may negate the parallelism benefit.
Theoretical Basis
Parallel execution of environments:
# Abstract algorithm
def async_step(workers, actions):
for worker, action in zip(workers, actions):
worker.send(action) # Non-blocking
results = [w.recv() for w in workers] # Blocking
return batch(results)
Total wall time:
Speedup over synchronous: when