Heuristic:Farama Foundation Gymnasium Shared Memory Vector Env Optimization
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, Optimization |
| Last Updated | 2026-02-15 03:00 GMT |
Overview
Use `shared_memory=True` in `AsyncVectorEnv` for large observations (images) to avoid serialization overhead, but disable for Graph/Sequence spaces.
Description
`AsyncVectorEnv` supports a `shared_memory` parameter that controls how observations are passed between worker processes and the main process. When `True`, observations are written directly to shared memory arrays (multiprocessing.Array), bypassing pickle serialization through pipes. This is significantly faster for large, fixed-shape observations like images. However, shared memory requires static shapes, so it cannot work with Graph or Sequence spaces which have dynamic shapes. Additionally, numpy arrays returned from `read_from_shared_memory` share the underlying buffer — mutations propagate bidirectionally, requiring explicit `np.copy()` to avoid side effects.
Usage
Use this heuristic when using AsyncVectorEnv with image observations or other large fixed-shape observation spaces. Disable shared memory when using Graph or Sequence spaces, or custom spaces not registered with the shared memory system.
The Insight (Rule of Thumb)
- Action: Set `shared_memory=True` in `AsyncVectorEnv` when observations are large numpy arrays with fixed shapes (e.g., image observations).
- Value: Default is `True` for standard spaces. Must be `False` for Graph and Sequence spaces.
- Trade-off: Shared memory avoids serialization overhead but the returned observations share the underlying buffer. Use `np.copy()` if you need to store observations across steps.
- Limitation: Custom `gymnasium.Space` subclasses raise `CustomSpaceError` with shared_memory=True. Register them or disable shared memory.
Reasoning
Without shared memory, each environment step serializes the full observation through multiprocessing pipes using pickle. For image observations (e.g., 84x84x4 stacked frames = 28KB per env), the serialization overhead is significant with many environments. Shared memory eliminates this by pre-allocating a fixed buffer that workers write to directly. The trade-off is that the returned numpy array is a view into the shared buffer, not a copy — the next `step()` call overwrites previous observations.
Graph and Sequence spaces have variable-length data structures (variable number of nodes/edges, variable sequence length), making it impossible to pre-allocate a fixed-size shared memory buffer.
Code Evidence
Graph/Sequence shared memory limitation from `gymnasium/vector/utils/shared_memory.py:102-107`:
@create_shared_memory.register(Graph)
@create_shared_memory.register(Sequence)
def _create_dynamic_shared_memory(space: Graph | Sequence, n: int = 1, ctx=mp):
raise TypeError(
f"As {space} has a dynamic shape so its not possible to make a static shared memory. For `AsyncVectorEnv`, disable `shared_memory`."
)
Shared memory aliasing warning from `gymnasium/vector/utils/shared_memory.py:116-119`:
# ..notes::
# The numpy array objects returned by `read_from_shared_memory` shares the
# memory of `shared_memory`. Any changes to `shared_memory` are forwarded
# to `observations`, and vice-versa. To avoid any side-effect, use `np.copy`.
Custom space error from `gymnasium/error.py:94-95`:
class CustomSpaceError(Error):
# "you can disable this feature with `shared_memory=False` however this is slower."