Implementation:Danijar Dreamerv3 Make Replay
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, Experience_Replay |
| Last Updated | 2026-02-15 09:00 GMT |
Overview
Concrete tool for constructing an experience replay buffer with configurable sampling strategies and disk-backed storage provided by the DreamerV3 embodied library.
Description
The make_replay() factory in dreamerv3/main.py computes the sequence length from batch parameters, creates a Replay instance with the configured capacity and chunk size, and optionally constructs a Mixture selector combining Uniform, Prioritized, and Recency sampling strategies based on configured fractions.
The Replay class in embodied/core/replay.py manages chunk-based storage with FIFO eviction, concurrent read/write access, disk persistence, and an optional online queue for fresh data.
Usage
Call make_replay(config, folder, mode) to create a replay buffer. The mode parameter controls capacity (eval mode uses 1/10 capacity) and sequence length (train uses batch_length, eval uses report_length).
Code Reference
Source Location
- Repository: dreamerv3
- File: dreamerv3/main.py (make_replay), embodied/core/replay.py (Replay class)
- Lines: dreamerv3/main.py L183-209, embodied/core/replay.py L14-394
Signature
def make_replay(config, folder, mode='train'):
"""
Factory function to create a Replay buffer.
Args:
config: elements.Config - Full configuration.
folder: str - Subdirectory name within logdir for chunk storage.
mode: str - 'train' or 'eval'. Controls capacity and sequence length.
Returns:
Replay: Configured replay buffer instance.
"""
class Replay:
def __init__(
self, length, capacity=None, directory=None, chunksize=1024,
online=False, selector=None, save_wait=False, name='unnamed', seed=0):
"""
Args:
length: int - Sequence length for each sample.
capacity: int - Maximum number of stored items (FIFO eviction).
directory: str - Disk path for chunk persistence.
chunksize: int - Number of timesteps per storage chunk.
online: bool - Enable online queue for fresh data.
selector: Selector - Sampling strategy (Uniform, Mixture, etc.).
save_wait: bool - Block until disk writes complete.
name: str - Buffer name for logging.
seed: int - Random seed for sampling.
"""
Import
from dreamerv3.main import make_replay
from embodied.core.replay import Replay
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| config | elements.Config | Yes | Configuration with replay.size, replay.chunksize, replay.online, replay.fracs, batch_size, batch_length, etc. |
| folder | str | Yes | Subdirectory name for chunk storage (e.g., 'replay', 'eval_replay') |
| mode | str | No | 'train' (default) or 'eval' — controls capacity and sequence length |
Outputs
| Name | Type | Description |
|---|---|---|
| replay | Replay | Buffer instance with add(step, worker), sample(batch, mode), update(data), save(), load() methods |
Usage Examples
Training Replay
from dreamerv3.main import make_replay
# Create training replay buffer (default 1M capacity)
replay = make_replay(config, 'replay', mode='train')
# Add transitions from environment
replay.add(transition_dict, worker=0)
# Sample a batch
batch = replay.sample(batch_size=16, mode='train')
# Update with new replay context from training
replay.update({'stepid': stepids, 'enc/deter': enc_deter, ...})
# Persist to disk
replay.save()
Evaluation Replay
# Create eval replay (1/10 capacity, report_length sequences)
replay_eval = make_replay(config, 'eval_replay', mode='eval')