Principle:Danijar Dreamerv3 Distributed Environment Execution

Knowledge Sources	DreamerV3
Domains	Reinforcement_Learning, Distributed_Systems, Environment
Last Updated	2026-02-15 09:00 GMT

Overview

Independent environment processes that each run a single environment instance, sending observations to the actor server and receiving actions via RPC, with automatic reconnection on failure.

Description

In distributed DreamerV3 training, each environment runs in its own OS process. The environment process:

1. Creates the environment from a deserialized factory function
2. Connects to the actor's BatchServer via portal.Client
3. Runs a step loop: call env.step(action) -> send observation to actor -> receive action
4. Handles disconnections by reconnecting and resetting the episode
5. The first environment (envid 0) additionally reports FPS and usage statistics to the logger

Each environment is tagged with an envid and is_eval flag, allowing the actor and replay to distinguish training from evaluation data.

Usage

Environment processes are spawned by combined() or launched as separate jobs via config.script == 'parallel_env'. This enables running environments on different machines from the GPU-equipped agent machine.

Theoretical Basis

Pseudo-code Logic:

# Abstract algorithm
env = make_env(envid)
actor_client = RPCClient(actor_addr)

done = True
while True:
    if done:
        action = random_action()
        action['reset'] = True

    obs = env.step(action)
    obs['envid'] = envid
    obs['is_eval'] = is_eval

    action = actor_client.act(obs)  # Send obs, receive action via RPC

Related Pages

Implemented By

• Implementation:Danijar_Dreamerv3_Parallel_Env