Principle:Farama Foundation Gymnasium Environment Interaction Loop
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, MDP |
| Last Updated | 2026-02-15 03:00 GMT |
Overview
The fundamental agent-environment interaction protocol in reinforcement learning where an agent observes state, takes actions, receives rewards, and transitions to new states through reset and step methods.
Description
The Environment Interaction Loop is the core protocol of the Markov Decision Process (MDP) framework implemented in code. An environment provides two primary methods:
- reset() initializes the environment to a starting state and returns the initial observation and info dictionary.
- step(action) advances the environment by one timestep given an agent action and returns a 5-tuple: (observation, reward, terminated, truncated, info).
The terminated flag indicates the agent reached a terminal MDP state (goal or failure). The truncated flag indicates the episode ended due to external constraints (time limit). When either flag is True, reset() must be called before further interaction.
This two-signal design (replacing the older single done flag) allows agents to correctly bootstrap value estimates: truncated episodes should not zero out the value function, while terminated episodes should.
Usage
Use this protocol for all interactions with Gymnasium environments. Every RL algorithm (Q-learning, policy gradient, actor-critic) follows this loop. The protocol is the same for both training and evaluation, differing only in action selection (exploratory vs. greedy).
Theoretical Basis
The interaction follows the MDP framework:
# Standard interaction loop (abstract)
obs, info = env.reset(seed=seed)
while not done:
action = agent.select_action(obs)
obs, reward, terminated, truncated, info = env.step(action)
agent.learn(obs, reward, terminated, truncated)
done = terminated or truncated
The critical distinction between terminated and truncated:
- terminated = True: MDP reached a natural end state. Bootstrap target = 0.
- truncated = True: Episode artificially ended (e.g., time limit). Bootstrap target = V(s').