Implementation:Farama Foundation Gymnasium StickyAction

Knowledge Sources	Farama_Foundation_Gymnasium Gymnasium Docs
Domains	Reinforcement_Learning, Wrappers
Last Updated	2026-02-15 03:00 GMT

Overview

An action wrapper that adds a probability of repeating the previous action instead of executing the new one, implementing sticky actions as proposed by Machado et al. (2018).

Description

The StickyAction wrapper introduces stochasticity into the action selection process. With a configurable probability (repeat_action_probability), the wrapper replaces the agent's chosen action with the previous action. This follows the implementation proposed by Machado et al. (2018) in "Revisiting the Arcade Learning Environment" (Section 5.2).

Key features:

• Repeat probability -- A float in [0, 1) that controls how likely the previous action is repeated.
• Repeat duration -- Configurable as a fixed integer or a range (tuple of two ints). When a sticky sequence is triggered, the duration is randomly sampled from the given range.
• State tracking -- Maintains internal state for the last action taken, whether sticky actions are currently active, and the remaining repeat count.
• Reset behavior -- All sticky action state is cleared on environment reset.

No vector version of the wrapper exists.

Usage

Use this wrapper to add stochastic action repetition for evaluating agent robustness, particularly in Atari environments. This makes environments more challenging by reducing the agent's effective control frequency stochastically.

Code Reference

Source Location

• Repository: Farama_Foundation_Gymnasium
• File: gymnasium/wrappers/stateful_action.py

Signature

class StickyAction(gym.ActionWrapper[ObsType, ActType, ActType], gym.utils.RecordConstructorArgs):
    def __init__(
        self,
        env: gym.Env[ObsType, ActType],
        repeat_action_probability: float,
        repeat_action_duration: int | tuple[int, int] = 1,
    ): ...

Import

from gymnasium.wrappers import StickyAction

I/O Contract

Inputs

Name	Type	Required	Description
env	Env	Yes	The environment to wrap
repeat_action_probability	float	Yes	Probability in [0, 1) of repeating the previous action
repeat_action_duration	int or tuple[int, int]	No	Number of steps to repeat (fixed or range, default 1)

Outputs

Name	Type	Description
action	ActType	Either the agent's new action or the repeated previous action

Usage Examples

import gymnasium as gym
from gymnasium.wrappers import StickyAction

# 90% chance of repeating the previous action
env = gym.make("CartPole-v1")
env = StickyAction(env, repeat_action_probability=0.9)
obs, info = env.reset(seed=123)
obs, reward, terminated, truncated, info = env.step(1)

# With variable repeat duration (2-5 steps)
env = gym.make("CartPole-v1")
env = StickyAction(env, repeat_action_probability=0.25, repeat_action_duration=(2, 5))
obs, info = env.reset(seed=123)

Related Pages

• Environment:Farama_Foundation_Gymnasium_Python_3_10_Runtime