Implementation:Facebookresearch Habitat lab Interactive Play

Knowledge Sources	Facebookresearch_Habitat_lab
Domains	Embodied_AI, Interactive_Simulation, Reinforcement_Learning
Last Updated	2026-02-15 00:00 GMT

Overview

The interactive_play.py script provides a PyGame-based interactive tool for manually controlling articulated agents (Fetch robot, Spot robot, or humanoid) in Habitat rearrangement environments, supporting velocity control, inverse kinematics end-effector control, trajectory recording/playback, and free camera mode.

Description

This script is the primary interactive debugging and demonstration tool for Habitat 2.0 rearrangement tasks. It creates a PyGame window that renders the agent's observations and allows real-time keyboard control of the agent. Key components include:

• get_input_vel_ctlr(): Translates PyGame keyboard inputs into environment actions. Supports three control modes: 7-joint velocity control (keys 1-7/Q-U), 4-joint velocity control for Spot, 10-joint velocity control for Stretch, and IK end-effector control (W/A/S/D/Q/E). Base movement uses I/J/K/L keys. Grasp/release is triggered with O/P keys.
• FreeCamHelper: A utility class enabling free camera movement independent of the agent. Toggled with Z key, it allows translating (W/A/S/D/Q/E) and rotating (I/J/K/L/U/O) the camera, with B to reset the camera position.
• play_env(): The main interaction loop that processes input, steps the environment, renders observations, and optionally records video or action trajectories. Supports humanoid control via HumanoidRearrangeController.

The script accepts numerous command-line arguments for configuration, including task selection (--cfg), rendering options (--no-render, --save-obs), action recording/playback (--save-actions, --load-actions), agent type selection (--control-humanoid, --use-humanoid-controller), and IK control toggling (--disable-inverse-kinematics). It automatically configures a third-person RGB camera for visualization.

Usage

Run this script directly from the command line to interactively control an articulated agent in a Habitat rearrangement environment. Useful for task debugging, demonstration recording, dataset verification, and manual testing of new task configurations.

Code Reference

Source Location

• Repository: Facebookresearch_Habitat_lab
• File: examples/interactive_play.py
• Lines: 1-803

Signature

def step_env(env, action_name, action_args):
    ...

def get_input_vel_ctlr(
    skip_pygame,
    cfg,
    arm_action,
    env,
    not_block_input,
    agent_to_control,
    control_humanoid,
    humanoid_controller,
):
    ...

class FreeCamHelper:
    def __init__(self):
        ...
    def update(self, env, step_result, update_idx):
        ...

def play_env(env, args, config):
    ...

Import

# This is a standalone script, typically run directly:
# python examples/interactive_play.py --cfg benchmark/rearrange/play/play.yaml

# Functions can be imported as:
from examples.interactive_play import play_env, FreeCamHelper, step_env

I/O Contract

Inputs

Name	Type	Required	Description
--cfg	str	No	Path to Habitat config YAML (default: benchmark/rearrange/play/play.yaml)
--no-render	bool	No	Run without PyGame rendering (headless mode)
--save-obs	bool	No	Save rendered observations as video
--save-obs-fname	str	No	Filename for saved video (default: play.mp4)
--save-actions	bool	No	Record action trajectory to file
--save-actions-count	int	No	Number of steps to record (default: 200)
--load-actions	str	No	Path to a previously recorded action trajectory to replay
--play-cam-res	int	No	Resolution of the third-person camera (default: 512)
--disable-inverse-kinematics	bool	No	Use velocity control instead of IK end-effector control
--control-humanoid	bool	No	Control a humanoid agent instead of a robot
--use-humanoid-controller	bool	No	Use the HumanoidRearrangeController for walking motion
--gfx	bool	No	Save a GFX replay file for visualization
--never-end	bool	No	Disable episode step limit

Outputs

Name	Type	Description
video file	MP4	Saved observation video (when --save-obs is used), written to data/vids/
action trajectory	numpy file	Saved action sequence (when --save-actions is used), written to data/interactive_play_replays/
GFX replay	string	Graphics replay data (when --gfx is used)

Usage Examples

Basic Usage

# Run with default configuration (Fetch robot, IK control)
# python examples/interactive_play.py

# Run with a specific task
# python examples/interactive_play.py --cfg benchmark/rearrange/close_cab.yaml

# Run with velocity control (no IK)
# python examples/interactive_play.py --disable-inverse-kinematics

# Record a video
# python examples/interactive_play.py --save-obs --save-obs-fname demo.mp4

# Record and replay actions
# python examples/interactive_play.py --save-actions --save-actions-count 200
# python examples/interactive_play.py --load-actions data/interactive_play_replays/play_actions.txt

# Control humanoid agent with walking controller
# python examples/interactive_play.py --control-humanoid --use-humanoid-controller

# Programmatic usage
import habitat
from examples.interactive_play import play_env

config = habitat.get_config("benchmark/rearrange/play/play.yaml")
with habitat.Env(config=config) as env:
    play_env(env, args, config)

Related Pages

• Facebookresearch_Habitat_lab_Rearrange_V2_UI