Implementation:Facebookresearch Habitat lab Habitat Lab Tutorial
| Knowledge Sources | |
|---|---|
| Domains | Embodied_AI, Tutorials, Navigation_Tasks |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
The Habitat_Lab tutorial is a comprehensive Jupyter notebook (in percent-format Python) that demonstrates the core Habitat Lab API including PointNav task setup, RL training with PPO, creating custom tasks, defining custom sensors, and building custom agents.
Description
This tutorial covers the foundational concepts of Habitat Lab through a series of progressively complex examples:
1. PointNav Task Setup: Configures and runs a PointNav navigation task using the Habitat test scenes. Demonstrates environment creation, action stepping with env.step(), and observation display (RGB, semantic, depth) via matplotlib. Shows both interactive and automated control using a list of valid actions.
2. RL Training: Sets up and runs a short PPO training session using habitat_baselines. Configures the trainer via baseline_registry.get_trainer(), sets random seeds, and trains for a configurable number of steps. References TensorBoard for visualization.
3. Key Concepts: Documents the seven core Habitat abstractions:
- HabitatSim - thin wrapper over habitat_sim
- Env - universe of agent, task, and simulator
- RLEnv - extends Env for reinforcement learning
- EmbodiedTask - task definition with observation/action spaces
- Dataset - episode dataset wrapper
- Measure - task metrics (e.g., SPL)
- habitat_baselines - baseline implementations
4. Custom Task (TestNav-v0): Creates NewNavigationTask extending NavigationTask with a custom _check_episode_is_active() that logs agent position and checks for collisions or stop actions.
5. Custom Sensor: Registers AgentPositionSensor that returns the agent's 3D position as a float32 observation. Demonstrates sensor registration via @registry.register_sensor, config integration via LabSensorConfig, and observation access via obs["agent_position"].
6. Custom Agent: Implements ForwardOnlyAgent extending habitat.Agent that moves forward until within a distance threshold of the goal, then calls stop. Demonstrates the agent interface (act() and reset() methods).
Usage
This tutorial is the primary introduction to Habitat Lab for new users. It should be run as a Jupyter notebook or executed cell-by-cell. Requires the MP3D example scene dataset (auto-downloadable) and habitat-baselines.
Code Reference
Source Location
- Repository: Facebookresearch_Habitat_lab
- File: examples/tutorials/nb_python/Habitat_Lab.py
- Lines: 1-414
Signature
# Key classes defined in the tutorial:
def display_sample(rgb_obs, semantic_obs=np.array([]), depth_obs=np.array([])):
...
@registry.register_task(name="TestNav-v0")
class NewNavigationTask(NavigationTask):
def __init__(self, config, sim, dataset):
...
def _check_episode_is_active(self, *args, **kwargs):
...
@registry.register_sensor(name="agent_position_sensor")
class AgentPositionSensor(habitat.Sensor):
def __init__(self, sim, config, **kwargs):
...
def _get_uuid(self, *args, **kwargs):
...
def _get_sensor_type(self, *args, **kwargs):
...
def _get_observation_space(self, *args, **kwargs):
...
def get_observation(self, observations, *args, episode, **kwargs):
...
class ForwardOnlyAgent(habitat.Agent):
def __init__(self, success_distance, goal_sensor_uuid):
...
def reset(self):
...
def act(self, observations):
...
Import
import habitat
from habitat.core.logging import logger
from habitat.core.registry import registry
from habitat.sims.habitat_simulator.actions import HabitatSimActions
from habitat.tasks.nav.nav import NavigationTask
from habitat_baselines.common.baseline_registry import baseline_registry
from habitat_baselines.config.default import get_config as get_baselines_config
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| PointNav config | str | Yes | Path to pointnav_habitat_test.yaml configuration |
| MP3D example scene | files | Yes | 3D scene data (downloadable via habitat_sim.utils.datasets_download) |
| PPO config | str | Yes | Path to ppo_pointnav_example.yaml for RL training |
Outputs
| Name | Type | Description |
|---|---|---|
| Matplotlib plots | images | RGB, semantic, and depth observation visualizations |
| Custom task | registered class | NewNavigationTask registered as "TestNav-v0" |
| Custom sensor | registered sensor | AgentPositionSensor registered as "agent_position_sensor" |
| Trained model | checkpoint | PPO model trained for the configured number of steps |
Usage Examples
Basic Usage
import habitat
from habitat.core.registry import registry
from habitat.tasks.nav.nav import NavigationTask
# Setup PointNav environment
config = habitat.get_config(
config_path="habitat-lab/habitat/config/benchmark/nav/pointnav/pointnav_habitat_test.yaml",
overrides=[
"habitat.environment.max_episode_steps=10",
"habitat.environment.iterator_options.shuffle=False",
],
)
env = habitat.Env(config=config)
# Run episode
obs = env.reset()
while not env.episode_over:
obs = env.step({"action": "move_forward"})
print(env.get_metrics())
env.close()
# Create a custom sensor
@registry.register_sensor(name="agent_position_sensor")
class AgentPositionSensor(habitat.Sensor):
def __init__(self, sim, config, **kwargs):
super().__init__(config=config)
self._sim = sim
def _get_uuid(self, *args, **kwargs):
return "agent_position"
def _get_sensor_type(self, *args, **kwargs):
return habitat.SensorTypes.POSITION
def _get_observation_space(self, *args, **kwargs):
from gym import spaces
import numpy as np
return spaces.Box(
low=np.finfo(np.float32).min,
high=np.finfo(np.float32).max,
shape=(3,),
dtype=np.float32,
)
def get_observation(self, observations, *args, episode, **kwargs):
return self._sim.get_agent_state().position