Workflow:Facebookresearch Habitat lab Custom Task Extension
| Knowledge Sources | |
|---|---|
| Domains | Embodied_AI, Framework_Extension, Task_Definition |
| Last Updated | 2026-02-15 02:00 GMT |
Overview
End-to-end process for extending Habitat-Lab with custom sensors, measures, actions, and tasks using the registry pattern and Hydra structured configuration system.
Description
This workflow covers how to add new functionality to Habitat-Lab from outside the source code using the decorator-based registry system. Custom components (sensors, measures, actions) are defined as Python classes, registered via `@habitat.registry.register_*` decorators, and wired into experiments through structured config dataclasses and Hydra YAML composition. This enables researchers to define novel observations, reward functions, action primitives, and complete tasks without modifying the core framework. The pattern follows a consistent define-register-configure cycle for all extension types.
Usage
Execute this workflow when you need to add a new observation sensor (e.g., a custom object detector), a new evaluation measure (e.g., a domain-specific reward), a new action primitive (e.g., strafe movement), or an entirely new embodied task to the Habitat-Lab framework.
Execution Steps
Step 1: Define the Component Class
Create a Python class that inherits from the appropriate base class: `habitat.Sensor` for sensors, `habitat.Measure` for measures, or `SimulatorTaskAction` for actions. Implement the required abstract methods that define the component's behavior (observation computation, metric update, action execution).
Key considerations:
- Sensors must implement `_get_uuid()`, `_get_sensor_type()`, `_get_observation_space()`, and `get_observation()`
- Measures must implement `_get_uuid()`, `reset_metric()`, and `update_metric()`
- Actions must implement `step()` which modifies the simulator state
- The UUID string is used for registry lookup and configuration binding
Step 2: Register with the Registry
Apply the appropriate registry decorator to the component class: `@habitat.registry.register_sensor`, `@habitat.registry.register_measure`, or `@habitat.registry.register_task_action`. The decorator registers the class under its class name (or an optional custom name) for runtime lookup.
Key considerations:
- Registration happens at import time when the module is loaded
- The registry uses string-based lookup from configuration to instantiate components
- Custom names can be provided via the `name` parameter of the decorator
- Ensure the module is imported before the environment is created
Step 3: Create Structured Config Dataclass
Define a dataclass that specifies the configuration schema for the new component. Sensor configs inherit from `LabSensorConfig`, measure configs from `MeasurementConfig`, and action configs from `ActionConfig`. Add typed fields for any parameters the component needs.
Key considerations:
- The `type` field in the config must match the registered name of the component
- Use Python type hints and default values for all configurable parameters
- The `MISSING` sentinel from OmegaConf marks required fields without defaults
- Config dataclasses enable type-safe configuration with IDE autocompletion
Step 4: Wire into Hydra Configuration
Register the structured config with Hydra's ConfigStore and compose it into an experiment configuration. This can be done programmatically via `get_config()` with overrides, or by referencing the config in YAML files. The new component appears in the task's sensor or measure list.
Key considerations:
- Use `habitat.get_config()` with the `overrides` parameter for programmatic configuration
- YAML composition allows adding components to existing experiment configs
- Multiple custom components can be composed together in a single experiment
- The Gym wrapper automatically includes registered sensors in the observation space
Step 5: Integration Testing
Create the environment with the custom configuration and verify the new component works correctly. For sensors, check the observation space and output shapes. For measures, verify metric computation across episode resets and steps. For actions, confirm the expected simulator state changes.
Key considerations:
- Use `habitat.Env` or `habitat.gym.make_gym_from_config` to test the environment
- Verify observation spaces match expected shapes and dtypes
- Test metric computation across multiple episode resets
- Confirm custom actions modify the agent or environment state correctly