Environment:Facebookresearch Habitat lab HITL Runtime Environment

Knowledge Sources	Habitat-Lab habitat-hitl README
Domains	Infrastructure, Interactive_Systems
Last Updated	2026-02-15 00:00 GMT

Overview

Python 3.9+ runtime environment with WebSocket, async HTTP, and Hydra configuration for Human-in-the-Loop (HITL) interactive applications.

Description

This environment provides the runtime stack for HITL interactive evaluation and data collection applications. It extends the core Habitat-Lab environment with WebSocket support for remote client communication, asynchronous HTTP for availability endpoints, and Hydra for experiment configuration. HITL applications support both headed (GUI desktop with OpenGL) and headless (server-only with WebSocket) modes.

Usage

Use this environment for running any HITL interactive application including the basic viewer, pick-throw VR, rearrangement tasks, and custom evaluation tools. It is the mandatory prerequisite for running `hitl_main()`, `LabDriver`, `AppService`, and `SessionRecorder` implementations.

System Requirements

Category	Requirement	Notes
OS	Linux or macOS	Windows not officially supported
Hardware	GPU recommended	For habitat-sim rendering; headless mode possible
Display	X11/Wayland (headed mode)	Not required for headless server mode
Network	Open port for WebSocket server	Default configured in hitl_defaults.yaml

Dependencies

System Packages

`python` >= 3.9
OpenGL libraries (for headed/GUI mode)
`habitat-sim` with rendering support

Python Packages

`websockets` (WebSocket server for remote clients)
`aiohttp` (async HTTP server for availability endpoints)
`hydra-core` (configuration management)
All habitat-lab core dependencies

Credentials

No credentials are required for the base HITL runtime. Applications using S3 upload (e.g., rearrange_v2) may require:

`AWS_ACCESS_KEY_ID`: For S3 checkpoint/session data upload.
`AWS_SECRET_ACCESS_KEY`: Companion to the access key ID.

Quick Install

# Install HITL dependencies
pip install websockets aiohttp hydra-core

# Or install habitat-hitl from source
cd habitat-hitl && pip install -e .

Code Evidence

HITL requirements from `habitat-hitl/requirements.txt`:

websockets
aiohttp
hydra-core

Python version classifiers from `habitat-hitl/setup.py`:

"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",

Common Errors

Error Message	Cause	Solution
`ImportError: No module named 'websockets'`	websockets not installed	`pip install websockets`
`OSError: [Errno 98] Address already in use`	WebSocket port conflict	Change the port in hitl_defaults.yaml or kill the conflicting process
`RuntimeError: Cannot create GUI application in headless mode`	No display available	Use headless server mode or set `DISPLAY` environment variable

Compatibility Notes

Headed mode: Requires a display server (X11 or Wayland) and OpenGL support. Uses Magnum for rendering.
Headless mode: Runs as a WebSocket server without a display. Suitable for remote evaluation.
Python 3.12+: Known issue with IntFlag behavior in `user_mask.py` that uses XOR workaround.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment