Principle:Facebookresearch Habitat lab Interactive Session Execution

Knowledge Sources	Habitat 3.0 Habitat-Lab
Domains	Human_in_the_Loop, Interactive_Systems
Last Updated	2026-02-15 02:00 GMT

Overview

The main execution loop for human-in-the-loop applications, orchestrating frame-by-frame simulation updates driven by user interaction and AI agent actions.

Description

Interactive Session Execution runs the real-time simulation loop for HITL applications. In headed mode, it uses the Magnum rendering engine to display the 3D scene and capture user input. In headless mode, it runs without a display and serves remote clients via WebSocket. Each frame processes user inputs, computes AI agent actions, steps the simulation, and renders the result.

Usage

This is the top-level entry point for any HITL application. It is called after all configuration and setup is complete.

Theoretical Basis

The execution follows a standard game loop pattern:

# Abstract execution loop
while session_active:
    user_input = capture_input()        # GUI or WebSocket
    ai_actions = policy.act(obs)        # AI controller
    sim_state = env.step(actions)       # Physics simulation
    render(sim_state, camera)           # Visual output
    record(sim_state, user_input)       # Data collection

The loop supports two modes:

• Headed: Local windowed application with Magnum rendering
• Headless: Server mode with WebSocket client connections

Related Pages

Implemented By

• Implementation:Facebookresearch_Habitat_lab_Hitl_main