Implementation:Facebookresearch Habitat lab ClientMessageManager

Knowledge Sources	Facebookresearch_Habitat_lab
Domains	Embodied_AI, Human_in_the_Loop, Client_Server_Communication
Last Updated	2026-02-15 00:00 GMT

Overview

Manages per-client server messages for the Human-in-the-Loop (HITL) framework, extending gfx-replay keyframes with client-specific rendering instructions such as highlights, lines, text, dialogs, outlines, camera transforms, and UI canvas updates.

Description

Deprecation Notice: The legacy UI classes (UIButton, UITextbox) within this module are deprecated (see source comment # TODO: Deprecated legacy UI system. at line 23). Prefer the UICanvasUpdate API via update_ui_canvas() for new code. See Facebookresearch_Habitat_lab_Warning_Deprecated_Legacy_UI_System.

The ClientMessageManager class acts as the central message broker between the HITL server and connected clients. Unlike keyframes (which are shared), messages are client-specific and indexed by user ID. The manager provides methods to:

• Draw visual primitives: add_highlight (circles), add_line, add_text, and draw_object_outline allow the server to annotate the 3D scene for specific clients.
• Show modal dialogs: show_modal_dialogue_box displays dialog boxes with buttons and optional textboxes.
• Control XR headset: rebase_xr_headset_position and set_xr_origin_transform manage VR/XR headset positioning.
• Manage viewports and cameras: set_viewport_properties, update_camera_transform, and set_object_visibility_layer control per-viewport rendering.
• Signal scene changes: signal_scene_change notifies clients of scene transitions.
• Update UI canvases: update_ui_canvas sends UI element updates to clients.
• Send navmesh data: update_navmesh_triangles transmits navigation mesh geometry.

All message methods accept a destination_mask parameter (of type Mask) to target specific users.

Usage

Use ClientMessageManager in HITL application states to communicate visual annotations, UI updates, and scene state changes to connected remote clients. It is typically instantiated by the HITL framework and accessed through the application service layer.

Code Reference

Source Location

• Repository: Facebookresearch_Habitat_lab
• File: habitat-hitl/habitat_hitl/core/client_message_manager.py
• Lines: 1-457

Signature

class ClientMessageManager:
    _messages: List[Message]
    _users: Users

    def __init__(self, users: Users): ...
    def any_message(self) -> bool: ...
    def get_messages(self) -> List[Message]: ...
    def clear_messages(self) -> None: ...
    def add_highlight(
        self,
        pos: List[float],
        radius: float,
        normal: List[float] = DEFAULT_NORMAL,
        billboard: bool = True,
        color: Optional[Union[mn.Color4, mn.Color3]] = None,
        destination_mask: Mask = Mask.ALL,
    ) -> None: ...
    def add_line(
        self,
        a: List[float],
        b: List[float],
        from_color: Optional[Union[mn.Color4, mn.Color3]] = None,
        to_color: Optional[Union[mn.Color4, mn.Color3]] = None,
        destination_mask: Mask = Mask.ALL,
    ) -> None: ...
    def add_text(self, text: str, pos: list[float], destination_mask: Mask = Mask.ALL): ...
    def show_modal_dialogue_box(
        self,
        title: str,
        text: str,
        buttons: List[UIButton],
        textbox: Optional[UITextbox] = None,
        destination_mask: Mask = Mask.ALL,
    ): ...
    def update_camera_transform(
        self,
        cam_transform: mn.Matrix4,
        viewport_id: int = MAIN_VIEWPORT,
        destination_mask: Mask = Mask.ALL,
    ) -> None: ...
    def update_ui_canvas(
        self,
        canvas_uid: str,
        canvas_update: UICanvasUpdate,
        destination_mask: Mask = Mask.ALL,
    ) -> None: ...

Import

from habitat_hitl.core.client_message_manager import ClientMessageManager, UIButton, UITextbox

I/O Contract

Inputs

Name	Type	Required	Description
users	Users	Yes	User management object defining the set of connected clients

Outputs

Name	Type	Description
get_messages()	List[Message]	List of message dictionaries indexed by user ID, each containing accumulated visual and UI updates for that client

Usage Examples

Drawing a Highlight Circle

import magnum as mn
from habitat_hitl.core.client_message_manager import ClientMessageManager
from habitat_hitl.core.user_mask import Mask, Users

users = Users(max_user_count=2, activate_users=True)
msg_mgr = ClientMessageManager(users)

# Draw a green highlight circle at a world position for all users
msg_mgr.add_highlight(
    pos=[1.0, 0.5, -2.0],
    radius=0.3,
    color=mn.Color3(0.0, 1.0, 0.0),
    destination_mask=Mask.ALL,
)

# Retrieve messages for sending to clients
messages = msg_mgr.get_messages()

Showing a Modal Dialog

from habitat_hitl.core.client_message_manager import ClientMessageManager, UIButton

ok_button = UIButton(button_id="ok", text="OK", enabled=True)
cancel_button = UIButton(button_id="cancel", text="Cancel", enabled=True)

msg_mgr.show_modal_dialogue_box(
    title="Confirm Action",
    text="Are you sure you want to proceed?",
    buttons=[ok_button, cancel_button],
    destination_mask=Mask.ALL,
)

Related Pages

• Heuristic:Facebookresearch_Habitat_lab_Warning_Deprecated_Legacy_UI_System