Implementation:Facebookresearch Habitat lab Selection

Knowledge Sources	Facebookresearch_Habitat_lab
Domains	Embodied_AI, Human_in_the_Loop
Last Updated	2026-02-15 00:00 GMT

Overview

Selection is a class that handles raycasting-based object selection in the Habitat simulator by tracking a GuiInput instance and applying configurable selection triggers and object discriminators.

Description

The Selection class provides a flexible mechanism for selecting objects in the 3D simulation environment. It works by casting a ray from the mouse position into the scene and finding the nearest object hit. The class is parameterized by two callables:

• selection_fn: Determines when selection should occur based on the current GuiInput state. Built-in options include hover_fn (every frame), left_click_fn (on left click), and right_click_fn (on right click).
• object_id_discriminator: Filters which object IDs are selectable. Objects rejected by this function are transparent to selection, meaning the ray continues through them to find the next hit. By default, all objects are selectable.

Once a selection is made, the class stores the selected object ID, the hit point, and the hit surface normal. The deselect method clears the current selection.

Usage

Use Selection when you need interactive 3D object picking in HITL applications, such as selecting objects to grasp, inspect, or manipulate. Configure the selection trigger and object filter to match your interaction design.

Code Reference

Source Location

• Repository: Facebookresearch_Habitat_lab
• File: habitat-hitl/habitat_hitl/core/selection.py
• Lines: 1-125

Signature

class Selection:
    def hover_fn(_gui_input: GuiInput) -> bool:
        ...

    def left_click_fn(_gui_input: GuiInput) -> bool:
        ...

    def right_click_fn(_gui_input: GuiInput) -> bool:
        ...

    def default_discriminator(_object_id: int) -> bool:
        ...

    def __init__(
        self,
        simulator: HabitatSim,
        gui_input: GuiInput,
        selection_fn: Callable[[GuiInput], bool],
        object_id_discriminator: Callable[[int], bool] = default_discriminator,
    ):
        ...

    @property
    def selected(self) -> bool:
        ...

    @property
    def object_id(self) -> Optional[int]:
        ...

    @property
    def point(self) -> Optional[mn.Vector3]:
        ...

    @property
    def normal(self) -> Optional[mn.Vector3]:
        ...

    def deselect(self) -> None:
        ...

    def update(self) -> None:
        ...

Import

from habitat_hitl.core.selection import Selection

I/O Contract

Inputs

Name	Type	Required	Description
simulator	HabitatSim	Yes	The simulator instance to raycast upon.
gui_input	GuiInput	Yes	The GUI input state to track for selection triggers.
selection_fn	Callable[[GuiInput], bool]	Yes	Function that returns True when the input indicates a selection attempt.
object_id_discriminator	Callable[[int], bool]	No	Function that determines whether an object ID is selectable. Defaults to accepting all objects.

Outputs

Name	Type	Description
selected	bool	Whether something is currently selected.
object_id	Optional[int]	The ID of the currently selected object, or None.
point	Optional[mn.Vector3]	The 3D point of the selection hit, or None.
normal	Optional[mn.Vector3]	The surface normal at the selection hit point, or None.

Usage Examples

Basic Usage

from habitat_hitl.core.selection import Selection

# Create a selection that triggers on left click
selection = Selection(
    simulator=sim,
    gui_input=gui_input,
    selection_fn=Selection.left_click_fn,
)

# Each frame, update the selection
selection.update()

if selection.selected:
    print(f"Selected object {selection.object_id}")
    print(f"Hit point: {selection.point}")
    print(f"Hit normal: {selection.normal}")

With Object Discriminator

from habitat_hitl.core.selection import Selection

# Only allow selection of specific object IDs
pickable_ids = {10, 20, 30}

selection = Selection(
    simulator=sim,
    gui_input=gui_input,
    selection_fn=Selection.hover_fn,
    object_id_discriminator=lambda obj_id: obj_id in pickable_ids,
)

selection.update()

if selection.selected:
    print(f"Hovering over pickable object {selection.object_id}")