Implementation:Facebookresearch Habitat lab GuiPickHelper

Knowledge Sources	Facebookresearch_Habitat_lab
Domains	Embodied_AI, Human_in_the_Loop
Last Updated	2026-02-15 00:00 GMT

Overview

GuiPickHelper is a helper class for picking up (selecting) objects from the GUI by proximity to a query position, with visual highlighting of graspable and candidate objects using colored rings.

Description

The GuiPickHelper class assists with object selection in HITL applications by finding the closest scene object to a query position and providing visual feedback. It operates on the rigid objects managed by the simulation's rigid object manager.

Key functionality includes:

• get_pick_object_near_query_position(query_pos): Finds the closest scene object to the given 3D position. If the distance is within the can_grasp_place_threshold from the HITL config, the object is marked as a pick candidate and its ID is returned. Otherwise, returns None.
• viz_objects(): Renders visual highlights for all scene objects. Pick candidates are shown with green preview rings (COLOR_GRASP_PREVIEW, radius 0.2), while other objects are shown with orange pulsing rings (COLOR_GRASPABLE, radius 0.15). Highlights are rendered only for the specific user via Mask.from_index.
• on_environment_reset(): Resets internal state when the environment is reset.

The class uses internal helper methods for computing closest points to rays and query positions using numpy vectorized operations.

Usage

Use GuiPickHelper in HITL rearrangement or manipulation tasks where users need to select objects by proximity (e.g., moving an agent near an object and picking it up).

Code Reference

Source Location

• Repository: Facebookresearch_Habitat_lab
• File: habitat-hitl/habitat_hitl/environment/gui_pick_helper.py
• Lines: 1-152

Signature

class GuiPickHelper:
    def __init__(self, app_service: AppService, user_index: int):
        ...

    def on_environment_reset(self):
        ...

    def get_pick_object_near_query_position(self, query_pos):
        ...

    def viz_objects(self):
        ...

Import

from habitat_hitl.environment.gui_pick_helper import GuiPickHelper

I/O Contract

Inputs

Name	Type	Required	Description
app_service	AppService	Yes	The HITL application service providing access to the simulator, GUI drawer, and HITL config.
user_index	int	Yes	The index of the user for whom pick highlights are rendered.

Outputs

Name	Type	Description
get_pick_object_near_query_position()	Optional[int]	The object ID of the nearest scene object within the grasp threshold, or None if no object is close enough.
viz_objects()	(visual output)	Highlight rings rendered around graspable objects and pick candidates via the GuiDrawer.

Constants

Name	Value	Description
DIST_HIGHLIGHT	0.15	Distance threshold for highlighting objects.
COLOR_GRASPABLE	mn.Color3(1, 0.75, 0)	Orange color for graspable object rings.
COLOR_GRASP_PREVIEW	mn.Color3(0.5, 1, 0)	Green color for pick candidate rings.
COLOR_FOCUS_OBJECT	mn.Color3(1, 1, 1)	White color for focused objects.
RADIUS_GRASPABLE	0.15	Radius of graspable object highlight rings.
RADIUS_GRASP_PREVIEW	0.2	Radius of pick candidate highlight rings.
RING_PULSE_SIZE	0.03	Amplitude of the pulsing animation on graspable rings.

Usage Examples

Basic Usage

from habitat_hitl.environment.gui_pick_helper import GuiPickHelper

# Create a pick helper for user 0
pick_helper = GuiPickHelper(
    app_service=app_service,
    user_index=0,
)

# Get the agent's hand position
agent_hand_pos = get_agent_hand_position(sim, agent_idx=0)

# Check if there is a pickable object near the hand
obj_id = pick_helper.get_pick_object_near_query_position(agent_hand_pos)
if obj_id is not None:
    print(f"Can pick up object {obj_id}")

# Render highlight rings around all objects
pick_helper.viz_objects()

# On environment reset
pick_helper.on_environment_reset()