Implementation:Facebookresearch Habitat lab Observations to image

Knowledge Sources	Habitat-Lab
Domains	Visualization, Evaluation
Last Updated	2026-02-15 02:00 GMT

Overview

Concrete utility functions for converting Habitat observations and metrics into composite images and videos for evaluation visualization.

Description

The observations_to_image function tiles sensor observations (RGB, depth, semantic) into a single composite image. The images_to_video function encodes a sequence of frames into an MP4 video file. The overlay_frame function adds metric text overlays to frames.

Usage

Call during evaluation when `video_option` is set. Used by `HabitatEvaluator` and custom evaluation scripts.

Code Reference

Source Location

• Repository: habitat-lab
• File: habitat-lab/habitat/utils/visualizations/utils.py
• Lines: L215-261 (observations_to_image), L99-146 (images_to_video), L359-376 (overlay_frame)

Signature

def observations_to_image(
    observation: Dict,
    info: Dict,
) -> np.ndarray:
    """
    Tile sensor observations into a single image.

    Args:
        observation: Dict of sensor observations (rgb, depth, etc.)
        info: Dict of metrics/info for overlay
    Returns:
        Composite image as numpy array (H, W, 3)
    """

def images_to_video(
    images: List[np.ndarray],
    output_dir: str,
    video_name: str,
    fps: int = 10,
    quality: Optional[float] = 5,
    **kwargs,
) -> None:
    """
    Encode frames to MP4 video.

    Args:
        images: List of frame arrays
        output_dir: Directory to save video
        video_name: Video filename (without extension)
        fps: Frames per second
        quality: Video quality (0-10)
    """

def overlay_frame(
    frame: np.ndarray,
    info: Dict,
    additional: Optional[str] = None,
) -> np.ndarray:
    """Add metric text overlay to frame."""

Import

from habitat.utils.visualizations.utils import (
    observations_to_image,
    images_to_video,
    overlay_frame,
)

I/O Contract

Inputs

Name	Type	Required	Description
observation	Dict	Yes	Sensor observations dict (rgb, depth, etc.)
info	Dict	Yes	Metrics/info dict for overlay
images	List[np.ndarray]	Yes (for video)	Sequence of frames
output_dir	str	Yes (for video)	Output directory path

Outputs

Name	Type	Description
image	np.ndarray	Composite image (H, W, 3) uint8
video file	.mp4	Encoded video file on disk

Usage Examples

Create Evaluation Video

from habitat.utils.visualizations.utils import (
    observations_to_image,
    images_to_video,
    overlay_frame,
)

frames = []
obs = env.reset()
while not env.episode_over:
    frame = observations_to_image(obs, env.get_metrics())
    frame = overlay_frame(frame, env.get_metrics())
    frames.append(frame)
    action = agent.act(obs)
    obs = env.step(action)

images_to_video(frames, "output/", "episode_0", fps=30)

Related Pages

Implements Principle

• Principle:Facebookresearch_Habitat_lab_Metric_Visualization

Requires Environment

• Environment:Facebookresearch_Habitat_lab_Python_3_9_Core_Dependencies