Implementation:Haosulab ManiSkill StereoDepthCamera

Knowledge Sources	Haosulab_ManiSkill
Domains	Robotics, Simulation, Depth Sensing
Last Updated	2026-02-15 08:00 GMT

Overview

Concrete tool for simulating a stereo depth camera sensor that provides depth, color, position, and segmentation images.

Description

The depth_camera.py module provides the StereoDepthCamera class and its configuration StereoDepthCameraConfig, implementing a stereo depth camera sensor that wraps SAPIEN's StereoDepthSensor.

StereoDepthCameraConfig: Extends CameraConfig with:

min_depth -- Minimum depth threshold (default: 0.05 meters).
rgb_resolution -- Property returning (width, height) tuple.
rgb_intrinsic -- Property computing the 3x3 intrinsic matrix from FOV and resolution.
fromCameraConfig() -- Factory to create from an existing CameraConfig.

StereoDepthCamera: Extends Camera with:

Mounts on an actor or link, or operates as a free-floating camera.
get_images() -- Returns raw texture images. Supports "Color" (float RGBA), "depth" (computed stereo depth), "Position" (3D positions with stereo depth replacing Z), and "Segmentation" (uint32 labels).
get_params() -- Returns camera extrinsic/intrinsic matrices and cam2world transformation.
observation_space -- Gymnasium observation space with proper shapes and dtypes.

Note: This module currently requires SAPIEN render system 3.0.

Usage

Used when stereo depth sensing is needed in an environment. Configured via the environment's sensor configuration and accessed through the standard sensor interface.

Code Reference

Source Location

Repository: Haosulab_ManiSkill
File: mani_skill/sensors/depth_camera.py

Signature

class StereoDepthCameraConfig(CameraConfig):
    def __init__(self, *args, min_depth: float = 0.05, **kwargs): ...
    @property
    def rgb_resolution(self) -> tuple: ...
    @property
    def rgb_intrinsic(self) -> np.ndarray: ...

class StereoDepthCamera(Camera):
    def __init__(self, camera_cfg, scene, renderer_type, articulation=None): ...
    def get_images(self, take_picture=False) -> dict: ...
    def get_params(self) -> dict: ...
    @property
    def observation_space(self) -> spaces.Dict: ...

Import

from mani_skill.sensors.depth_camera import StereoDepthCamera, StereoDepthCameraConfig

I/O Contract

Inputs

Name	Type	Required	Description
camera_cfg	StereoDepthCameraConfig	Yes	Camera configuration with resolution, FOV, mount info
scene	sapien.Scene	Yes	The SAPIEN scene
renderer_type	str	Yes	Must be "sapien"
articulation	PhysxArticulation	No	Articulation for link-mounted cameras

Outputs

Name	Type	Description
images	dict[str, np.ndarray]	Raw images keyed by texture name (Color, depth, Position, Segmentation)
params	dict	Camera matrices (extrinsic_cv, cam2world_gl, intrinsic_cv)

Usage Examples

Basic Usage

from mani_skill.sensors.depth_camera import StereoDepthCamera, StereoDepthCameraConfig

cfg = StereoDepthCameraConfig(
    uid="depth_cam", width=640, height=480, fov=1.0, min_depth=0.05
)
camera = StereoDepthCamera(cfg, scene, renderer_type="sapien")
images = camera.get_images(take_picture=True)
depth = images["depth"]  # shape (H, W, 1)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment