Implementation:Haosulab ManiSkill RenderCamera

Knowledge Sources	Haosulab_ManiSkill
Domains	Robotics, Simulation, Rendering
Last Updated	2026-02-15 08:00 GMT

Overview

Concrete tool for managing batched render cameras across parallel simulation environments with support for mounted cameras and matrix caching.

Description

The RenderCamera dataclass wraps a list of sapien.render.RenderCameraComponent objects, providing a unified interface for camera management across parallel sub-scenes. It supports both free-floating and mounted cameras (attached to an Actor or Link).

Key features:

Camera group management: All managed cameras must have the same width and height, enabling them to be grouped together for batch rendering.
Matrix computation: Provides get_extrinsic_matrix(), get_intrinsic_matrix(), and get_model_matrix() with internal caching to avoid redundant computation.
Mount support: When a camera is mounted on an Actor or Link, the model and extrinsic matrices are dynamically computed from the mount's current pose combined with the camera's local pose.
GPU-aware: On GPU simulation, matrices are computed from the GPU-resident pose data for efficiency.
Local pose: local_pose property returns the camera's pose relative to its mount (or world if unmounted).

The class factory create() constructs a RenderCamera from a list of render camera components and an optional mount.

Usage

RenderCamera objects are created by the environment's sensor system and are not typically instantiated directly by users. They are accessed through the Camera sensor class for generating observations.

Code Reference

Source Location

Repository: Haosulab_ManiSkill
File: mani_skill/utils/structs/render_camera.py

Signature

@dataclass
class RenderCamera:
    _render_cameras: list[sapien.render.RenderCameraComponent]
    name: str
    scene: Any
    camera_group: sapien.render.RenderCameraGroup = None
    mount: Union[Actor, Link] = None

    @classmethod
    def create(
        cls,
        render_cameras: list[sapien.render.RenderCameraComponent],
        scene: Any,
        mount: Union[Actor, Link] = None,
    ) -> "RenderCamera": ...

    def get_extrinsic_matrix(self) -> torch.Tensor: ...
    def get_intrinsic_matrix(self) -> torch.Tensor: ...
    def get_model_matrix(self) -> torch.Tensor: ...

Import

from mani_skill.utils.structs.render_camera import RenderCamera

I/O Contract

Inputs

Name	Type	Required	Description
render_cameras	list[RenderCameraComponent]	Yes	SAPIEN camera components (one per sub-scene)
scene	ManiSkillScene	Yes	The parent scene
mount	Union[Actor, Link]	No	Entity to mount the camera on

Outputs

Name	Type	Description
extrinsic_matrix	torch.Tensor	Camera extrinsic matrices of shape (N, 4, 4)
intrinsic_matrix	torch.Tensor	Camera intrinsic matrices of shape (N, 3, 3)
model_matrix	torch.Tensor	Camera-to-world matrices of shape (N, 4, 4)

Usage Examples

Basic Usage

# RenderCamera is typically accessed through the Camera sensor
camera_sensor = env.scene.sensors["base_camera"]

# Access the underlying RenderCamera
render_cam = camera_sensor.camera

# Get camera matrices
intrinsic = render_cam.get_intrinsic_matrix()
extrinsic = render_cam.get_extrinsic_matrix()

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment