Implementation:Haosulab ManiSkill Camera Pose Randomization

Field	Value
Implementation Name	Camera Pose Randomization
Type	API Doc
Domain	Sim2Real
Source Files	`mani_skill/envs/utils/randomization/camera.py` (L12-60), `mani_skill/envs/utils/randomization/pose.py` (L13-34)
Date	2026-02-15
Repository	Haosulab/ManiSkill

Overview

The Camera Pose Randomization implementation provides utility functions for randomizing camera viewpoints and object orientations during environment initialization or episode reset. These functions are used in digital twin environments to implement domain randomization for sim-to-real transfer, making trained policies robust to camera calibration errors and object pose variations.

Description

Three primary functions are provided:

make_camera_rectangular_prism: Samples camera positions uniformly within a rotated rectangular prism volume.
noised_look_at: Generates camera poses by adding Gaussian noise to a nominal look-at configuration.
random_quaternions: Generates random orientations by sampling Euler angles uniformly within configurable bounds.

All functions operate on PyTorch tensors and support batch dimensions for GPU-parallel environment randomization.

Usage

from mani_skill.envs.utils.randomization.camera import (
    make_camera_rectangular_prism,
    noised_look_at,
)
from mani_skill.envs.utils.randomization.pose import random_quaternions

# Sample N camera positions and create noisy look-at poses
positions = make_camera_rectangular_prism(n=256, scale=[0.1, 0.1, 0.05],
                                          center=[0.5, 0.0, 0.6])
poses = noised_look_at(eye=positions, target=[0.0, 0.0, 0.15])

# Generate random yaw-only orientations for objects
quats = random_quaternions(n=256, lock_x=True, lock_y=True)

Code Reference

make_camera_rectangular_prism

def make_camera_rectangular_prism(
    n,
    scale=[0.1, 0.1, 0.1],
    center=[0, 0, 0],
    theta=0,
    device=None,
) -> torch.Tensor:
    """
    Args:
        n: number of sampled points within the geometry
        scale: [x, y, z] scale for unit cube
        center: [x, y, z] scaled unit cube coordinates
        theta: [0, 2pi] rotation about the z axis
    Returns:
        torch.Tensor of shape (n, 3) with sampled positions
    """
    scale = common.to_tensor(scale, device=device)
    center = common.to_tensor(center, device=device)
    xyz = (torch.rand(n, 3, device=device) - 0.5) * scale
    rot_mat = euler_angles_to_matrix(
        torch.tensor([0, 0, theta], dtype=torch.float32, device=device),
        convention="XYZ",
    )
    return (xyz @ rot_mat.T) + center

Parameter	Type	Default	Description
`n`	`int`	(required)	Number of points to sample.
`scale`	`list[float]`	`[0.1, 0.1, 0.1]`	Extent of the rectangular prism in x, y, z (meters).
`center`	`list[float]`	`[0, 0, 0]`	Center position of the prism in world coordinates.
`theta`	`float`	`0`	Rotation of the prism about the z-axis (radians).
`device`	`torch.device`	`None`	PyTorch device for the output tensor.

noised_look_at

def noised_look_at(
    eye,
    target,
    look_at_noise=1e-2,
    view_axis_rot_noise=2e-1,
    device=None,
) -> Pose:
    """
    Args:
        eye: mean camera position (n, 3)
        target: mean target position (3,)
        look_at_noise: std of noise added to target in lookat transform
        view_axis_rot_noise: std of noise added to rotation about the viewing direction
    Returns:
        Pose object with randomized camera poses
    """

Parameter	Type	Default	Description
`eye`	`torch.Tensor`	(required)	Camera positions of shape (n, 3).
`target`	`torch.Tensor`	(required)	Look-at target position of shape (3,).
`look_at_noise`	`float`	`1e-2`	Standard deviation of Gaussian noise added to the target point (meters).
`view_axis_rot_noise`	`float`	`2e-1`	Standard deviation of Gaussian noise for rotation about the viewing axis (radians).
`device`	`torch.device`	`None`	PyTorch device.

random_quaternions

def random_quaternions(
    n: int,
    device: Device = None,
    lock_x: bool = False,
    lock_y: bool = False,
    lock_z: bool = False,
    bounds=(0, np.pi * 2),
):
    """
    Generates random quaternions by generating random Euler angles uniformly,
    with each X, Y, Z angle ranging from bounds[0] to bounds[1].
    Can optionally lock X, Y, and/or Z Euler angles to 0.
    """

Parameter	Type	Default	Description
`n`	`int`	(required)	Number of quaternions to generate.
`device`	`Device`	`None`	PyTorch device.
`lock_x`	`bool`	`False`	If True, fix X Euler angle to 0.
`lock_y`	`bool`	`False`	If True, fix Y Euler angle to 0.
`lock_z`	`bool`	`False`	If True, fix Z Euler angle to 0.
`bounds`	`tuple`	`(0, 2*pi)`	Angular range for each Euler angle (radians).

I/O Contract

Function	Input	Output
`make_camera_rectangular_prism`	`n`, `scale`, `center`, `theta`	`torch.Tensor` shape (n, 3) -- positions in world frame
`noised_look_at`	`eye` (n,3), `target` (3,)	`Pose` with `.p` (n,3) and `.q` (n,4) -- camera-to-world poses
`random_quaternions`	`n`, axis locks, bounds	`torch.Tensor` shape (n, 4) -- quaternions [w, x, y, z]

External Dependencies

Package	Purpose
torch	Tensor operations and random sampling
numpy	Constant values (`np.pi`)
transforms3d	Rotation conversions (imported but used indirectly)
mani_skill.utils.geometry.rotation_conversions	Euler-to-matrix, matrix-to-quaternion, axis-angle-to-quaternion conversions
mani_skill.utils.structs.pose	`Pose` class for representing SE(3) poses

Usage Examples

import torch
from mani_skill.envs.utils.randomization.camera import (
    make_camera_rectangular_prism,
    noised_look_at,
)
from mani_skill.envs.utils.randomization.pose import random_quaternions

device = torch.device("cuda:0")

# Randomize camera for 512 parallel environments
eye_positions = make_camera_rectangular_prism(
    n=512,
    scale=[0.15, 0.15, 0.08],
    center=[0.5, 0.0, 0.55],
    theta=0.0,
    device=device,
)
camera_poses = noised_look_at(
    eye=eye_positions,
    target=torch.tensor([0.0, 0.0, 0.1], device=device),
    look_at_noise=0.015,
    view_axis_rot_noise=0.15,
    device=device,
)
# camera_poses.p has shape (512, 3), camera_poses.q has shape (512, 4)

# Randomize object orientations (yaw only, 0 to 180 degrees)
import numpy as np
object_quats = random_quaternions(
    n=512,
    device=device,
    lock_x=True,
    lock_y=True,
    lock_z=False,
    bounds=(0, np.pi),
)
# object_quats has shape (512, 4)

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment