Principle:Haosulab ManiSkill Domain Randomization
| Field | Value |
|---|---|
| Principle Name | Domain Randomization |
| Domain | Sim2Real |
| Overview | Randomizing simulation parameters to improve policy robustness for sim-to-real transfer |
| Date | 2026-02-15 |
| Repository | Haosulab/ManiSkill |
Overview
The Domain Randomization principle describes how ManiSkill introduces controlled variability into simulation parameters during training to produce policies that are robust to the inevitable differences between simulation and the real world. By training on a wide distribution of visual and physical conditions, the policy learns to be invariant to domain-specific details, improving transfer to the target (real) domain.
Description
ManiSkill provides randomization utilities for several aspects of the simulation environment:
Camera Pose Randomization
Camera viewpoints are randomized to make policies robust to imprecise camera mounting in the real world. Two complementary techniques are provided:
- Rectangular prism sampling: Camera positions are sampled uniformly within a rectangular prism volume defined by a center, scale, and rotation about the z-axis. This allows specifying a spatial region where the camera might plausibly be located.
- Noised look-at: Given a nominal camera position and target point, Gaussian noise is added to the look-at target (controlling gaze direction variation) and to the rotation about the viewing axis (controlling camera roll variation). This produces poses that are similar to but not identical to the nominal calibrated pose.
Object Pose Randomization
Object orientations are randomized using uniformly sampled Euler angles converted to quaternions. Individual axes (X, Y, Z) can be locked to constrain the randomization (e.g., allowing only yaw variation for objects on a table). The angular bounds are configurable.
Additional Randomization (Environment-Specific)
Individual digital twin environments may additionally randomize:
- Lighting: Direction, intensity, and color of scene lights.
- Materials: Surface roughness, metallic properties, and colors of objects.
- Object geometry: Scale variations or mesh deformations.
These are typically implemented within the environment's _load_scene or _initialize_episode methods.
Usage
Randomization utilities are typically called during environment initialization or episode reset:
from mani_skill.envs.utils.randomization.camera import (
make_camera_rectangular_prism,
noised_look_at,
)
from mani_skill.envs.utils.randomization.pose import random_quaternions
# Sample camera positions in a 20cm x 20cm x 10cm volume
eye_positions = make_camera_rectangular_prism(
n=num_envs,
scale=[0.2, 0.2, 0.1],
center=[0.5, 0.0, 0.5],
theta=0,
device=device,
)
# Create noisy look-at poses
camera_poses = noised_look_at(
eye=eye_positions,
target=[0.0, 0.0, 0.1],
look_at_noise=0.02,
view_axis_rot_noise=0.1,
device=device,
)
# Random object orientations (yaw only)
orientations = random_quaternions(
n=num_envs,
device=device,
lock_x=True,
lock_y=True,
lock_z=False,
bounds=(0, 2 * 3.14159),
)
Theoretical Basis
- Domain Randomization (Tobin et al., 2017): The foundational paper demonstrated that by randomizing visual properties (textures, lighting, camera pose) during training in simulation, policies can transfer to the real world without explicit domain adaptation. The key insight is that the real world becomes just one more sample from the training distribution.
- Visual randomization for sim2real: Subsequent work has shown that camera pose randomization is particularly effective for bridging the visual gap, because small camera calibration errors in the real world can cause large distribution shifts if the policy is trained with a single fixed viewpoint.
- Uniform vs. Gaussian noise: ManiSkill uses uniform sampling for positions (rectangular prism) and Gaussian noise for angular perturbations. Uniform sampling ensures coverage of the entire plausible space, while Gaussian noise naturally concentrates perturbations near the nominal value, matching the expected distribution of real-world calibration errors.
- GPU-parallel randomization: All randomization functions operate on PyTorch tensors and support batch dimensions, enabling efficient GPU-parallel randomization across thousands of environments simultaneously.