Workflow:Haosulab ManiSkill Custom Task Development
| Knowledge Sources | |
|---|---|
| Domains | Robotics_Simulation, Environment_Design, GPU_Parallelization |
| Last Updated | 2026-02-15 12:00 GMT |
Overview
End-to-end process for creating a custom GPU-parallelizable robot manipulation task environment in ManiSkill.
Description
This workflow covers the complete process of developing a new task environment in ManiSkill, from environment class scaffolding to GPU-parallel testing. ManiSkill environments inherit from BaseEnv and follow a structured lifecycle of reconfiguration (loading assets) and episode initialization (randomizing poses). The framework abstracts GPU memory management and parallelization, enabling developers to write task logic once and have it run both on CPU (single environment) and GPU (hundreds of parallel environments). The process covers environment registration, robot selection, scene and object loading, observation definition, reward computation, and success condition specification.
Usage
Execute this workflow when you need to define a new manipulation, locomotion, or interaction task that is not already provided by ManiSkill's built-in task library. This is appropriate for researchers developing custom benchmarks, practitioners prototyping new robot skills, or anyone needing a simulation environment with specific object arrangements, reward structures, or success criteria.
Execution Steps
Step 1: Environment Class Scaffolding
Create a new Python file with a class that inherits from BaseEnv and register it with ManiSkill's environment registry. Use the @register_env decorator to assign a unique environment ID and set default parameters like max_episode_steps. Define the supported robot types and configure the default robot, control mode, and simulation settings.
Key considerations:
- Use the @register_env decorator with a unique ID and version suffix (e.g., "MyTask-v1")
- Specify SUPPORTED_ROBOTS to restrict which robots can be used with this task
- Override the _default_sim_config property to tune GPU memory allocation for the task
- Set appropriate max_episode_steps based on expected task difficulty
Step 2: Scene and Object Loading
Override the _load_scene method to build the simulation scene with all necessary objects, articulations, and fixtures. Use ManiSkill's builder utilities to create primitive shapes, load mesh files, or import URDF/MJCF models. Optionally use a SceneBuilder (e.g., TableSceneBuilder) for common scene setups.
Key considerations:
- Use ActorBuilder for rigid body objects and ArticulationBuilder for articulated objects
- Scene builders (TableSceneBuilder, etc.) provide reusable scene configurations
- For GPU parallelization, all objects must be created consistently across all parallel environments
- Use the scene's heterogeneous object support for tasks with varying objects per environment
Step 3: Episode Initialization
Override the _initialize_episode method to randomize object poses, robot configurations, and goal states at the start of each episode. This method is called on every environment reset and must handle batched operations for GPU parallelization. Use ManiSkill's randomization utilities for sampling poses and positions.
Key considerations:
- All pose setting must use batched tensor operations for GPU compatibility
- Use the BatchedRNG for reproducible per-environment random number generation
- The UniformPlacementSampler helps generate non-overlapping object placements
- Robot initial joint positions should include small noise for diversity
Step 4: Observation Definition
Override the _get_obs_extra method to add task-specific observations beyond the default robot proprioception and sensor data. Define what privileged state information (e.g., object poses, goal positions) is included in the observation when using state-based observation modes.
Key considerations:
- The base class automatically provides robot proprioception (qpos, qvel) and sensor data
- Extra observations should be returned as a dictionary of torch tensors
- For GPU mode, all observation tensors must be properly batched (num_envs, ...)
- Visual observations (RGBD, point clouds) are handled automatically by the sensor system
Step 5: Reward and Success Conditions
Implement the evaluate method to compute success conditions and the compute_dense_reward/compute_normalized_dense_reward methods for reward shaping. Success conditions drive evaluation metrics while dense rewards guide RL training. Both must operate in batched mode for GPU parallelization.
Key considerations:
- The evaluate method returns a dictionary with at least a "success" boolean tensor
- Dense rewards should be shaped to provide learning signal throughout the episode
- Use vectorized torch operations for all computations (no Python loops over environments)
- Normalized dense reward should map to [0, 1] range for consistent RL training
Step 6: Testing and Validation
Test the environment on both CPU (single instance) and GPU (parallel instances) backends to verify correctness. Run the demo_random_action script to confirm basic functionality, then test with the GPU backend to verify parallelization. Validate that reset distributions, rewards, and success conditions behave as expected.
Key considerations:
- Test with both sim_backend="cpu" and sim_backend="gpu" to catch parallelization issues
- Use the demo_random_action.py script as a quick smoke test
- Verify that rewards are consistent between CPU and GPU backends
- Check that reconfiguration and episode initialization work correctly with multiple environments