Workflow:Haosulab ManiSkill Motion Planning Demo Generation
| Knowledge Sources | |
|---|---|
| Domains | Motion_Planning, Data_Collection, Robotics_Simulation |
| Last Updated | 2026-02-15 12:00 GMT |
Overview
End-to-end process for generating expert demonstration trajectories using motion planning solvers in ManiSkill environments.
Description
This workflow covers the automated generation of expert demonstration trajectories using scripted motion planning solutions. ManiSkill provides pre-built motion planning solvers for the Panda, SO100, and xArm6 robots across a variety of tabletop manipulation tasks. The motion planners use inverse kinematics and path planning to compute collision-free trajectories, then record the resulting actions, states, and observations as HDF5 trajectory files. These demonstrations serve as training data for imitation learning pipelines. The workflow supports parallel generation via CPU multiprocessing, optional video recording, and trajectory merging.
Usage
Execute this workflow when you need high-quality expert demonstration data for imitation learning and want to generate it automatically without human involvement. This is appropriate for tasks that have pre-built motion planning solutions (e.g., PickCube, StackCube, PegInsertion) or when you have implemented a custom motion planning solver for your task.
Execution Steps
Step 1: Environment and Solver Selection
Select the target task environment and its corresponding motion planning solver. ManiSkill provides pre-built solutions for many Panda robot tasks (PickCube, StackCube, PushCube, PegInsertionSide, PlugCharger, DrawTriangle, DrawSVG, etc.) as well as SO100 and xArm6 variants. The environment is created with pd_joint_pos control mode which is required for motion planning.
Key considerations:
- Motion planning solutions are robot-specific (Panda, SO100, xArm6 each have their own solver)
- The environment must use pd_joint_pos control mode for motion planning compatibility
- Observation mode can be set to "none" during generation since observations can be replayed later
- Choose the simulation backend (CPU for multiprocessing, GPU for single-process speed)
Step 2: Motion Planning Execution
Run the motion planning solver iteratively across multiple episodes. For each episode, the solver computes a sequence of robot joint positions that achieve the task goal, using inverse kinematics for target pose computation and collision-aware path planning for trajectory generation. The solver breaks the task into sub-goals (approach, grasp, lift, place) and plans between each waypoint.
Key considerations:
- Each task solver decomposes the task into sequential sub-goals with waypoint targets
- The base motion planner uses mplib for collision-free path planning between waypoints
- Gripper open/close actions are interleaved with arm motion commands
- Some episodes may fail due to infeasible configurations; use only_count_success to filter
Step 3: Trajectory Recording
Capture the executed trajectories using ManiSkill's RecordEpisode wrapper. Each trajectory is stored as an HDF5 file containing actions, environment states, and optionally observations at each timestep. A companion JSON metadata file records the environment configuration, control mode, and episode statistics.
Key considerations:
- The RecordEpisode wrapper automatically captures all trajectory data
- Actions are stored in the control mode used during generation (pd_joint_pos)
- Environment states enable exact trajectory replay and control mode conversion later
- Video recording can be enabled for visual verification of demonstration quality
Step 4: Parallel Generation (Optional)
Optionally parallelize trajectory generation across multiple CPU processes for faster dataset creation. The run script supports multiprocessing where each process generates a subset of trajectories independently, then results are merged into a single dataset.
Key considerations:
- CPU multiprocessing is used since motion planning requires CPU-based collision checking
- Each process generates trajectories with non-overlapping random seeds
- The merge step combines individual HDF5 files into a single consolidated dataset
- Set num_procs based on available CPU cores
Step 5: Trajectory Merging and Validation
Merge trajectories from parallel generation into a single HDF5 file and validate the dataset. The merge utility combines multiple trajectory files while preserving episode indexing. Validation confirms that the expected number of successful demonstrations were generated and that trajectory data is well-formed.
Key considerations:
- The merge_trajectories utility handles renumbering episodes across merged files
- Validate success rate to ensure demonstration quality
- Check that the total number of transitions is sufficient for training
- The merged dataset is ready for trajectory replay or direct use in training