Heuristic:Haosulab ManiSkill Num Envs Backend Selection
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Optimization |
| Last Updated | 2026-02-15 08:00 GMT |
Overview
Decision framework for choosing between CPU and GPU simulation backends based on num_envs, observation mode, and deployment target (sim vs. real).
Description
ManiSkill automatically selects the simulation backend based on `num_envs`: CPU for single environments, GPU for multiple. However, several additional constraints affect this decision: ray-tracing is incompatible with multi-env, sim2real requires CPU, visual observations with `parallel_in_single_scene` are unsupported, and evaluation with geometry randomization requires single environments. Understanding these constraints prevents runtime errors and guides performance optimization.
Usage
Use this heuristic when configuring environment creation for any workflow. The decision tree should be consulted whenever setting `num_envs`, `sim_backend`, `render_backend`, `obs_mode`, or `shader_dir` parameters.
The Insight (Rule of Thumb)
- Backend auto-selection:
- `num_envs == 1` → `physx_cpu` (automatic)
- `num_envs > 1` → `physx_cuda` (automatic)
- CPU backend + `num_envs > 1` = RuntimeError
- Ray-tracing constraint:
- `shader_dir="rt"` or `"rt-fast"` + `num_envs > 1` + `parallel_in_single_scene=False` = RuntimeError
- Use `num_envs=1` for ray-traced rendering or switch to `"default"` or `"minimal"` shaders
- Sim2Real constraint:
- Sim2RealEnv enforces `physx_cpu` backend
- Always use `num_envs=1` for real robot deployment
- Parallel scene constraint:
- `parallel_in_single_scene=True` is incompatible with visual obs modes (`sensor_data`, `pointcloud`, `rgb`, `depth`, `rgbd`)
- Only use `parallel_in_single_scene` with state-based observations
- Evaluation with geometry randomization:
- Tasks with randomized geometry (e.g., PickSingleYCB) require `num_envs=1` for trajectory replay fidelity
- Trade-off: GPU simulation is 10-100x faster for RL training but restricts some features (ray-tracing, sim2real). CPU is required for single-env debugging, motion planning, and real-robot deployment.
Reasoning
Auto-selection logic from `mani_skill/envs/sapien_env.py:233-237`:
if sim_backend is None:
if num_envs == 1:
sim_backend = "cpu"
else:
sim_backend = "gpu"
CPU multi-env restriction from `mani_skill/envs/sapien_env.py:248-251`:
if self.backend.sim_backend in CPU_SIM_BACKENDS and num_envs > 1:
raise RuntimeError(
"""Cannot set the sim backend to 'cpu' and have multiple environments.
If you want to do CPU sim backends and have environment vectorization
you must use multi-processing across CPUs.
This can be done via the gymnasium's AsyncVectorEnv API"""
)
Ray-tracing restriction from `mani_skill/envs/sapien_env.py:254-256`:
if "rt" == shader_dir[:2]:
if num_envs > 1 and parallel_in_single_scene == False:
raise RuntimeError(
"""Currently you cannot run ray-tracing on more than one environment
in a single process"""
)
Sim2Real CPU enforcement from `mani_skill/envs/sim2real_env.py:66-68`:
assert (
self.sim_env.unwrapped.backend.sim_backend == "physx_cpu"
), "For the Sim2RealEnv we expect the simulation to be using the physx_cpu simulation backend"
Evaluation note from `examples/baselines/ppo/README.md:34-35`:
Note that with `--evaluate`, trajectories are saved from GPU simulation. In order to support replaying these trajectories correctly, the number of evaluation environments must be fixed to `1`. This is necessary for tasks with randomizations on geometry (e.g. PickSingleYCB).