Workflow:Isaac sim IsaacGymEnvs Custom Task Development
| Knowledge Sources | |
|---|---|
| Domains | Reinforcement_Learning, Robotics, Environment_Design |
| Last Updated | 2026-02-15 09:00 GMT |
Overview
End-to-end process for creating a new GPU-accelerated RL environment by extending the VecTask base class, configuring Hydra YAML files, and registering the task in IsaacGymEnvs.
Description
This workflow covers how to create a custom RL task within the IsaacGymEnvs framework. All environments inherit from VecTask, which provides the simulation loop, buffer management, rendering, domain randomization infrastructure, and the Gym-compatible step/reset API. The developer implements three core methods (create_sim, pre_physics_step, post_physics_step), creates matching YAML configuration files for task and training, and registers the task in the task map. The result is a fully GPU-accelerated environment compatible with rl_games PPO training.
Key capabilities:
- GPU-parallel simulation with configurable environment count
- Pre-built simulation loop with automatic buffer management
- Configurable viewer with keyboard controls (V to toggle rendering, ESC to quit)
- Domain randomization support via apply_randomizations()
- Compatible with all existing training infrastructure (PPO, multi-GPU, PBT, WandB)
Usage
Execute this workflow when you need to create a new robotics environment that is not already provided by IsaacGymEnvs, or when adapting an environment from a previous Isaac Gym release. You need familiarity with the Isaac Gym API (gymapi, gymtorch) and a robot/scene asset (URDF or MJCF format).
Execution Steps
Step 1: Define Task Requirements
Determine the observation space, action space, reward structure, and reset conditions for your new environment. Identify which robot and object assets are needed, what physics interactions are required, and the target control frequency.
Key considerations:
- numObservations: size of the observation vector per environment
- numActions: size of the action vector per environment
- numEnvs: default number of parallel environments (can be overridden via CLI)
- Optional: numStates for asymmetric actor-critic, numAgents for multi-agent
Step 2: Create the Task Python File
Create a new Python file in isaacgymenvs/tasks/ that defines a class inheriting from VecTask. Import gymtorch and gymapi from isaacgym. Implement the constructor to call super().__init__() with the config, then acquire and wrap the GPU state tensors needed for your task.
What happens:
- The constructor receives cfg, rl_device, sim_device, graphics_device_id, headless, and virtual_screen_capture/force_render parameters
- super().__init__() triggers create_sim(), initializes GPU buffers, and sets up the viewer
- After super init, acquire state tensors (DOF states, rigid body states, root states) using gym.acquire_*_tensor() and wrap them with gymtorch.wrap_tensor()
Step 3: Implement Core Simulation Methods
Implement the three required methods: create_sim() to set up the physics scene, pre_physics_step() to apply actions, and post_physics_step() to compute observations and rewards. Optionally implement reset_idx() for environment-specific reset logic.
create_sim():
- Set the up-axis direction
- Call super().create_sim() with device arguments
- Create the ground plane
- Create environments in a loop, loading assets and creating actors
pre_physics_step(actions):
- Process raw actions from the policy (e.g., scale, clip)
- Apply actions to the simulation (e.g., set DOF targets, apply forces)
post_physics_step():
- Refresh state tensors (gym.refresh_dof_state_tensor, etc.)
- Compute observations from the current state
- Compute rewards based on task-specific criteria
- Detect done conditions and trigger environment resets
Step 4: Create Configuration Files
Create two YAML configuration files: a task config (cfg/task/MyTask.yaml) defining environment parameters, and a training config (cfg/train/MyTaskPPO.yaml) defining rl_games PPO hyperparameters. The task config name field must match the key used in the task registry.
Task config requirements:
- A name field matching the task map key
- An env section with numEnvs, numObservations, numActions, and episodeLength
- Physics parameters and any task-specific configuration
Training config requirements:
- rl_games-compatible PPO configuration
- Network architecture (MLP units, optional LSTM)
- Learning rate, batch size, horizon length, mini-epochs
- Reward shaper scale and other PPO hyperparameters
Step 5: Register the Task
Add the task class import and mapping entry to isaacgymenvs/tasks/__init__.py. The key in isaacgym_task_map must match the name field in the task YAML config. This enables launching the task via the CLI.
What happens:
- Add import: from isaacgymenvs.tasks.my_task import MyTask
- Add entry: "MyTask": MyTask to the isaacgym_task_map dictionary
- The task can now be launched with python train.py task=MyTask
Step 6: Test and Iterate
Run the task with rendering enabled to visually verify the simulation setup, observation correctness, and reward behavior. Test with a small number of environments first, then scale up. Verify that training converges on the expected behavior.
Key considerations:
- Start with num_envs=4 and rendering enabled for visual debugging
- Check that observations are normalized to reasonable ranges
- Verify reward signals provide useful learning gradients
- Use the V key to toggle rendering for faster training once verified
- Monitor training curves in TensorBoard or WandB