Implementation:ARISE Initiative Robosuite RenderDatasetOmniverse

Knowledge Sources	ARISE_Initiative_Robosuite
Domains	Robotics, 3D Rendering, Dataset Processing
Last Updated	2026-02-15 07:00 GMT

Overview

The render_dataset_with_omniverse script renders robosuite demonstration datasets using NVIDIA Omniverse Isaac Sim, producing high-fidelity RGB, normal map, and semantic segmentation outputs from recorded HDF5 trajectories.

Description

This script provides an end-to-end pipeline for converting robosuite demonstration recordings stored in HDF5 format into photorealistic rendered imagery using NVIDIA Omniverse. It supports both the robomimic and robosuite dataset formats. The pipeline consists of three main stages: loading the robosuite environment from dataset metadata, converting the MuJoCo scene to USD format via the USDExporter, and rendering the USD scene using Omniverse's rendering engine.

The script defines three core classes. RobosuiteEnvInterface handles loading a robosuite environment from an HDF5 dataset, restoring simulation states, creating the USD exporter, and stepping through trajectory frames. RobosuiteWriter is a custom Omniverse Replicator writer that captures annotator outputs (RGB, normals, semantic segmentation) and saves them as image files organized by camera and modality. DataGenerator orchestrates the recording loop, managing render products, the timeline, and frame advancement for both online (live) and offline (pre-exported USD) rendering modes.

The script supports multiple rendering modes (RayTracedLighting, PathTracing), multiple camera views, frame skipping, site hiding, model reloading, and optional video compilation from rendered frames using OpenCV.

Usage

Use this script to generate high-quality rendered imagery from robosuite demonstration datasets. It requires NVIDIA Omniverse Isaac Sim to be installed. Run it from the command line with the path to an HDF5 dataset and desired rendering options.

Code Reference

Source Location

Repository: ARISE_Initiative_Robosuite
File: robosuite/scripts/render_dataset_with_omniverse.py

Signature

class RobosuiteEnvInterface:
    def __init__(self, dataset, episode, output_directory,
                 cameras="agentview", reload_model=False, keep_models=[]) -> None

class RobosuiteWriter(rep.Writer):
    def __init__(self, output_dir: str = None, image_output_format: str = "png",
                 rgb: bool = False, normals: bool = False,
                 semantic_segmentation: bool = False, frame_padding: int = 4)

class DataGenerator:
    def __init__(self, robosuite_env) -> None

def main()

Import

# This is a standalone script, typically run from the command line:
# python robosuite/scripts/render_dataset_with_omniverse.py --dataset path/to/data.hdf5

# Internal imports used by the script:
import robosuite.utils.usd.exporter as exporter
import robosuite

I/O Contract

Inputs

Name	Type	Required	Description
--dataset	str	Yes	Path to the HDF5 dataset file
--ds_format	str	No	Dataset format: "robomimic" or "robosuite" (default: "robomimic")
--episode	int (list)	No	Episode number(s) to render; defaults to all episodes
--output_directory	str	No	Directory for output files; defaults to dataset directory
--cameras	str (list)	No	Camera name(s) to render (default: ["agentview"])
--width	int	No	Viewport width in pixels (default: 1280)
--height	int	No	Viewport height in pixels (default: 720)
--renderer	str	No	Rendering mode: "RayTracedLighting" or "PathTracing" (default: "RayTracedLighting")
--save_video	flag	No	Save rendered frames as video files
--online	flag	No	Enable online rendering mode (no USD file saved)
--skip_frames	int	No	Render every nth frame (default: 1)
--hide_sites	flag	No	Hide all sites in the scene
--rgb	flag	No	Enable RGB image output
--normals	flag	No	Enable surface normals output
--semantic_segmentation	flag	No	Enable semantic segmentation output

Outputs

Name	Type	Description
USD files	.usd files	Exported USD scene files in frames/ subdirectory (offline mode)
Image frames	.png files	Rendered images organized by camera and modality
Video files	.mp4 files	Compiled videos from rendered frames (when --save_video is set)

Usage Examples

# Render all episodes from a dataset with RGB output
# python robosuite/scripts/render_dataset_with_omniverse.py \
#     --dataset path/to/demo.hdf5 \
#     --rgb \
#     --cameras agentview robot0_eye_in_hand \
#     --save_video

# Render specific episodes with semantic segmentation
# python robosuite/scripts/render_dataset_with_omniverse.py \
#     --dataset path/to/demo.hdf5 \
#     --episode 0 1 2 \
#     --semantic_segmentation \
#     --renderer PathTracing

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment