Principle:ARISE Initiative Robosuite USD Scene Export
| Knowledge Sources | |
|---|---|
| Domains | Robotics, Computer_Graphics |
| Last Updated | 2026-02-15 07:00 GMT |
Overview
A pipeline for converting physics simulation scenes into the Universal Scene Description format, enabling photorealistic re-rendering of simulated trajectories in external renderers.
Description
Physics simulators produce visually adequate but not photorealistic renderings. For applications such as dataset generation, synthetic training data, and publication-quality visualizations, it is desirable to re-render simulation trajectories using advanced rendering engines that support ray tracing, global illumination, and physically-based materials. Universal Scene Description (USD) is an interchange format developed by Pixar that serves as a common language between simulation and rendering tools.
The export pipeline operates by iterating over the simulation's visual scene graph at each timestep, converting each geometric primitive (mesh, box, sphere, cylinder, capsule, plane) and its associated transform, material, and texture into the corresponding USD representation. The exporter maintains an in-memory USD stage onto which it creates and updates USD primitives (Xforms, Meshes) frame by frame. Keyframed transforms allow the resulting USD file to encode an entire trajectory as a time-varying scene that external renderers can play back.
The pipeline handles several categories of scene elements. Predefined meshes (loaded from the simulator's mesh assets) are exported with their vertex, face, and UV coordinate data. Primitive shapes (boxes, spheres, cylinders, capsules, ellipsoids, planes) are procedurally tessellated into triangle meshes with appropriate UV mappings. Tendons (visual connectors between bodies) receive special treatment, being decomposed into cylinder-plus-hemisphere components with per-part scaling. Lights and cameras are exported with their positions and parameters, enabling faithful reproduction of the viewpoint and illumination in the external renderer.
Texture handling involves extracting raw pixel data from the simulator's texture buffers, saving them as PNG images, and referencing them through USD material shader graphs that support both image-based and solid-color materials with configurable metallic and roughness properties.
Usage
Use scene export when simulation trajectories need to be visualized at higher fidelity than the built-in renderer provides, when generating photorealistic training data for vision-based policies, or when producing publication-quality videos. The export pipeline is invoked after the simulation has been run (or during live simulation in online mode), producing USD files that can be opened in any USD-compatible renderer such as NVIDIA Omniverse, Blender, or Houdini.
Theoretical Basis
Scene graph translation:
The exporter performs a mapping between the simulation's scene representation and USD's hierarchical scene graph:
Simulation Scene Graph USD Scene Graph
------------------- ---------------
Geom (type, mesh_id, ---> /World/Mesh_Xform_{name}/Mesh_{name}
pos, mat, rgba, - UsdGeom.Mesh (vertices, faces, UVs)
texid) - UsdShade.Material (texture or solid)
- TransformOp (4x4 matrix, keyframed)
Light (pos, diffuse, ---> /World/Light_{id}
dir) - UsdLux.SphereLight (pos, intensity, color)
Camera (pos, forward, ---> /World/Camera_Xform_{name}/Camera_{name}
up) - UsdGeom.Camera (pos, orientation)
Primitive tessellation:
Each geometric primitive type is tessellated into a triangle mesh:
- Box: 12 triangles (2 per face)
- Sphere: Parametric tessellation with configurable resolution, using spherical coordinates
(theta, phi) - Cylinder: End caps (fan triangulation) plus side walls (quad strips split into triangles)
- Capsule: Cylinder body plus two hemisphere caps
- Ellipsoid: Unit sphere scaled by
(sx, sy, sz)
UV mapping:
UV coordinates are computed based on the texture type:
- 2D textures: Planar projection using the
(x, y)vertex coordinates, scaled by the texture repeat and geometry size parameters - Cube map textures: Dominant-axis cube map projection, mapping each vertex to the appropriate face of the cube based on its position relative to the mesh center
Material shader graph:
Each material is represented as a USD Preview Surface shader:
Material
|-- Principled_BSDF (diffuseColor, opacity, metallic, roughness)
|-- Image_Texture (file, wrapS=repeat, wrapT=repeat)
|-- UVMap_Reader (varname="UVMap")