Principle:Haosulab ManiSkill Scene Construction

Knowledge Sources	Haosulab_ManiSkill ManiSkill Docs
Domains	Robotics, Simulation, Procedural_Generation
Last Updated	2026-02-15 08:00 GMT

Overview

Scene construction is the process of procedurally assembling reusable simulation environments -- tables, kitchens, apartments, and other structured spaces -- from asset datasets and layout specifications, enabling diverse and scalable task settings without manual scene authoring.

Description

Robotics simulation tasks take place in environments that range from simple tabletops to fully furnished kitchens and multi-room apartments. The Scene Construction principle provides a framework for building these environments programmatically through scene builder classes. Each scene builder encapsulates the logic for loading assets from a specific dataset (ReplicaCAD, RoboCasa, AI2THOR/ProcTHOR), placing them according to layout rules, configuring articulated objects (cabinets with openable doors and drawers, appliances with moving parts), and setting up lighting and ground planes.

This principle solves several problems. First, it enables environment diversity: by parameterizing scene construction, the same builder can produce many different scene configurations, supporting domain randomization for sim-to-real transfer. Second, it provides reusability: the same kitchen scene builder can be used across multiple task definitions (e.g., opening cabinets, fetching objects, rearranging items). Third, it handles the complexity of loading heterogeneous asset formats (MJCF for RoboCasa fixtures, GLTF/GLB for ReplicaCAD objects, USD for AI2THOR scenes) and converting them into the simulation engine's internal representation.

Scene builders follow a two-phase lifecycle: an initialization phase where the builder reads metadata and layout specifications, and a build phase where it creates actors and articulations in the simulation scene. The build phase can produce different configurations on each episode reset, supporting procedural environment variation.

Usage

This principle applies whenever:

• A task requires a structured environment beyond a simple flat surface (kitchens, rooms, furnished spaces).
• Environment diversity is needed for training robust policies that generalize across scene layouts.
• Assets from external datasets (ReplicaCAD, RoboCasa, AI2THOR) must be loaded and composed into coherent scenes.
• Articulated environment objects (cabinets, drawers, appliances) must be placed and configured with correct joint properties.
• Multiple tasks need to share the same scene configuration without duplicating scene-building code.

Theoretical Basis

1. Scene Builder Abstraction: Each scene builder is a class that implements a standard interface: build() to create the scene geometry and initialize() to set the initial configuration of dynamic elements. Builders are registered by name so that environments can reference them declaratively.

2. Asset Dataset Integration: Scene builders read from curated asset datasets that provide 3D models, metadata (bounding boxes, joint configurations, semantic labels), and layout specifications. The builder translates dataset-specific formats into the simulation engine's actor and articulation primitives.

3. Layout Specification: Scenes are defined by spatial layouts that specify where fixtures and objects are placed. Layouts can be:

• Fixed: Exact positions and orientations from a pre-authored configuration file (e.g., ReplicaCAD scene configs).
• Procedural: Generated algorithmically from layout grammars or sampling rules (e.g., ProcTHOR room layouts).
• Template-based: Defined by fixture slots (counter positions, cabinet mounting points) that are filled from a catalogue (e.g., RoboCasa kitchen styles).

4. Fixture Composition: Complex scene elements are built by composing simpler fixtures. A kitchen counter is composed of a base cabinet, countertop surface, and optional accessories. A fixture stack layers multiple fixtures vertically. This compositional approach mirrors the physical modularity of real environments.

5. Placement Sampling: Objects within scenes are placed using constrained samplers that respect spatial bounds, non-overlap constraints, and surface attachment rules. Placement samplers provide the randomization needed for diverse episode initialization while ensuring physically plausible configurations.

6. Multi-Scene Batching: In GPU-parallelized simulation, different environment instances can run different scene configurations simultaneously. Scene builders support this by managing per-environment asset indices and configuration parameters.

Related Pages

• Implementation:Haosulab_ManiSkill_ReplicaCADSceneBuilder -- Scene builder for ReplicaCAD photorealistic apartment environments.
• Implementation:Haosulab_ManiSkill_ReplicaCADRearrangeSceneBuilder -- Scene builder for ReplicaCAD rearrangement task variants.
• Implementation:Haosulab_ManiSkill_RoboCasaSceneBuilder -- Scene builder for RoboCasa procedural kitchen environments.
• Implementation:Haosulab_ManiSkill_AI2THORSceneBuilder -- Scene builder for AI2THOR/ProcTHOR indoor environments.
• Implementation:Haosulab_ManiSkill_ActorBuilder_TableSceneBuilder -- Table scene builder for tabletop tasks.
• Implementation:Haosulab_ManiSkill_RoboCasaFixture -- Base fixture class for RoboCasa kitchen elements.
• Implementation:Haosulab_ManiSkill_RoboCasaCabinet -- RoboCasa cabinet fixture with doors and drawers.
• Implementation:Haosulab_ManiSkill_RoboCasaCabinetPanels -- RoboCasa cabinet panel decorative elements.
• Implementation:Haosulab_ManiSkill_RoboCasaCounter -- RoboCasa kitchen counter fixture.
• Implementation:Haosulab_ManiSkill_RoboCasaStove -- RoboCasa stove, stovetop, and oven fixtures.
• Implementation:Haosulab_ManiSkill_RoboCasaSink -- RoboCasa sink fixture with faucet control.
• Implementation:Haosulab_ManiSkill_RoboCasaMicrowave -- RoboCasa microwave fixture with door state.
• Implementation:Haosulab_ManiSkill_RoboCasaHandles -- RoboCasa cabinet and appliance handle fixtures.
• Implementation:Haosulab_ManiSkill_RoboCasaWindows -- RoboCasa window fixtures.
• Implementation:Haosulab_ManiSkill_RoboCasaAccessories -- RoboCasa kitchen accessory and decoration fixtures.
• Implementation:Haosulab_ManiSkill_RoboCasaStructuralFixtures -- RoboCasa structural fixtures (walls, floors, boxes).
• Implementation:Haosulab_ManiSkill_FixtureStack -- Vertical fixture stacking composition class.
• Implementation:Haosulab_ManiSkill_MujocoObject -- Base class bridging MJCF objects to the simulation engine.
• Implementation:Haosulab_ManiSkill_KitchenObjects -- RoboCasa kitchen object catalogue.
• Implementation:Haosulab_ManiSkill_KitchenObjectUtils -- Kitchen object utility functions.
• Implementation:Haosulab_ManiSkill_PlacementSamplers -- Constrained placement sampling for object positioning.
• Implementation:Haosulab_ManiSkill_ProcTHORMeta -- ProcTHOR scene metadata and layout definitions.
• Implementation:Haosulab_ManiSkill_PartNetChairMeta -- PartNet-Mobility chair asset metadata.
• Implementation:Haosulab_ManiSkill_PartNetFaucetMeta -- PartNet-Mobility faucet asset metadata.