Implementation:Facebookresearch Habitat lab Benchmark init

Knowledge Sources	Habitat-Lab
Domains	Evaluation, Software_Architecture
Last Updated	2026-02-15 02:00 GMT

Overview

Concrete Benchmark class constructor that creates a Habitat evaluation environment from a config path, provided by habitat-lab core.

Description

The Benchmark.__init__ method loads configuration from the provided path, creates a Habitat `Env` with the specified task and dataset, and prepares for evaluation. It also supports remote evaluation via the `eval_remote` flag for Habitat Challenge submissions.

Usage

Create a Benchmark instance with the path to your evaluation config YAML. Then call `evaluate()` with your agent.

Code Reference

Source Location

• Repository: habitat-lab
• File: habitat-lab/habitat/core/benchmark.py
• Lines: L27-42 (__init__)

Signature

class Benchmark:
    def __init__(
        self,
        config_paths: Optional[str] = None,
        eval_remote: bool = False,
    ) -> None:
        """
        Args:
            config_paths: Path to evaluation config YAML
            eval_remote: Use remote evaluation API (for Habitat Challenge)
        """

Import

from habitat.core.benchmark import Benchmark

I/O Contract

Inputs

Name	Type	Required	Description
config_paths	Optional[str]	No	Path to benchmark config YAML
eval_remote	bool	No	Use remote evaluation (default False)

Outputs

Name	Type	Description
benchmark	Benchmark	Initialized benchmark with internal Env ready for evaluation

Usage Examples

Create Benchmark

from habitat.core.benchmark import Benchmark

benchmark = Benchmark(
    config_paths="habitat/config/benchmark/nav/pointnav/pointnav_habitat_test.yaml"
)

Related Pages

Implements Principle

• Principle:Facebookresearch_Habitat_lab_Benchmark_Configuration

Requires Environment

• Environment:Facebookresearch_Habitat_lab_Python_3_9_Core_Dependencies