Implementation:Facebookresearch Habitat lab PacmanTrainer

Knowledge Sources	Facebookresearch_Habitat_lab
Domains	Embodied_AI, Embodied_Question_Answering, Navigation
Last Updated	2026-02-15 00:00 GMT

Overview

PACMANTrainer is a trainer class for the Planner and Controller Module (PACMAN) navigation model used in Embodied Question Answering, implementing hierarchical imitation learning with separate planner and controller losses.

Description

The PACMANTrainer class extends BaseILTrainer and is registered under the name "pacman" in the baseline registry. It trains the NavPlannerControllerModel, which uses a hierarchical approach to navigation in embodied environments:

• Planner: An RNN-based module that takes image features, question embeddings, and previous actions to produce high-level navigation actions (forward, left, right, stop).
• Controller: An MLP-based module that decides whether to continue the current planner action or request a new one, using the planner's hidden state along with current visual and action features.

Training: The trainer loads navigation data via NavDataset (which requires a live Habitat environment), creates a DataLoader, and trains both the planner and controller simultaneously. The training loop sorts batches by planner action lengths for efficient packed sequence processing, computes masked NLL losses for both modules, and backpropagates the combined loss. Metrics (planner_loss, controller_loss) are tracked via NavMetric and logged to TensorBoard.

Evaluation: The evaluation procedure is significantly more complex. For each episode, the trainer runs the model in an interactive loop within the Habitat simulator at multiple spawn distances (10, 30, 50 steps, and random initialization). It evaluates both the full model prediction and a forward-only baseline. The evaluation collects distance-to-target metrics (d_T, d_D, d_min), stop accuracy, and episode length. Results can be optionally saved as video.

Usage

Use this trainer to train and evaluate the PACMAN navigation model as part of the EQA pipeline. The model requires a pre-trained MultitaskCNN checkpoint for visual feature extraction. Training requires the EQA-v0 dataset and a running Habitat simulator environment.

Code Reference

Source Location

• Repository: Facebookresearch_Habitat_lab
• File: habitat-baselines/habitat_baselines/il/trainers/pacman_trainer.py
• Lines: 1-674

Signature

@baseline_registry.register_trainer(name="pacman")
class PACMANTrainer(BaseILTrainer):
    supported_tasks = ["EQA-v0"]

    def __init__(self, config=None): ...
    def _save_nav_results(
        self,
        ckpt_path: str,
        ep_id: int,
        questions: torch.Tensor,
        imgs: List[np.ndarray],
        q_vocab_dict: VocabDict,
        results_dir: str,
        writer: TensorboardWriter,
        video_option: list,
    ) -> None: ...
    def train(self) -> None: ...
    def _eval_checkpoint(
        self,
        checkpoint_path: str,
        writer: TensorboardWriter,
        checkpoint_index: int = 0,
    ) -> None: ...

Import

from habitat_baselines.il.trainers.pacman_trainer import PACMANTrainer

I/O Contract

Inputs

Name	Type	Required	Description
config	DictConfig	Yes	Full Habitat baselines configuration containing dataset, environment, navigation training parameters (batch_size, lr, max_epochs, max_controller_actions, max_episode_length), and logging settings

Outputs

Name	Type	Description
checkpoint files	file	Model state dict saved as epoch_{N}.ckpt at configured checkpoint intervals
TensorBoard logs	file	Planner loss and controller loss metrics logged during training
navigation videos	file	Optional video results saved during evaluation showing the agent's trajectory with the question text overlay
evaluation metrics	dict	Distance-to-target metrics (d_T, d_D, d_min), stop accuracy, and episode lengths at various spawn distances

Usage Examples

Basic Usage

from habitat_baselines.common.baseline_registry import baseline_registry

# The trainer is registered and typically invoked via the run script:
# python -m habitat_baselines.run --config-name=eqa/il_pacman_nav.yaml

# Programmatic usage:
config = ...  # load Hydra config
trainer_cls = baseline_registry.get_trainer("pacman")
trainer = trainer_cls(config=config)

# Training
trainer.train()

# Evaluation (called internally by eval loop)
# trainer.eval()