Implementation:Open compass VLMEvalKit MMHelix Constants
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Puzzle Solving, Configuration |
Overview
Defines prompt templates and constants for all puzzle types in the MMHelix benchmark evaluation framework.
Description
This module provides prompt template constants used by MMHelix puzzle evaluators to guide model responses. It includes PROMPT_SOKOBAN_IMAGE for Sokoban box-pushing puzzles with direction definitions and image-based state representation, PROMPT_MAZE_IMAGE for maze navigation with cell-based movement rules, and additional prompt templates for other puzzle types. Each prompt defines game rules, visual representation conventions (colors for players, boxes, targets, walls), direction definitions, coordinate systems, and expected output formats including think/answer tag structures.
Usage
Called internally by the MMHelix evaluators when constructing prompts for various puzzle types.
Code Reference
- Source:
vlmeval/dataset/utils/mmhelix/utils/constants.py, Lines: L1-392 - Import:
from vlmeval.dataset.utils.mmhelix.utils.constants import PROMPT_SOKOBAN_IMAGE, PROMPT_MAZE_IMAGE
Key Functions:
PROMPT_SOKOBAN_IMAGE = """...""" # Sokoban puzzle prompt with rules and visual conventions
PROMPT_MAZE_IMAGE = """...""" # Maze navigation prompt with direction definitions
PROMPT_EULERO = """...""" # Eulero puzzle prompt template
I/O Contract
| Direction | Description |
|---|---|
| Inputs | None (constant definitions); puzzle-specific parameters are interpolated by evaluators |
| Outputs | String constants containing formatted prompt templates for each puzzle type |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.mmhelix.utils.constants import PROMPT_SOKOBAN_IMAGE
prompt = PROMPT_SOKOBAN_IMAGE # Use as template for Sokoban puzzle evaluation