Implementation:Open compass VLMEvalKit MMHelix Graph Problems Eval
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, Puzzle Solving, Graph Theory |
Overview
Implements evaluators for graph theory problems including Hamiltonian path verification in the MMHelix benchmark.
Description
This module provides the HamiltonianPathEvaluator and related graph problem evaluators. The safe_parse_answer function safely parses answer strings from JSON or Python literal formats, handling "No" responses for unsolvable instances. The HamiltonianPathEvaluator verifies whether a predicted path is a valid Hamiltonian path by checking that it visits every node exactly once and that consecutive nodes in the path are connected by edges in the adjacency list representation. It supports both string and list input formats for predicted answers and graph representations.
Usage
Called internally by the MMHelix dataset class during graph problem evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/mmhelix/evaluators/graph_problems_eval.py, Lines: L1-990 - Import:
from vlmeval.dataset.utils.mmhelix.evaluators.graph_problems_eval import HamiltonianPathEvaluator, safe_parse_answer
Key Functions:
def safe_parse_answer(answer_str, verbose=False): ...
class HamiltonianPathEvaluator:
def __init__(self, verbose=False): ...
def evaluate(self, predicted_answer, ground_truth, initial_state): ...
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Predicted path as a list of node indices or "No"; ground-truth answer; graph adjacency list as a dictionary |
| Outputs | Boolean indicating whether the predicted path is a valid Hamiltonian path (or correctly identifies no path exists) |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.mmhelix.evaluators.graph_problems_eval import HamiltonianPathEvaluator
evaluator = HamiltonianPathEvaluator()
is_correct = evaluator.evaluate([4, 5, 3, 1, 2, 0], ground_truth, adjacency_list)