Implementation:Open compass VLMEvalKit Spatial457

Field	Value
source	VLMEvalKit
domain	Vision, Benchmarking, Spatial Reasoning

Overview

Benchmark dataset implementation for Spatial457 evaluation in VLMEvalKit.

Description

Spatial457 inherits from ImageBaseDataset and implements the Spatial457 benchmark for spatial reasoning evaluation. The TYPE field is set to 'VQA'. It includes a ROBUST mode where non-conforming responses are treated as full answers.

Usage

Registered in vlmeval/dataset/__init__.py and invoked through build_dataset() by benchmark name.

Code Reference

Source: vlmeval/dataset/spatial457.py, Lines: L1-212
Import: from vlmeval.dataset.spatial457 import Spatial457

Signature:

class Spatial457(ImageBaseDataset):
    TYPE = 'VQA'
    DATASET_URL = {...}
    DATASET_MD5 = {...}
    ...

I/O Contract

Direction	Description
Inputs	TSV dataset file with image/video paths and questions
Outputs	Evaluation results DataFrame with scores per category

Usage Examples

from vlmeval.dataset import build_dataset
dataset = build_dataset('Spatial457')

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment